Update -
Security Update Thursday 20 April 2023 – Initial Intrusion Vector Found
https://www.3cx.com/blog/news/mandiant-security-update2/
Posted on April 20th, 2023 by Agathocles Prodromou, CNO, 3CX
Mandiant identifies the source of internal network compromise
While Mandiant’s investigation is still ongoing, we now have a clear overall understanding of the attack. Following our previous update, we would like to share some additional technical details to support our customers and the community. We have also published additional indicators of compromise that organizations can leverage for their network defenses.
Initial Intrusion Vector
Mandiant identified the source of our internal network compromise began in 2022 when an employee installed the Trading Technologies X_TRADER software on the employee’s personal computer. Although the X_TRADER installation software was downloaded from the Trading Technologies website, it contained VEILEDSIGNAL malware, which enabled the threat actor (identified as UNC4736) to initially compromise and maintain persistence on the employee’s personal computer.
The X_TRADER installer (X_TRADER_r7.17.90p608.exe) was digitally signed by a valid code signing certificate with the subject of “Trading Technologies International, Inc”. It was hosted on hxxps://download.tradingtechnologies[.]com. While the X_TRADER software was reportedly retired in 2020 by Trading Technologies, the software was still available for download on the Trading Technologies website in 2022. The code signing certificate used to digitally sign the malicious software was set to expire in October 2022.
For more technical detail on the X_TRADER software supply chain attack, including YARA Rules for hunting, please read Mandiant’s blog at https://www.mandiant.com/resources/blog/3cx-software-supply-chain-compromise.
Lateral Movement
Following the initial compromise of the employee’s personal computer using VEILEDSIGNAL malware, Mandiant assesses the threat actor stole the employee's 3CX corporate credentials from his system. VEILEDSIGNAL is a fully-featured malware that provided the threat actor with administrator-level access and persistence to the compromised system. The earliest evidence of compromise uncovered within the 3CX corporate environment occurred through the VPN using the employee's corporate credentials two days after the employee's personal computer was compromised.
Additionally, Mandiant identified the use of the Fast Reverse Proxy tool (https://github.com/fatedier/frp) which the threat actor used to move laterally within the 3CX environment. The tool was named MsMpEng.exe and located in the C:\Windows\System32 directory.
CI/CD Build Environment Compromise
Mandiant’s investigation was able to reconstruct the threat actor’s steps through our environment as they harvested credentials and moved laterally. Eventually, the threat actor was able to compromise both the Windows and macOS build environments. On the Windows build environment, the attacker deployed the TAXHAUL launcher and COLDCAT downloader which persisted by performing DLL hijacking for the IKEEXT service and ran with LocalSystem privileges. The macOS build server was compromised using a POOLRAT backdoor using LaunchDaemons as a persistence mechanism.
Attribution
Based on the Mandiant investigation into the 3CX intrusion and supply chain attack thus far, they attribute the activity to a threat actor cluster named UNC4736. Mandiant assesses with high confidence that UNC4736 has a North Korean nexus.
Indicators of Compromise
X_TRADER_r7.17.90p608.exe
SHA256: fbc50755913de619fb830fb95882e9703dbfda67dbd0f75bc17eadc9eda61370
SHA1: ced671856bbaef2f1878a2469fb44e9be8c20055
MD5: ef4ab22e565684424b4142b1294f1f4d
Setup.exe
SHA256: 6e11c02485ddd5a3798bf0f77206f2be37487ba04d3119e2d5ce12501178b378
SHA1: 3bda9ca504146ad5558939de9fece0700f57c1c0
MD5: 00a43d64f9b5187a1e1f922b99b09b77
Code signing certificate serial #
9599605970805149948
MsMpEng.exe
SHA256: 24d5dd3006c63d0f46fb33cbc1f576325d4e7e03e3201ff4a3c1ffa604f1b74a
SHA1: d7ba13662fbfb254acaad7ae10ad51e0bd631933
MD5: 19dbffec4e359a198daf4ffca1ab9165
Command and Control
Mandiant identified that malware within the 3CX environment made use of the following additional command and control infrastructure.
www.tradingtechnologies[.]com/trading/order-management
Going Forward
Our priority throughout this incident has been transparency around what we know as well as the actions we’ve taken.
As we wind down our incident investigation, 3CX has taken this opportunity to continue to strengthen our policies, practices, and technology to further protect against future attacks. With that, we’re announcing a 7 Step Security Action Plan. In this plan, we’re committing to actionable steps to harden our defenses. You can read in more detail here.
Apr 20, 2023 - 13:03 PDT
Update -
Security Update Mandiant Initial Results
Posted on April 11th, 2023 by Pierre Jourdan, CISO, 3CX - https://www.3cx.com/blog/news/mandiant-initial-results/?fbclid=IwAR3Ix_kwrA-bfHYexmRp01UBq466PUdxdHD4QTotfgK1U7xHek6NP3YNzwU
Initial Results from Mandiant Incident Response
Following the appointment of Mandiant as our security incident response team, forensic analysis on our network and product is in progress. In a nutshell, the interim assessment concluded:
Attribution
Based on the Mandiant investigation into the 3CX intrusion and supply chain attack thus far, they attribute the activity to a cluster named UNC4736. Mandiant assesses with high confidence that UNC4736 has a North Korean nexus.
Windows-based Malware
Mandiant determined that the attacker infected targeted 3CX systems with TAXHAUL (AKA “TxRLoader”) malware. When executed on Windows systems, TAXHAUL decrypts and executes shellcode located in a file named .TxR.0.regtrans-ms located in the directory C:\Windows\System32\config\TxR\. The attacker likely chose this file name and location to attempt to blend into standard Windows installations. The malware uses the Windows CryptUnprotectData API to decrypt the shellcode with a cryptographic key that is unique to each compromised host, which means the data can only be decrypted on the infected system. The attacker likely made this design decision to increase the cost and effort of successful analysis by security researchers and incident responders.
In this case, after decrypting and loading the shellcode contained within the file .TxR.0.regtrans-ms was a complex downloader which Mandiant named COLDCAT. It is worth noting, however, this malware differs from GOPURAM referenced in Kaspersky’s report.
The following YARA rule can be used to hunt for TAXHAUL (TxRLoader):
rule TAXHAUL
{
meta:
author = "Mandiant"
created = "04/03/2023"
modified = "04/03/2023"
version = "1.0"
strings:
$p00_0 = {410f45fe4c8d3d[4]eb??4533f64c8d3d[4]eb??4533f64c8d3d[4]eb}
$p00_1 = {4d3926488b01400f94c6ff90[4]41b9[4]eb??8bde4885c074}
condition:
uint16(0) == 0x5A4D and any of them
}
Please note that in a similar way to any YARA rule, this should be properly assessed within a test environment first before usage in production. This also comes with no guarantees regarding false positive rates, as well as coverage for this entire malware family and eventual variants.
MacOS-based Malware
Mandiant also identified a MacOS backdoor, currently named SIMPLESEA, located at /Library/Graphics/Quartz (MD5: d9d19abffc2c7dac11a16745f4aea44f). Mandiant is still analysing SIMPLESEA to determine if it overlaps with another known malware family.
The backdoor written in C communicates via HTTP. Supported backdoor commands include shell command execution, file transfer, file execution, file management, and configuration updating. It can also be tasked to test the connectivity of a provided IP and port number.
The backdoor checks for the existence of its configuration file at /private/etc/apdl.cf. If it does not exist, it creates it with hard-coded values. The config file is single-byte XOR encoded with the key 0x5e. C2 comms are sent over HTTP requests. A bot id is generated randomly seeded with the PID of the malware upon initial execution. The id is sent with C2 communications. A brief host survey report is included in beacon requests. Message contents are encrypted with the A5 stream cipher according to the function names in the binary.
Persistence
On Windows, the attacker used DLL side-loading to achieve persistence for TAXHAUL malware. DLL side-loading triggered infected systems to execute the attacker's malware within the context of legitimate Microsoft Windows binaries, reducing the likelihood of malware detection. The persistence mechanism also ensures the attacker malware is loaded at system start-up, enabling the attacker to retain remote access to the infected system over the internet.
The malware was named C:\Windows\system32\wlbsctrl.dll to mimic the legitimate Windows binary of the same name. The DLL was loaded by the legitimate Windows service IKEEXT through the legitimate Windows binary svchost.exe.
Command and Control
Mandiant identified that malware within the 3CX environment made use of the following command and control infrastructure:
azureonlinecloud[.]com
akamaicontainer[.]com
journalide[.]org
msboxonline[.]com
Apr 11, 2023 - 10:16 PDT
Update -
New Desktop App Build Number 18.12.425 Released
Posted on April 6th, 2023 by Nick Galea, CEO, 3CX - https://www.3cx.com/blog/releases/v18u7-desktop-app-electron/
Update 7 Windows Electron App build number 18.12.424 has been checked by our advisors Mandiant who found no evidence of compromise. Due to certificate changes, we had to update the naming convention. We re-opened the build to address this. Build 18.12.425 has been created and released. Below is our most up to date information.
Still Preferred Option: Install 3CX as a Native Web App (PWA)
Installing 3CX as a native app (PWA) is still our preferred and recommended option. To summarise:
There’s no need to be logged into 3CX with a tab open
When a call or chat comes in you’ll be notified with a PUSH notification
Depending upon your browser, this is what you can expect:
Microsoft Edge, your browser will be auto started
Google Chrome, your browser will NOT be auto started if closed
Safe and practical use it together with iOS and Android apps or deskphone to control calls on your smartphone or deskphone
Setting the App to Auto Start
To ensure zero admin, along with automatic updates and secure running within the browser, we recommend auto starting your browser on system startup. Read the instructions on how to set the app to auto start.
How to Download Windows & Mac Electron
Ensure Your Server Has the Latest Update Installed
Important! First, ensure your server has the latest update installed. Details about this and the steps, if any, that you need to take are shown below.
Customers on 3CX Hosted / StartUP - No action needed
3CX Hosted and StartUP users do not need to update their servers as we will be updating them automatically today. Servers will be restarted and the new Electron App MSI/DMG will be installed on the server. We still recommend that you use the PWA App.
Apr 06, 2023 - 08:34 PDT
Update -
We are continuing to monitor for any further issues.
You can keep up to date with all the latest information from 3CX here - https://www.3cx.com/blog/
Apr 04, 2023 - 09:01 PDT
Update -
We are continuing to monitor for any further issues.
You can keep up to date with all the latest information from 3CX here - https://www.3cx.com/blog/
Apr 03, 2023 - 09:10 PDT
Update - We are continuing to monitor for any further issues.
Apr 03, 2023 - 08:35 PDT
Update -
We are continuing to monitor for further issues, in the interim useful guides to remove the problematic Electron Apps is as below
Uninstalling the Desktop App - https://www.3cx.com/blog/news/uninstalling-the-desktop-app/
Posted on April 1st, 2023 by Pierre Jourdan, CISO, 3CX
The Desktop App can be uninstalled as explained below. On some Windows machines where antivirus software already deleted some of the files the uninstaller may fail.
On Windows:
Start
Type “Control Panel”, Enter
Select “Programs and Features”
Find 3CX Desktop App, select and press “Uninstall”.
On Mac:
Go to “Applications”
Tap on “3CX Desktop APP”
Right click then “Move to Bin”
Ensure that it isn’t also present on Desktop otherwise delete it from there as well.
Empty the Bin
Mass / Network Uninstall of Electron App
Partners on our forums have kindly contributed Powershell scripts that allow companies to mass uninstall the electron app from their Network. We have merged them into one that will attempt to uninstall and forcibly delete any remaining files and entries associated with the Desktop App. We hereby thank the original authors of the scripts that were merged. This powershell script hasn't been thoroughly tested yet from our end, we recommend testing it on one machine first before executing it on your customers infrastructure. This must be run on client machines not the server.
Important: There are many scripts being suggested on the internet. Please be careful with any script or executable found on the internet, do not blindly trust them as they may be harmful.
# Kill 3CX processes first
Get-process | Where-Object {$_.name -Like "*3CX*"} | stop-process
# Attempt #1 - via EXE uninstall method
$3cxapps = Get-WMIObject -Class Win32_product | where {$_.name -like "3CX Desktop APP"}
foreach ($app in $3cxapps) {
try {
$app.Uninstall()
Remove-Item C:\Users\$env:UserName\AppData\Roaming\3CXDesktopApp -Recurse
Remove-Item C:\Users\$env:UserName\AppData\Local\Programs\3CXDesktopApp -Recurse
Remove-Item C:\Users\$env:UserName\Desktop\3CX Desktop App.lnk -Recurse
Write-Host "Uninstalled $($app.Name)"
}
catch {
Write-Host "Error uninstalling $($app.Name): $($_.Exception.Message)"
}
}
# Attempt #2 - via MSIEXEC ~ Requires Set-ExecutionPolicy to be changed
$appInstalled = Get-WmiObject -Class Win32_Product | Where-Object {$_.Name -eq "3CX Desktop App" }
if ($appInstalled) {
try {
$uninstallString = $appInstalled.UninstallString
Start-Process msiexec.exe -ArgumentList "/x `"$uninstallString`" /qn" -Wait -NoNewWindow
Remove-Item C:\Users\$env:UserName\AppData\Roaming\3CXDesktopApp -Recurse
Remove-Item C:\Users\$env:UserName\AppData\Local\Programs\3CXDesktopApp -Recurse
Remove-Item C:\Users\$env:UserName\Desktop\3CX Desktop App.lnk -Recurse
Write-Host "Uninstalled $($appName)"
}
catch {
Write-Host "Error uninstalling $($appName): $($_.Exception.Message)"
}
}
else {
Write-Host "$appName is not installed"
}
Apr 02, 2023 - 08:19 PDT
Update -
Continuing to monitor, latest recommendations are to use the PWA application until newly signed Windows and MacOS Desktop Applications are made available, see link below for more details on the current secure options
https://www.3cx.com/blog/news/pwa-vs-windows-legacy-app/
Mar 31, 2023 - 13:33 PDT
Update - Chrome blocks latest 3CX MSI installer
Posted on March 31st, 2023 by Nick Galea, CEO, CTO & Founder, 3CX
Unfortunately Google has invalidated our software security certificate. This means the MSI DesktopApp files that we released yesterday afternoon can no longer be downloaded via Google Chrome (nor can the originally infected MSI files). Furthermore several AV vendors are blocking any software signed with the old security certificate.
We are now going to make new MSI installers with a new certificate. This will take at least 8 hours, as we have decided to build a completely new build server. The following MSI installers have been blocked:
SBC for Windows
Windows Desktop APP
Call Flow Designer
We repeat that we recommend using our PWA app. The PWA app is completely web based and does 95% of what the Electron App does. The advantage is that it does not require any installation or updating and chrome web security is applied automatically.
The MAC desktop application will not be rebuilt for the time being as we focus on the Windows app as well as the actual security breach.
Mar 31, 2023 - 08:31 PDT
Update - We are continuing to monitor for any further issues.
Mar 31, 2023 - 08:20 PDT
Update - We are continuing to monitor for any further issues.
Mar 30, 2023 - 16:38 PDT
Update - We are continuing to monitor for any further issues.
Mar 30, 2023 - 14:53 PDT
Update -
3CX DesktopApp Security Alert - Mandiant Appointed to Investigate
https://www.3cx.com/blog/news/desktopapp-security-alert-updates/
Posted on March 30th, 2023 by Nick Galea, CEO, CTO & Founder, 3CX
Early this morning we informed our partners and customers that our electron windows app shipped in Update 7, version numbers 18.12.407 & 18.12.416, included a severe security issue. We since learned that Electron Mac App version numbers 18.11.1213, 18.12.402, 18.12.407 & 18.12.416 have also been affected. Fortunately, anti-virus vendors flagged the executable 3CXDesktopApp.exe and blocked it.
3CX Appoints Leading Incident & Forensics Company Mandiant
In response to this, 3CX has appointed Mandiant a renowned American cybersecurity firm and subsidiary of Google - and the market leader in threat intelligence. With their help we will be able to review this incident in full. Whilst their investigation is underway, we ask you to follow the instructions below immediately.
Ensure Your Server Has the Latest Update Installed
Customers on 3CX Hosted / StartUP - No action needed
3CX Hosted and StartUP users do not need to update their servers as we will be updating them over the night automatically. Servers will be restarted and the new Electron App MSI/DMG will be installed on the server. We recommend that you DO NOT install or deploy the Electron App. This update is only to ensure that the trojan has been removed from the 3CX Server where Desktop Apps are stored and in case any users decide to deploy the app anyway. During the restart there might be disruption for a few minutes while we restart your server.
Self-Hosted and On-Premise - Install update
For Self-Hosted and On-Premise follow these steps:
Launch Management Console
Go to Updates
Download Mac Desktop App - 18.12.422
Download Windows Desktop App - 18.12.422
On the Clients / Desktops
Uninstall the Electron App
Follow these steps to uninstall the Electron App for Mac or Windows
For Windows:
Start
Type “Control Panel”, Enter
Select “Programs and Features”
Find 3CX Desktop App, select and press “Uninstall”.
On Mac:
Go to “Applications”
Tap on “3CX Desktop APP”
Right click then “Move to Bin”
Ensure that it isn’t also present on Desktop otherwise delete it from there as well.
Empty the Bin
Use PWA instead of the Electron APP - Here's how
Install the Web Client as an app (PWA)
Login to the Web Client
You have two options:
Click on the OS icon below the user avatar. A new dialog will open, select “Web App (PWA)” and then hit the “Install” button.
OR click on the “Install button” (A screen with an arrow) located in the address bar and confirm. See the icon circled red in the screenshot.
To set the app to auto start:
On Google Chrome: Open your Chrome browser and type ‘chrome://apps’ into the address bar. Right click on “3CX” and enable “Start app when you sign in”.
On Microsoft Edge: On Edge, select to Auto-start in the dialog that appears after installation.
PWA only works on Google Chrome and Microsoft Edge - not on Safari or Firefox
You can read more in the Web Client user manual.
Avoid Using the Electron App Unless Absolutely Essential
In a day or two from now, we will have another Electron App rebuilt from the ground up with a new signed certificate. This is expected to be completely secure. We strongly recommend that you avoid using the Electron App unless there is absolutely no alternative. The Electron App update that we are releasing today is considered to be secure but there is no guarantee given that we only had 24 hours to make the necessary adjustments.
More Information to Come - Transparency Assured
We are still working to decipher the full extent of the attack and we promise full transparency as soon as we are clear on everything. We don’t want to jump the gun and make wrong assumptions. Please follow our forum and blog as well as our LinkedIn, Twitter, Facebook and Instagram pages as we’ll continue to update our customers and partners regularly.
Our Continued and Very Sincere Apologies
We continue to offer our very sincere apologies to all our partners and customers worldwide. The entire 3CX team continues to work around the clock.
Mar 30, 2023 - 10:41 PDT
Update - Update 30 March 2023 - 8:48 AM
3CX have released Desktop App version 18.12.422 to patch the security issue, please update the desktop app to this version as soon as possible
Mar 30, 2023 - 08:49 PDT
Monitoring -
Dear Client,
3CX Security Notice - 30 March 2023
What is the issue, and what does it affect?
We regret to inform our customers that the 3CX Electron Windows App shipped in Update 7, with version numbers 18.12.407 & 18.12.416, includes a security issue. Anti Virus vendors have flagged the executable 3CXDesktopApp.exe and in many cases uninstalled it. Electron Mac App version numbers 18.11.1213, 18.12.402, 18.12.407 & 18.12.416 are also affected.
The issue appears to be one of the bundled libraries that were compiled into the Windows Electron App via GIT.
3CX are still researching the matter to be able to provide a more in depth response later today.
Here’s some information on what has been done so far.
What has been done to mitigate the impact by 3CX?
Domains Have Been Taken Down
The domains contacted by this compromised library have already been reported, with the majority taken down overnight. A github repository which listed them has also been shut down, effectively rendering it harmless.
It is worth mentioning - that this appears to have been a targeted attack from an Advanced Persistent Threat, perhaps even state sponsored, that ran a complex supply chain attack and picked who would be downloading the next stages of their malware. The vast majority of systems, although they had the files dormant, were in fact never infected.
New Windows App in Progress
Currently, 3CX are working on a new Windows App that does not have the issue. They have also decided to issue a new certificate for this app. This will delay things by at least 24 hours so please bear with them whilst this happens.
What can you do until the new desktop App is ready for release?
Use the PWA App Instead!
We strongly suggest that you use the 3CX PWA app instead. The PWA app is completely web based and does 95% of what the electron app does. The advantage is that it does not require any installation or updating and chrome web security is applied automatically.
The reason 3CX has two apps is that when the 3CX Electron App was released, the PWA technology was not available yet. Now it's mature and working really well. More information on how to install it here. One deficiency at this time is the lack of the BLF (Busy Lamp Field) Option in the PWA dial pad.
The Full 3CX security notice is available here - https://www.3cx.com/blog/news/desktopapp-security-alert/
Regards
Servcomm Support
Mar 30, 2023 - 07:30 PDT