Detecting Log4j in the Carbon Black Console - An evaluation Campaign by our Top MDR Analysts

If you haven’t heard by now, Log4Shell is the latest of large-scale vulnerabilities that have security professionals, system administrators, and developers reeling.  So, what does this attack look like in the Carbon Black Cloud? This is a top question for our Managed Detection and Response (MDR) analysts who immediately set out to identify what post-exploitation behavior would look like. 

In late 2021, our team identified a widespread campaign targeting the educational sector and immediately leaped into action to protect our customers. After containing the threat, our MDR analysts evaluated the attacks in order to provide insight into the first large-scale operation to weaponize the Log4j exploitation.

Yet Another Log4j Overview

With a Common Vulnerability Severity Score (CVSS) of 10 out of 10, Log4j (CVE-2021-44228) leverages a security flaw within the Java Naming and Directory Interface (JNDI) feature allowing a malicious actor to execute arbitrary code within the vulnerable network.  Since its re-discovery, there have been endless publications about how to detect and mitigate this vulnerability from network environments.  Until organizations are able to upgrade to newer versions of Apache Log4j2 or disable message lookup features, exploitation of this vulnerability has proven itself to be fairly simple due to the feature’s broad use within a vast number of common applications.  This same widespread feature adoption makes this vulnerability low-hanging fruit for nefarious cyber threat actors to target.  Observed attacks have used this technique to gain the initial access into the network and thus far the versatility in attack methods has made it even more difficult to detect.

Ideally, by following currently available mitigation guidance, a Log4Shell attack would be dead-on-arrival at the network perimeter.  However, in the case that those efforts are still delayed for a myriad of reasons like business operations or limited staffing, we are proud to say that once again our next-generation Endpoint Detection and Response (EDR) products in combination with our Managed Detection and Response (MDR) team have proven to be up to the challenge to stop these ever-changing attacks. For a more technical overview of observed attacks in the wild and how to use various VMware Carbon Black products to detect and block these attacks, check out this threat notice from our expert Threat Analysis Unit: TAU-TIN-Log4Shell Exploitation

An Actual Sighting: The Analyst Perspective

The Log4Shell vulnerability has a low technical barrier of entry and has been leveraged to infiltrate victim networks with just a single line of code. Once initial access is gained attackers have their choice of tactics and techniques to further infiltrate the network.  Recently, analysts on the Managed Detection and Response (MDR) team observed two alerts from separate organizations that came in within a minute of each other.  Both alerts were triggered because the sensor installed on the host device detected PowerShell attempting to run potentially malicious fileless scripts.  The first indicator pointing to malicious activity for these alerts was encoded PowerShell commands.  Encoded PowerShell is commonly used by attackers as a form of obfuscation to avoid detection.  With our VMware Carbon Black Cloud (CBC) product, our analysts were able to see that the attackers were attempting to establish a web client session to a command-and-control server located in Russia.

The other indicator consistent in both alerts was the use of ws_tomcatservice.exe invoking PowerShell to run the previously observed commands.  Unpatched instances of Apache Tomcat have been a primary leverage point for this type of attack. One of the impacted customers had deployed our Enterprise Endpoint Detection and Response (EEDR) product which provided additional telemetry and insight into the malicious activity.  This data allowed our team to thoroughly analyze the encoded PowerShell script that was downloaded from the initial attack.  We were able to classify the Indicators of Compromise (IOCs) present in these attacks to identify this behavior as part of a widespread campaign by malicious threat actors to run various Log4j related attacks.

NOTE: Click here to learn about the suite of VMware Carbon Black security products, and the differences between CB Endpoint Standard (EDR) and Enterprise Endpoint Detection and Response (EEDR).

Figure 1: ws_tomcatservice.exe invoking PowerShell

Figure 2: Encoded PowerShell Script

Our analysts determined that these key factors were enough to raise alarms and initiate our internal incident reporting process.  In both cases, the team contacted the organizations’ respective account managers and alerted them of the suspicious activity going on.

The greatest difference between VMware Carbon Black’s Managed Detection (MD) and Managed Detection and Response (MDR) products is threat containment.  Both customers in this case had purchased our MD offering, which includes the same caliber of team talent, thorough analysis of threat information, scope within the network, and security policy recommendations to help contain and remediate any identified incidents.  With our MDR product, when the customer received the information about the threat within their network, they also would have had the peace of mind that the threat was already contained, rather than calling in their security team to take on that initial action. This allows teams to focus on remediation actions, speeding up recovery efforts.

Carbon Black Threat Detection Lifecycle

Once new attack vectors, such as the Log4Shell attacks, are observed in the wild, the Managed Detection and Response (MDR) team will begin the coordinated sprint across all the different VMware Carbon Black security departments to ensure that all customers are protected against this threat. 

In tandem with 24/7 network monitoring and any ongoing investigations, MDR analysts collaborate with the VMware Carbon Black Threat Analysis Unit (TAU) to ensure our product is constantly improving and ready to block evolving attack techniques.  These real-world attacks help to better establish true positive alerts for our customers and prevent alert fatigue.  Additionally, TAU will coordinate with analysts highly experienced in threat intelligence to determine similarities between attack methods and previously reported threat groups or targeted customer industry.  If the analyst determines that this may be the start of a targeted campaign, the MDR team will conduct active threat hunting within our MD and MDR customer environments to include policy review and ensure they are always protected against the latest threats.

Using this process, our team was able to handle both incidents and provide remediation steps to our MD customers quickly.  Within 24 hours all hosts with a CB EDR sensor would alert on this new activity with an elevated severity score of 8 within the Carbon Black Cloud (CBC) Console.

It's a Marathon, Not a Sprint

It is paramount to understand this vulnerability impacts millions of devices worldwide.  In addition, the exploitation potential of this vulnerability is very much in its infancy stage.  With time we expect to see new exploitations to be unveiled as log4j matures.  VMware Carbon Black highly suggests implementing patches and best practices that aid in risk reduction against Log4Shell as soon as they are released.  Unfortunately, due to its wide reach, this vulnerability will be around for years to come.  Vigilance is the primary key to protect your organization from malicious threat actors.