Now You Serial, Now You Don’t — Systematically Hunting for Deserialization Exploits

Written by: Alyssa Rahman

Deserialization vulnerabilities are a class of bugs that have plagued multiple languages and applications over the years. These include Exchange (CVE-2021-42321), Zoho ManageEngine (CVE-2020-10189), Jira (CVE-2020-36239), Telerik (CVE-2019-18935), Jenkins (CVE-2016-9299), and more. Fundamentally, these bugs are a result of applications placing too much trust in data that a user (or attacker) can tamper with.

Attackers have leveraged these vulnerabilities for years to upload files, access unauthorized resources, and execute malicious code on targeted servers. Within the past 2 years, Mandiant has particularly observed APT41 using .NET ViewState and Java deserialization exploits to target companies and government entities within North America.

Given the prevalence and impact of these vulnerabilities, our goal was to create a process to systematically hunt for exploitation attempts. In this blog post, we will share our new rule generation (HeySerial.py) and validation (CheckYoself.py) tools and walk through the research process we used to create them.

While this blog post mainly focuses on deserialization exploits, the tools and processes presented here can help with hunting for the exploitation of other types of zero-days. For example, we can use HeySerial to generate hunting rules for the JNDI code injection zero-day released last week for log4j (CVE-2021-44228). For more details, check the “A Note on CVE-2021-44228” section later in the post.

Understanding the Problem

What is a Deserialization Vulnerability?

“Serialized” data is just an object or data structure that has been encoded in a way that can be transferred easily – for example over the network. Developers do this regularly to pass objects between different parts of an application or between a client and server to maintain state. Once it’s transferred, it can be "deserialized" and used like it never left the original function.

Deserialization vulnerabilities result from applications putting too much trust in data that a user (or attacker) can modify. Deserialization can become dangerous when 3 conditions are met:

• The serialized object is provided by or can be modified by a user.
• An application attempts to deserialize and use the object without validation.
• The object is deserialized by a portion of the application with valuable libraries in the "class path".

Exploiting a deserialization issue involves crafting a payload that replaces what should be a benign object or data structure – such as a session token or a ViewState – with code in the targeted language that executes something malicious for the attacker.

If dangerous classes or libraries are imported and accessible in the application “class path”, an attacker can reference useful functions or object types (also referred to as “gadgets”) to execute their payload. Due to how applications are structured, the dangerous functions may not be directly accessible, so successful exploitation often requires chaining several gadgets together.

Projects such as YSoSerial (Java) and YSoSerial .NET (C#) consolidate public research on successful gadget chains for common libraries and make it easy for anyone to generate a payload with one of these chains. This is then encoded and can be passed to servers with deserialization bugs. When an application with these gadgets imported unsafely deserializes the payload, the chain will automatically be invoked and execute the embedded command on the affected server.

What Does Successful Exploitation Look Like?

Deserialization issues in HTTP servers can appear in many places – session/state data, Cookies, HTML form inputs, etc. In one recent example (CVE-2019-18211), the C1 CMS application unsafely deserialized objects passed through certain SOAP requests, leading to remote code execution.

An HTTP SOAP request with a malicious payload is shown in Figure 1, and the server response is shown in Figure 2. In this case, the server returned a simple HTTP 200 OK response after deserializing and executing the provided object.

Problem Surface Area

As mentioned, deserialization vulnerabilities can affect a wide variety of languages and libraries which results in a very large surface area for attackers. Before we can develop a comprehensive hunting strategy, we need a thorough understanding of the problem space.

Varying Protocols

This class of vulnerabilities is most commonly discussed in the context of web applications, but they’re not the only applications affected.

In 2017, Google Project Zero demonstrated how to use .NET deserialization to target systems over Managed DCOM instead of HTTP. While not specifically deserialization, in 2016, two different Black Hat talks outlined ways to exploit the Java Messaging Service (JMS) and Java Naming and Directory Interface (JNDI) APIs through object/command injection. Just last week, a JNDI command injection 0-day was released for the log4j Java logging package (CVE-2021-44228).

For this blog post, we will limit our scope to attacks that occur over HTTP. This will allow us to focus our initial research while targeting the majority of deserialization exploitation attempts.

Languages and Objects

While any language can theoretically be at risk, some of the common languages/object types exploited with this class of vulnerability are serialized Java objects, .NET ViewStates, pickled Python objects, and serialized PHP objects. The following table details the header values used as a prefix for each of these object types.

*PHP serialized objects start with the ASCII O:LENGTH_OF_NAME: where LENGTH_OF_NAME is an integer. 4F 3A is the hex encoding of O:, but further tuning with a regular expression like O:[0-9]+: is necessary to avoid false positives.

Objects may also be serialized using language-specific formatters. For example, .NET applications may use formatters such as the BinaryFormatter, LosFormatter, NetDataContractSerializer, Json.NET, or SoapFormatter depending on the vulnerable gadget. Of note, YSoSerial.NET generates UTF-16LE objects for most of its supported formatters, so many of these payloads start with 0xFFFE.

Encoding

While serialized objects can be transferred using the raw bytes, more frequently they will be encoded or encrypted in some way to make network transfer simpler. Base64 is very common as it is language agnostic and simple to implement.

Encoding methods can also be layered—for example by GZIP compressing and then Base64 encoding an object. For this blog post, we will focus on simple Base64 encoding due to its prevalence.

Known vs Unknown Chains

Finally, this problem space includes a lot of unknowns. Deserialization vulnerabilities are regularly disclosed and, as mentioned, can affect any part of an application. Attackers are much more limited in how they can weaponize these vulnerabilities, and as defenders, we have an opportunity to hunt for novel exploitation by looking for the gadget chains and keywords that attackers must use to execute their final payload.

Security researchers (and attackers) are constantly looking for new dangerous gadget chains in common libraries or applications. Several well-known projects that centralize this research are listed in the following table, although this is by no means a comprehensive list. For this blog post, we’ll start with the ysoserial Java project, which was the first major tool and research to be released for this class of bugs.

For this research effort we want to identify a way to generate rules for known gadget chains, but we also want a more generalized approach that can let us proactively identify exploitation attempts for novel vulnerabilities and/or gadget chains. One method here could be looking for the payload vs the chain used to execute the payload. For example, we could hunt for serialized objects with DOS headers, malicious commands, or suspicious binaries.

Solution Surface Area

Now that we understand what we are trying to hunt for, we need to determine how we will hunt for it. The available detection surface area varies depending on your goals and visibility, but hunting opportunities typically fall into three categories:

• Network
  • The most direct method for detecting this method of attack is to observe the exploitation attempt as the requests are made.
• Endpoint - Dynamic
  • This may involve looking for uncommon process execution or behavior from web servers. (For example, IIS servers running cmd.exe /c whoami.) This will limit us to observing successful exploitation attempts only, though, and it will be biased towards exploitation for remote code execution (RCE). We may have limited visibility into exploitation for other objectives like file upload or remote URL inclusion.
• Endpoint - Static (Log/File)
  • Depending on your network traffic visibility, using YARA rules to look at decrypted requests in server logs may provide the same (or better) visibility into exploitation attempts than a network IOC. One limitation here is that logs may not include the full server response, so we will have incomplete evidence.

For this blog post, we will focus on network hunting (through Snort rules) with some static log file hunting (through YARA rules).

Make or Break

Now that we have a grasp of the problem space we want to address, we can define a hunting plan. We need an approach that will let us generate hunting logic and translate it into a variety of detection rule formats for both suspicious gadget chains and suspicious keywords. We also need to account for variability in object type and encoding method.

Tools

Since this is a complex class of bugs, we started out by creating HeySerial, a Python tool for rule generation. This lets us rapidly prototype detection logic, and it will let us keep up with new vulnerabilities as they are found.

As of publishing, HeySerial supports the following options:

To generate rules for ViewState objects with a known vulnerable chain:

python3 heyserial.py -c 'ExampleChain::mscorlib+ActivitySurrogateSelector' -t NETViewState

To generate rules for all object types with suspicious keywords:

python3 heyserial.py -k cmd.exe whoami ‘This file cannot be run in DOS mode’

To generate rules for ViewState objects with UTF-16LE encoded Base64 encoded keywords:

python3 heyserial.py -k Process.Start -t NETViewState -e “base64+utf16le”

Although HeySerial supports a limited number of initial encoding and object types, it was designed to be extensible. For more details on how to extend HeySerial and add new encoding methods, object types, or rule formats, check out the Developer Guide.

Solve for Ex(ploits)

The next step is gathering (or creating) payloads to test our rules against. There are many public projects, as mentioned, but we will focus on the ysoserial Java gadget chains. YSoSerial lists the supported chains in the README, but the list is more than three items so let’s automate it!

The following command generates a payload that launches “calc.exe” using the CommonsCollections1 chain. The raw hex bytes of the payload are shown in Figure 4. If we print the file contents directly, we will see some interesting strings mixed in with other non-printable (non-ASCII) characters, as seen in Figure 5.

java -jar utils/ysoserial.jar CommonsCollections1 calc.exe > commonscollections1.bin

We can use a simple Bash script to loop through the list of supported chains and output them to a file. Some chains require different payload inputs than others, so check out generate_payloads.sh to see how we can handle that with a switch-case block. For YSoSerial.NET payloads, try the generate_payloads.ps1 script on a Windows system.

Since we want to test Base64 rules, we can also add a command to the payload generation script that will save the Base64 encoded payloads to a separate file for us.

base64 -w 0 “$filename.bin” > “$filename.base64”

PCAPs or It Didn’t Happen

Now that we have our generated payloads, we can simply run YARA against the files to validate our rules. However, to test our Snort rules, we’ll need to generate a PCAP with the payloads demonstrating a “malicious” web request.

For that we can start a basic HTTP server using this server.py script, and we can execute the following Bash commands in a separate terminal. This will do a plain HTTP POST request to our local server with each payload in the body of the request.

for sample in `ls *.bin`; do curl -m 3 --data-binary “@$sample” http://127.0.0.1:12345; done;

for sample in `ls *.base64`; do curl -m 3 --data “@$sample” http://127.0.0.1:12345; done;

For testing purposes, we don’t need to exploit an actual vulnerability or web server. We just need network traffic that looks like an attempted exploit. Using these PCAPs and the payload files we already generated, we will be able to test both Snort and YARA rules.

The Golden Rules

Ok we have payloads, and we have a script. It’s finally time to make our hunting rules! There are many ways to identify the list of keywords we want to include for each chain. The YSoSerial repo includes details of the classes and functions that make up a payload, but this is

1. Scattered in different files
2. Not necessarily an exact match for what will be in the final serialized object

Another simple method for bulk extracting imported classes or functions is looping through all (raw hex) payloads and getting the first 5 strings with a “\..*\.” pattern. We can accomplish this by adding the following line to our Bash script:

strings “$filename.bin” | grep -E ‘\..*\.’ | head -5 > “$filename.strings”

This approach requires a bit of manual cleanup on the strings, but once we’re done, we can generate rules knowing these classes or functions are included in a default YSoSerial Java payload.

(Please note, these—especially YARA—are all hunting rules for research purposes, not detections. Do not deploy these to production systems without testing and tuning for your environment.)

The following command generates Snort and YARA rules with both raw and Base64 encoded chains for some common payloads in YSoSerial (Java).

Putting It to the Test

Now for the exciting part—seeing it all in action!

YARA Rules

Testing YARA rules on files is really simple with YARA installed. We can save our generated rules to a file, and then run the following command:

yara ysoserial_CommonsCollections1.yar ysoserial_CommonsCollection1.bin

If it works, we will see a line with the rule name and the file it matched on. The very last line of Figure 6 shows that our YARA rule matches!

Snort Rules

We can also test out our Snort rules using a local installation of Snort. Snort will throw an error if you have invalid (or duplicate) signature IDs, so we can bulk edit our rules with this command. (Please update the SIDs to valid values if you deploy them in your environment.)

perl -pe 'BEGIN{$A=100;} s/<REPLACE_SID>/$A++/ge' -i rules/*/*snort

The following command will run a specific rule file against a test PCAP:

sudo snort -A console -k none -q -r ysoserial_java_rawbase64.pcap -c CommonsCollections1_3.snort

The “-k none” option tells Snort to disable checksum mode, because our test data was generated with localhost as the source/destination and will be ignored by Snort otherwise. We can also set this by changing “config checksum_mode: all” to “none” in the /etc/snort/snort.conf configuration file. As shown in Figure 7, our Snort rule matches on the PCAP!