Cloudy with a Chance of Bad Logs: Cloud Platform Log Configurations to Consider in Investigations
Mandiant
Written by: David Pany, Caitlin Hanley
More and more organizations utilize cloud technology for applications, file storage, and more. However, if an attacker compromises a cloud environment, organizations may not know how to investigate those technologies, or may not even be logging the evidence that could allow the organization to identify what an attacker did.
This blog post describes a hypothetical scenario of a cloud platform compromise with multiple components that would require investigation. Each component is an example of a real intrusion tactic that Mandiant has investigated across various cloud platforms, sometimes with logs available and sometimes without logs available.
Cloud Technology Themes
For each part of the compromise, we provide recommended logging configurations and investigation processes organized into cloud technology “themes” that group cloud services from Google Cloud Platform (GCP), Amazon Web Services (AWS), and Microsoft Azure together:
- Cloud Virtual Machines
- GCP Compute Engine Virtual Machines, AWS EC2 Instance, Azure Virtual Machine
- Cloud Applications or Cloud Containers
- GCP Kubernetes Engine, AWS Elastic Kubernetes Service, Azure Kubernetes Service
- Cloud Serverless Functions
- GCP Cloud Functions,AWS Lambda, Azure Functions
- Cloud Database Services
- GCP Datastore, GCP Cloud Bigtable, GCP Cloud SQL, AWS DynamoDB, AWS Aurora, AWS Relational Database Service, Azure Database, Azure SQL Database
- Cloud Authentication Services
- GCP Cloud Identity, Azure Active Directory, AWS Directory Service
- Cloud Management Console
- GCP Console, Azure Portal, AWS Console,
- Cloud Email
- Google Workspace, Microsoft 365, Amazon Simple Email Service
- Cloud Code Repositories
- GCP Cloud Source, AWS CodeCommit, Azure Repos
- Cloud Logging Platforms
- GCP Logs Explorer, AWS Athena, Azure Monitor, Microsoft Sentinel, Azure Log Analytics
- Cloud Log Analysis Formats
- GCP Audit Logs, GCP VPC Flow Logs, AWS CloudTrail, AWS VPC Flow Logs, Azure AD Audit Logs, Azure AD Sign In Logs, Azure Resource Logs, Azure Activity Logs, Azure NSG Flow Logs
- Cloud Networking
- GCP Virtual Private Cloud, AWS Virtual Private Cloud, Azure Virtual Network
- Cloud File Storage
- GCP Cloud Storage, AWS Simple Storage Solution (S3), Azure Blob Storage
Main Takeaways
After reading through this scenario, you should be able to:
- Understand an example attack technique that targets each cloud technology theme
- Identify event log configurations that should be reviewed in your cloud platform to facilitate an investigation
- Develop and test incident response playbooks using the investigation recommendations
- Utilize the event log checklists to review logging configurations and create logging standards
Areas to Research Further
While we review many concepts, there are some limitations to be aware of in the scope of this post:
- These logging and investigation themes are just starting points to be aware of as you design cloud platforms unique to your environment. Not all of the logs discussed may be available or feasible, but if implemented they would assist in helping investigators identify malicious activity that may have only been recorded in the logs. This improves the timeliness and accuracy of the investigation
- Since this blog post discusses a wide variety of cloud platforms, and configurations are frequently changing, we do not provide log implementation steps. Please work with your cloud administration team and cloud vendors to identify the considerations, configurations, and costs associated with the logs discussed here.
- There are many hardening and configuration practices available to mitigate the malicious actions that occur in the post that are not covered here.
The Attack Path
1. Credential Stuffing
The attacker gained access to the Cloud Email platform through a credential stuffing attack against a cloud administrator account. Once the attacker found a valid password, the attacker authenticated with those credentials and the Cloud Email platform asked them which type of multi-factor authentication (MFA) process they preferred. The attacker chose the “push” option, which sent an approval request to the legitimate user. The administrator user deals with push authentication requests throughout the day for various services and mistakenly accepted the authentication request, which provided initial access to the attacker.
Investigation Theme: Cloud Authentication Services and Cloud Email
- Analyze logins for the cloud administrator account.
- Analyze Cloud Authentication Service alerts for risk-based patterns such as credential stuffing or authentications from unexpected locations.
- Identify if IP addresses associated with failed logons have any successful logons.
- Identify user accounts logging in from multiple IP addresses in multiple locations, particularly if the IP addresses are unexpected based on previous legitimate user activity.
- Utilize threat intelligence to enrich context for suspicious IP addresses identified.
- Review emails received by users for possible credential harvesting phishing links, particularly if the user reported the email as phishing.
- Review Cloud Email alerts for suspicious emails identified by Cloud Email provider and users.
- Review logs from Cloud Authentication Service risk-based detections for user sign-ins.
Logging Theme: Cloud Authentication Services
- Log user authentication with timestamp, username, and source IP address.
- Log multi-factor authentication details.
- Turn on risk-based detections, if available.
2. Reconnaissance
Once the attacker identified the cloud administrator credentials and authenticated, they logged in to the Cloud Management Console to identify other applications that the user could access.
Investigation Theme: Cloud Authentication Services
- Analyze the Cloud Management Console authentication logs for the previously identified suspicious source IP addresses and compromised user account.
- Analyze the Cloud Management Console application access logs to identify unusual application access activity.
Logging Theme: Cloud Authentication Service
- Log user authentication with timestamp, username, and source IP address.
3. Reconfiguring Privileges
The attacker identified that the cloud administrator account had access to the Cloud Authentication Services application and authenticated to it. In the Cloud Authentication Services application, the attacker changed the privileges of the cloud administrator to the highest global administrator account privileges available and removed the multi-factor requirement.
Investigation Theme: Cloud Authentication Services
- Analyze changes to user accounts, including password, permissions, and contact information such as phone numbers for MFA or password reset.
- Analyze accounts that have weak security controls such as disabled MFA requirements.
- Analyze applications that have weak security controls such as disabled MFA requirements or access to unexpected user accounts.
- Analyze MFA settings per account for anomalies such as disabled MFA, multiple MFA methods registered, recent MFA configuration changes, or configuration changes outside of policy.
Logging Theme: Cloud Authentication Services
- Log access to all cloud services for authenticated users.
- Log user authentication with timestamp, username, and source IP address.
- Log changes to user permissions and configurations.
4. Identifying Hard-coded Credentials in Code
While in the Cloud Management Console, the attacker identified that the organization uses a custom Cloud Application. The attacker accessed the Cloud Code Repository with the global administrator account and identified the Cloud Application source code hosted there. The attacker accessed the code and identified plain-text hard-coded credentials for an application service account.
Investigation Theme: Cloud Applications or Containers, Cloud Code Repositories
- Analyze user access to application source code.
- Analyze creation and modification of application source code.
- Review accessed code to identify impact of exposed data, such as credentials.
- Review logs related to application-related files and code download, if available.
Logging Themes: Cloud Authentication Services, Cloud Applications and Containers, and Cloud Code Repositories
- Log access to all cloud services for authenticated users.
- Log creation, modification, and access to application code.
- Log download of files and code related to application.
- Log web-based code views, downloads, and edits.
- Log code management access and modification through tools such as git.
- Log user authentication with timestamp, username, and source IP address.
5. Identifying Hard-coded Credentials in Logs
While in the Cloud Authentication Services application, the attacker identified that the Administrator had access to the Cloud Logging platform. The attacker authenticated to the Cloud Logging platform and searched logs for keywords related to plain-text credentials. The attacker exported logs that contained those keywords, particularly database user credentials.
Investigation Theme: Cloud Logging
- Analyze access to cloud log aggregation platforms.
- Analyze log queries performed.
- Analyze exported logs.
- Analyze log modification and deletion.
Logging Theme: Cloud Logging
- Log authentication to logging services.
- Log queries executed for log data.
- Log data exports.
- Log modification/deletion of log data.
6. Environment Enumeration
The attacker returned to the cloud Authentication Service application and performed reconnaissance on systems and users. The attacker exported all environment objects including systems and accounts.
Investigation Theme: Cloud Authentication Services
- Analyze access to Authentication Service queries and configurations viewed.
- Analyze exported Authentication Service and domain data.
- Analyze Authentication Service modifications for permissions and security parameters.
Logging Theme: Cloud Authentication Services
- Log access to all cloud services for authenticated users.
- Log changes to user permissions and configurations.
- Log exported domain data.
- Log created user accounts.
7. Infrastructure Creation
Next, the attacker pivoted to the Cloud Virtual Machine infrastructure and created a templated virtual machine. The attacker assigned the virtual machine to the application service account previously identified in the application source code. The attacker configured the Cloud Networking rules to allow remote desktop protocol (RDP) access from the internet. The application service account did not require MFA for any authentication activity because of its intended use. The attacker logged on to the virtual machine through RDP from their command and control (C2) server.
Investigation Theme: Virtual Machines
- Analyze virtual machine creation and modification events.
- Analyze virtual IP address actions such as create, delete, and modify.
- Analyze changes made to network configurations.
- Analyze modifications to network controls.
- Analyze Authentication Service authentications for systems.
Logging Themes: Virtual Machines and Cloud Networking
- Configure system event logs to follow standard endpoint logging policies for authentication, user activity, and privileged account use.
- Log virtual machine management actions such as start, pause, backup, snapshot, Create, Delete, and Command executions.
- Log changes made to network configurations.
- Log virtual IP address management actions such as create, delete, and modify.
- Log network flow metadata.
8. Database Access
While logged on to the newly created virtual machine, the attacker identified a database server based on the hostname SQLDB01. The attacker moved laterally from the virtual machine they created to the database server via RDP using the application service account.
The attacker connected to the database, which utilized a Cloud Database Service backend, using the database user credentials previously identified in logs and explored the data by enumerating the table schema and running “select *” queries.
Investigation Theme: Cloud Database Services
- Analyze database authentication logs to identify unexpected authentications based on account name, timeframe, or source of authentication.
- Analyze queries for reconnaissance activity such as “select *” or access to unexpected data.
- Analyze queries for modification and deletion activity.
Logging Theme: Cloud Database Services
- Log database user authentication and source network address.
- Log data access including source network address and user.
- Log data modification and deletion including source network address and user.
- Log errors and long running queries, which could be indicative of data transfer or reconnaissance.
9. Network Scanning
While logged on to the attacker-created virtual machine, the attacker also performed internal reconnaissance to identify other systems of interest. The attacker scanned the network for other systems using custom port scanning utilities that searched for open SSH, RPD, and SMB ports.
Investigation Themes: Cloud Virtual Machines and Cloud Networking
- Analyze endpoint artifacts on virtual machines based on endpoint forensic processes.
- Review internal network log data for patterns of network scanning.
Logging Themes: Cloud Virtual Machines and Cloud Networking
- Configure system event logs to follow standard endpoint logging policies for authentication, user activity, and privileged account use.
- Forward system logs to a log management platform or SIEM as part of standard polices and processes.
- Log network flow metadata.
10. File Theft
The attacker identified a network-shared file server that hosted files on a Cloud File Storage solution. After enumerating files stored on the network share, the attacker copied files to their C2 system using a bulk network file transfer utility.
Investigation Theme: Cloud File Storage
- Analyze files accessed by user accounts and source IP addresses.
- Analyze users with a large number of file downloads during the timeframe.
- Analyze users with a large number of file deletions during the timeframe.
Logging Themes: Cloud File Storage and Cloud Networking
- Log file download events with user account, source IP address, and timestamp.
- Log network flow metadata.
- Log file creation, modification, upload, and deletion events with user account, IP address, and timestamp.
- Log API access to file storage locations, folders, and files.
- Log file and directory listing metadata view.
11. Placing Malware
While accessing the file server, the attacker also decided to stage further backdoors in trojanized files that are likely to be opened by users.
Investigation Theme: File Storage
- Analyze file uploads, creations, modifications, and deletions, particularly from compromised accounts and IP addresses.
- Analyze access to trojanized files to identify users whose systems need further investigation.
- Scan files with anti-virus.
- Analyze quarantined files.
Logging Theme: File Storage
- Log user authentication.
- Log file creation, upload, modification, and deletion events, including IP addresses.
- Log file download events with user account, source IP address, and timestamp.
- Turn on alerts for suspicious activity, including malware and mass downloads, if available.
12. Email Theft
While logged on to cloud email for the administrator account, the attacker browsed through the last several days of messages. The attacker looked at email folders named “finance” and “hr” and downloaded attachments from sent messages.
Investigation Theme: Collaboration — Cloud Email
- Analyze messages viewed in a mailbox, particularly by compromised accounts and IP addresses.
- Analyze attachments downloaded in a mailbox.
- Analyze searches performed in a mailbox.
Logging Theme: Collaboration — Cloud Email
- Log authentication to mailboxes.
- Log access and views of email messages.
- Log download and access of email attachments.
- Log searches of mailboxes.
13. Spreading Malware
The attacker shared the uploaded trojanized backdoor file through the collaboration platform’s file sharing service with 20 users.
Investigation Theme: Collaboration — Cloud File Sharing
- Analyze known bad files to see what accounts shared them and with whom.
- Analyze known bad file downloads.
Logging Theme: Collaboration — Cloud File Sharing
- Log authentication of user account and source IP address.
- Log file creation, modification, upload, and deletion events with user account, IP address, and timestamp.
- Log file download events with user account, source IP address, and timestamp.
- Log location, folder, and file permission changes.
- Log API access to file storage locations, folders, and files.
14. Impersonating Users
Several users messaged the administrator’s account and asked questions about errors opening the new document they downloaded through the collaboration platform based on an automated file shared email link. The attacker replied to tell the users the document is legitimate.
Investigation Theme: Collaboration — Cloud Chat
- Analyze chat message logs sent by compromised accounts.
- Analyze chat message logs sent from users logged in from known malicious IP addresses.
Logging Theme: Collaboration — Cloud Chat
- Log authentication of user account and source IP address.
- Log messages sent, received, edited, and deleted.
- Log files transferred and store content for review.
15. Anti-forensics
Finally, in an attempt to delay detection, the attacker created a mailbox rule to automatically delete replies to the compromised file share email.
Investigation Steps
- Analyze current mailbox rule configurations to identify active mailbox rules.
- Analyze mailbox rule logs to identify if the attacker modified existing rules or deleted rules they no longer needed.
- Analyze messages currently in “Deleted” folders.
- Analyze logs of messages permanently deleted.
- Analyze other email message storage locations such as security tools or e-discovery retention platforms.
Logging theme: Collaboration — Email
- Log mailbox rule creation, modification, and deletion.
- Log message deletion.
Detection and Response
The aforementioned hypothetical scenario took place in a matter of several days, reflecting how quickly the threat actors moved in the real scenarios this one is based on. In these cases, information security teams commonly have only a few medium priority alerts fire that go unnoticed due to the abundance of alerts feeding from their tools.
In this scenario, suspicion started when several helpdesk team members realized they had separate reports of users who had suspicious files shared with them. The helpdesk team escalated to Information Security per their documented processes and the Incident Response (IR) team started an investigation into the cloud file sharing platform associated with the file sharing.
The IR team quickly realized that the default logging available with their lowest cost license subscription recorded many useful logs such as:
- Failed and successful logons associated with credential stuffing and initial compromise
- File sharing activity
- Mailbox rules created
- Files accessed in the cloud file sharing platform
Unfortunately, the investigation could not answer the question “did the attacker access any email messages or synchronize any mailboxes?” due to the default logging levels. The IR team also realized they were lucky the incident was detected relatively quickly because the default license subscription only stored logs for 90 days with their Cloud Logging platform.
After a post-mortem review several months later, the organization realized the IR team only reviewed collaboration platform authentications and did not cross reference against domain authentication logs. This meant that the internal team never identified that the attacker compromised the cloud infrastructure platform and performed follow-on activities such as creating and accessing a VM, elevating to domain administrator privileges, and interacting with file servers. They focused only the collaboration platform because the initial incident identification occurred after the sharing of files on the Collaboration Cloud File Sharing platform. The investigation had to be reopened several months later when evidence had started to disappear from Cloud Logging sources.
Conclusion
As the scenario demonstrates, attackers have a wider surface area to persist and steal data because of the adoption of cloud infrastructure and collaboration platforms. The move to these cloud platforms brings useful functionality and security features, but configuring everything correctly can be overwhelming for a team that is new to the technology.
Not only are there many access, permission, and protection configurations to consider, but teams should also make sure that they would be able to fully investigate various attacks that could happen by storing the correct logs.
Understanding what technologies your organization uses and performing threat modeling is one way to make sure you have these logs and investigative processes set up should you need to investigate.
For details on how Mandiant can assist with your cloud security, please check out the following resources:
Critical Attack Path
The following attack path diagram visualizes how the actor accessed a wide range of cloud platforms from outside a standard perimeter in this scenario. The actor also used cloud technologies to interact with systems in the non-cloud environment as well through connections and integrations.


Infrastructure Logging Checklist
The following checklist is designed to be copied or printed for your cloud infrastructure logging review efforts. The provided logs are example categories of commonly utilized event logs for forensic investigations.