Analyze secrets with Cloud Asset Inventory

In this topic, learn how to leverage Cloud Asset Inventory to analyze Secret Manager resources.

This is an advanced topic for Secret Manager. Before reading this guide, we recommend reviewing the following:

• Platform overview to understand the overall Google Cloud landscape
• Secret Manager overview to understand Secret Manager
• Cloud Asset Inventory overview to understand Cloud Asset Inventory and to see its comprehensive asset management features

Overview

Secret Manager is integrated with Cloud Asset Inventory, Google Cloud's managed metadata inventory system. With this integration, you can identify and audit secrets across your organization, folder or project, and discover any configurations that aren't conformant to your organization's requirements. This guide covers monitoring your assets, exporting assets to BigQuery, and samples of Cloud Asset Inventory queries over Secret Manager resources.

Notes

• Although all queries have samples written with the Google Cloud CLI and BigQuery, we recommend exporting your secret and secret versions to BigQuery. By exporting your assets to BigQuery, you can write SQL-like queries to produce and store meaningful analysis.
• Secret Manager is not integrated with Asset Search or Policy Analyzer. The queries below leverage native properties of Google Cloud CLI and BigQuery to search through assets.
• Cloud Asset Inventory supports exporting and listing snapshots only from the past five weeks.

Monitor Asset Changes

Cloud Asset Inventory tracks real-time updates and supports monitoring these changes. You can configure feeds to send notifications to a set of configured Pub/Sub topics each time there's a modification to your resources. Additionally, Cloud Asset Inventory supports configuring conditions on your feeds, so you can monitor specific changes for certain asset types. See the Pub/Sub documentation to learn how to trigger workflows on asset changes.

Export Assets to BigQuery

Exporting your secrets and secret versions to BigQuery lets you run SQL-like queries over large amounts of data and produce meaningful insights about your assets. Prior to exporting your assets, ensure that your dataset and service accounts are configured correctly. To export your assets, run the following command:

gcloud

$ gcloud asset export \
   --content-type CONTENT_TYPE \
   --project PROJECT_ID \
   --snapshot-time SNAPSHOT_TIME \
   --bigquery-table BIGQUERY_TABLE \
   --output-bigquery-force

where:

• CONTENT_TYPE: Asset content type (RESOURCE).
• PROJECT_ID: Project ID containing assets to be monitored.
• SNAPSHOT_TIME: Time at which to snapshot resources. This be between now and 5 weeks in the past.
• BIGQUERY_TABLE: Table to export data to, in the format: projects/PROJECT_ID/datasets/DATASET_ID/tables/TABLE_NAME.

Note: Cloud Asset Inventory does not support appending data to an existing table. Data must be sent to a new table, or overwrite an existing table.

For more information, see Exporting to BigQuery.

Sample Queries

Secret created in the last two weeks

Discover any secrets (and their properties) that were added to your organization in the past two weeks:

BigQuery

SELECT name, FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/Secret' AND
DATE(JSON_VALUE(resource.data, '$.createTime')) > DATE_SUB(CURRENT_DATE(), INTERVAL 2 WEEK);

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.createTime>-P2W"

Automatically replicated secrets

Find all secrets replicated automatically:

BigQuery

SELECT * FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/Secret' AND
JSON_EXTRACT(resource.data, '$.replication.automatic') IS NOT NULL;

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.replication.automatic != NULL"

Secret replicated to a specified region

Find all secrets replicated to us-central1:

BigQuery

SELECT * FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE
(
  SELECT * FROM
  UNNEST(JSON_EXTRACT_ARRAY(resource.data, '$.replication.userManaged.replicas')) AS location
  WHERE JSON_VALUE(JSON_EXTRACT(location, '$.location')) = "us-central1"
)
IS NOT NULL;

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.replication.userManaged.replicas.location=us-central1"

Enabled secret versions over 180 days old

List all secret versions that were created more than 180 days ago:

BigQuery

SELECT * FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/SecretVersion' AND
DATE(JSON_VALUE(resource.data, '$.createTime')) < DATE_SUB(CURRENT_DATE(), INTERVAL 180 DAY) AND
JSON_VALUE(resource.data, '$.state') = "ENABLED";

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/SecretVersion' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.createTime < P6M AND resource.data.state=ENABLED"

Secret without CMEK configured

List all secrets (automatic and user-managed) that are not encrypted with customer-manager encryption keys (CMEK):

BigQuery

SELECT * FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/Secret'
  AND (
    JSON_VALUE(resource.data, "$.replication.automatic.customerManagedEncryption.kmsKeyName") IS NULL
    AND JSON_VALUE(resource.data, "$.replication.userManaged.replicas[0].customerManagedEncryption.kmsKeyName") IS NULL
  );

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.replication.userManaged.replicas.customerManagedEncryption = NULL OR resource.data.replication.automatic.customerManagedEncryption=NULL"

Secret with CMEK configured

List all secrets (automatic and user-managed) that are encrypted with CMEK:

BigQuery

SELECT * FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/Secret'
AND (
  JSON_VALUE(resource.data, "$.replication.automatic.customerManagedEncryption.kmsKeyName") IS NOT NULL
  OR JSON_VALUE(resource.data, "$.replication.userManaged.replicas[0].customerManagedEncryption.kmsKeyName") IS NOT NULL
);

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.replication.userManaged.replicas.customerManagedEncryption != NULL OR resource.data.replication.automatic.customerManagedEncryption!=NULL"

Secrets encrypted with a specific CMEK

Find secrets that are encrypted secret versions with a given CMEK:

BigQuery

SELECT * FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/Secret'
  AND (
    JSON_VALUE(resource.data, "$.replication.automatic.customerManagedEncryption.kmsKeyName") = KMS_KEY_NAME
    OR JSON_VALUE(resource.data, "$.replication.userManaged.replicas[0].customerManagedEncryption.kmsKeyName") = KMS_KEY_NAME
  );

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.replication.userManaged.replicas.customerManagedEncryption.kmsKeyName=KMS_KEY_NAME"

Secret versions without CMEK configured

Find all enabled secret versions that are not encrypted with CMEK:

BigQuery

SELECT * FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/SecretVersion'
AND (
  JSON_VALUE(resource.data, "$.replicationStatus.automatic.customerManagedEncryption.kmsKeyVersionName") IS NULL
  AND JSON_VALUE(resource.data, "$.replicationStatus.userManaged.replicas[0].customerManagedEncryption.kmsKeyVersionName") IS NULL
)
AND JSON_VALUE(resource.data, "$.state") = "ENABLED";

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/SecretVersion' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="(resource.data.replicationStatus.userManaged.replicas.customerManagedEncryption = NULL OR resource.data.replicationStatus.automatic.customerManagedEncryption=NULL) AND resource.data.state=ENABLED"

Secret versions encrypted with a specific CMEK

List all enabled secret versions encrypted with a specific CMEK version:

BigQuery

SELECT * FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/SecretVersion'
AND (
  JSON_VALUE(resource.data, "$.replicationStatus.automatic.customerManagedEncryption.kmsKeyVersionName") = KMS_KEY_VERSION_NAME
  OR JSON_VALUE(resource.data, "$.replicationStatus.userManaged.replicas[0].customerManagedEncryption.kmsKeyVersionName") = KMS_KEY_VERSION_NAME
)
AND JSON_VALUE(resource.data,"$.state")="ENABLED";

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/SecretVersion' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.replicationStatus.userManaged.replicas.customerManagedEncryption.kmsKeyVersionName=$FULL_KMS_KEY_VERSION_RESOURCE_NAME AND resource.data.status=ENABLED"

Secrets without rotation configured

Find all secrets that don't have a rotation schedule:

BigQuery

SELECT name FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/Secret' AND
JSON_EXTRACT(resource.data, '$.rotation') IS NULL;

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.rotation=NULL"

Secrets with a specific rotation period

Find all secrets scheduled to be rotated less than once every 90 days:

BigQuery

SELECT *
FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE
  CAST(
    TRIM(
      JSON_VALUE(JSON_EXTRACT(resource.data, "$.rotation.rotationPeriod")),"s")
    AS INT64)
< 86400 * 90 #Rotation period in seconds (86400s in 1 day * 90 days)

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ ROTATION_PERIOD_SECONDS=$((90 * 24 * 60 * 60))
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.rotation != null AND resource.data.rotation.rotationPeriod < ${ROTATION_PERIOD_SECONDS}s"

Secrets that will expire in the next 30 days

List secrets that will expire in the next 30 days:

BigQuery

SELECT * FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/Secret' AND
DATE(JSON_VALUE(resource.data, '$.expireTime')) < DATE_ADD(CURRENT_DATE(), INTERVAL 30 DAY);

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.expireTime < PD30"

Secrets with a Pub/Sub topic configured

List all secrets that have at least one Pub/Sub topic configured:

BigQuery

SELECT name, ARRAY_LENGTH(JSON_EXTRACT_ARRAY(resource.data, '$.topics')) AS topics_count,
FROM PROJECT_ID.DATASET_ID.TABLE_NAME
WHERE asset_type='secretmanager.googleapis.com/Secret' AND
ARRAY_LENGTH(JSON_EXTRACT_ARRAY(resource.data, '$.topics')) > 0

gcloud

$ NOW=$(TZ=GMT date +"%Y-%m-%dT%H:%M:%SZ")
$ gcloud asset list --project=PROJECT_ID \
    --asset-types='secretmanager.googleapis.com/Secret' \
    --snapshot-time=$NOW \
    --content-type='resource' \
    --filter="resource.data.topics !=NULL"

What's next

• Learn more about using Secret Manager with other products.