Export asset metadata by using Cloud Asset Inventory

Export asset metadata by using Cloud Asset Inventory

This quickstart shows you how to export asset metadata at a point in time using Cloud Asset Inventory and the Google Cloud CLI gcloud asset commands.

Before you begin

Before you can start working with Cloud Asset Inventory, you must enable the Cloud Asset Inventory API, the Google Cloud CLI, and assign permissions. The Google Cloud CLI interacts with Cloud Asset Inventory and other Google Cloud services. Learn more about the gcloud CLI.

Enabling the Cloud Asset Inventory API and Google Cloud CLI

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Enable the required API.

   Enable the API

4. Install the Google Cloud CLI.

5. To initialize the gcloud CLI, run the following command:

   gcloud init

6. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

7. Enable the required API.

   Enable the API

8. Install the Google Cloud CLI.

9. To initialize the gcloud CLI, run the following command:

   gcloud init

Configuring permissions

To call the Cloud Asset Inventory API, you must first configure permissions.

Searching resources

To search resource metadata, run the following command.

gcloud asset search-all-resources \
    --scope=SCOPE \
    --query="QUERY" \
    --asset-types=ASSET_TYPE_1,ASSET_TYPE_2,... \
    --order-by="ORDER_BY" \
    --read-mask="READ_MASK"

Provide the following values:

• SCOPE: Optional. A scope can be a project, folder, or an organization. The search is limited to the Google Cloud resources in this scope. The caller must be granted the cloudasset.assets.searchAllResources permission on the desired scope. If not specified, the configured project property is used.

  The allowed values are:

  • projects/PROJECT_ID

  • projects/PROJECT_NUMBER

    How to find a Google Cloud project number

    Console

    To find a Google Cloud project number, complete the following steps:

    1. Go to the Dashboard page in the Google Cloud console.

       Go to Dashboard

    2. Click the switcher box in the menu bar.
    3. Select your organization from the Select from box, and then search for your project name.
    4. Click the project name to switch to that project. The project number is shown in the Project info card.

    gcloud CLI

    You can retrieve a Google Cloud project number with the following command:

    gcloud projects describe PROJECT_ID --format="value(projectNumber)"

  • folders/FOLDER_ID

    How to find a Google Cloud folder ID

    Console

    To find a Google Cloud folder ID, complete the following steps:

    1. Go to the Google Cloud console.

       Go to the Google Cloud console

    2. Click the switcher box in the menu bar.
    3. Click the Select from box, and then select your organization.
    4. Search for your folder name. The folder ID is shown next to the folder name.

    gcloud CLI

    You can retrieve a Google Cloud folder ID that's located at the organization level with the following command:

    gcloud resource-manager folders list \
        --organization=$(gcloud organizations describe ORGANIZATION_NAME \
          --format="value(name.segment(1))") \
        --filter='"DISPLAY_NAME":"TOP_LEVEL_FOLDER_NAME"' \
        --format="value(ID)"

    Where TOP_LEVEL_FOLDER_NAME can be a full or partial string match. Remove the --format option to see further information about the found folders.

    To get the ID of a folder within another folder, list the subfolders:

    gcloud resource-manager folders list --folder=FOLDER_ID

  • organizations/ORGANIZATION_ID

    How to find a Google Cloud organization ID

    Console

    To find a Google Cloud organization ID, complete the following steps:

    1. Go to the Google Cloud console.

       Go to the Google Cloud console

    2. Click the switcher box in the menu bar.
    3. Click the Select from box, and then select your organization.
    4. Click the All tab. The organization ID is shown next to the organization name.

    gcloud CLI

    You can retrieve a Google Cloud organization ID with the following command:

    gcloud organizations describe ORGANIZATION_NAME --format="value(name.segment(1))"

• QUERY: Optional. The query statement. See Query syntax for more information. If not specified or empty, Cloud Asset Inventory searches all the resources in the specified scope.

  Examples:

  • name:Important to find Google Cloud resources whose name contains Important as a word.

  • name=Important to find the Google Cloud resource whose name is exactly Important.

  • displayName:Impor* to find Google Cloud resources whose display name contains Impor as a prefix.

  • location:us-west* to find Google Cloud resources whose location contains both us and west as prefixes.

  • labels:prod to find Google Cloud resources whose labels contain prod as a key or value.

  • labels.env:prod to find Google Cloud resources that have a label env and its value is "prod".

  • labels.env:* to find Google Cloud resources that have a label env.

  • kmsKeys:key to find Google Cloud resources encrypted with customer-managed encryption keys whose name contains key as a word.

  • sccSecurityMarks.key=value to find Google Cloud resources that are attached with a pair of security marks whose key is key and value is value.

  • sccSecurityMarks.key:* to find Google Cloud resources that are attached with a pair of security marks whose key is key.

  • state:ACTIVE to find Google Cloud resources whose state contains ACTIVE as a word.

  • NOT state:ACTIVE to find Google Cloud resources whose state doesn't contain ACTIVE as a word.

  • createTime<1609459200 or createTime<2021-01-01 or createTime<"2021-01-01T00:00:00" to find Google Cloud resources that were created before 2021-01-01 00:00:00 UTC. 1609459200 is the epoch timestamp of 2021-01-01 00:00:00 UTC in seconds.

  • updateTime>1609459200 or updateTime>2021-01-01 or updateTime>"2021-01-01T00:00:00" to find Google Cloud resources that were updated after 2021-01-01 00:00:00 UTC. 1609459200 is the epoch timestamp of 2021-01-01 00:00:00 UTC in seconds.

  • project:12345 to find Google Cloud resources that belong to a Google Cloud project with number 12345. Project ID is not supported.

  • folders:(123 or 456) to find Google Cloud resources that are a descendant of Google Cloud folder with numbers 123 or 456.

  • organization:123 to find Google Cloud resources that are a descendant of Google Cloud organization with number 123.

  • parentFullResourceName:ImportantName to find Google Cloud resources whose parent's name contains ImportantName.

  • parentAssetType:Project to find Google Cloud resources whose parent's asset type contains Project.

  • Important to find Google Cloud resources that contain Important as a word in any of the searchable fields.

  • Impor* to find Google Cloud resources that contain Impor as a prefix in any of the searchable fields.

  • Important location:(us-west1 OR global) to find Google Cloud resources that contain "Important" as a word in any of the searchable fields and are also located in the us-west1 region or the global location.

• ASSET_TYPE: Optional. A comma-separated list of searchable asset types. RE2-compatible regular expressions are supported. If the regular expression does not match any supported asset type, an INVALID_ARGUMENT error is returned.

• ORDER_BY: Optional. A comma-separated list of fields specifying the sorting order of the results. The default order is ascending. Add " DESC" after the field name to indicate descending order. Redundant space characters are ignored. Example: "location DESC, name". Only singular primitive fields in the response are sortable:

  • name

  • assetType

  • project

  • displayName

  • description

  • location

  • createTime

  • updateTime

  • state

  • parentFullResourceName

  • parentAssetType

  All the other fields such as repeated fields (for example, networkTags, kmsKeys), map fields (for example, labels), and struct fields (for example, additionalAttributes) are not supported.

• READ_MASK: Optional. A comma-separated list of fields specifying which fields to be returned in the results. If not specified, all fields except versionedResources are returned. If only "*" is specified, all fields are returned. Examples: "name,location", "name,versionedResources", "*".

To learn more about how to search resources, see Searching resources.

Searching IAM policies

To search Identity and Access Management (IAM) policies, run the following command.

gcloud asset search-all-iam-policies \
  --scope=SCOPE \
  --query="QUERY" \
  --asset-types=ASSET_TYPE_1,ASSET_TYPE_2,... \
  --order-by="ORDER_BY"

Provide the following values:

• SCOPE: Optional. A scope can be a project, folder, or an organization. The search is limited to the IAM resources in this scope. The caller must be granted the cloudasset.assets.searchAllIamPolicies permission on the desired scope. If not specified, the configured project property is used.

  The allowed values are:

  • projects/PROJECT_ID

  • projects/PROJECT_NUMBER

    How to find a Google Cloud project number

    Console

    To find a Google Cloud project number, complete the following steps:

    1. Go to the Dashboard page in the Google Cloud console.

       Go to Dashboard

    2. Click the switcher box in the menu bar.
    3. Select your organization from the Select from box, and then search for your project name.
    4. Click the project name to switch to that project. The project number is shown in the Project info card.

    gcloud CLI

    You can retrieve a Google Cloud project number with the following command:

    gcloud projects describe PROJECT_ID --format="value(projectNumber)"

  • folders/FOLDER_ID

    How to find a Google Cloud folder ID

    Console

    To find a Google Cloud folder ID, complete the following steps:

    1. Go to the Google Cloud console.

       Go to the Google Cloud console

    2. Click the switcher box in the menu bar.
    3. Click the Select from box, and then select your organization.
    4. Search for your folder name. The folder ID is shown next to the folder name.

    gcloud CLI

    You can retrieve a Google Cloud folder ID that's located at the organization level with the following command:

    gcloud resource-manager folders list \
        --organization=$(gcloud organizations describe ORGANIZATION_NAME \
          --format="value(name.segment(1))") \
        --filter='"DISPLAY_NAME":"TOP_LEVEL_FOLDER_NAME"' \
        --format="value(ID)"

    Where TOP_LEVEL_FOLDER_NAME can be a full or partial string match. Remove the --format option to see further information about the found folders.

    To get the ID of a folder within another folder, list the subfolders:

    gcloud resource-manager folders list --folder=FOLDER_ID

  • organizations/ORGANIZATION_ID

    How to find a Google Cloud organization ID

    Console

    To find a Google Cloud organization ID, complete the following steps:

    1. Go to the Google Cloud console.

       Go to the Google Cloud console

    2. Click the switcher box in the menu bar.
    3. Click the Select from box, and then select your organization.
    4. Click the All tab. The organization ID is shown next to the organization name.

    gcloud CLI

    You can retrieve a Google Cloud organization ID with the following command:

    gcloud organizations describe ORGANIZATION_NAME --format="value(name.segment(1))"

• QUERY: Optional. The query statement. See Query syntax for more information. If not specified or empty, Cloud Asset Inventory searches all the IAM policies in the specified scope. The query string is compared against each IAM policy binding, including its principals, roles, and IAM conditions. The returned IAM policies only contain the bindings that match your query. To learn more about IAM policy structure, see Policy structure.

  Examples:

  • policy:[e-mail] to find IAM policy bindings that specify the user [e-mail].

  • policy:roles/compute.admin to find IAM policy bindings that specify the Compute Admin role.

  • policy:comp* to find IAM policy bindings that contain comp as a prefix of any word in the binding.

  • policy.role.permissions:storage.buckets.update to find IAM policy bindings that specify a role containing the storage.buckets.update permission. If callers don't have iam.roles.get access to a role's included permissions, policy bindings that specify this role will be dropped from the search results.

  • policy.role.permissions:upd* to find IAM policy bindings that specify a role containing upd as a prefix of any word in the role permission. Note that if callers don't have iam.roles.get access to a role's included permissions, policy bindings that specify this role will be dropped from the search results.

  • resource:organizations/123456 to find IAM policy bindings that are set on "organizations/123456".

  • resource=//cloudresourcemanager.googleapis.com/projects/myproject to find IAM policy bindings that are set on the project named myproject.

  • Important to find IAM policy bindings that contain Important as a word in any of the searchable fields (except for the included permissions).

  • resource:(instance1 OR instance2) policy:amy to find IAM policy bindings that are set on resources instance1 or instance2 and also specify user amy.

  • roles:roles/compute.admin to find IAM policy bindings that specify the Compute Admin role.

  • memberTypes:user to find IAM policy bindings that contain the user principal type.

• ASSET_TYPE: Optional. A comma-separated list of searchable asset types that the IAM policies are attached to. If empty, Cloud Asset Inventory searches the IAM policies that are attached to all searchable asset types. Regular expressions are supported. If the regular expression does not match any supported asset type, an INVALID_ARGUMENT error is returned.

• ORDER_BY: Optional. A comma-separated list of fields specifying the sorting order of the results. The default order is ascending. Add " DESC" after the field name to indicate descending order. Redundant space characters are ignored. Example: "location DESC, name". Only singular primitive fields in the response are sortable:

  • resource

  • assetType

  • project

  All the other fields such as repeated fields (for example, folders) and non-primitive fields (for example, policy) are not supported.

To learn more about how to search IAM policies, see Searching IAM policies.

Exporting an asset snapshot to Cloud Storage

To export all the asset metadata at a given timestamp to a file in a Cloud Storage bucket, complete the following steps.

1. Create a new bucket if you don't have one already.

2. To export asset metadata in your project, run the following command. This command stores the exported snapshot in a file at gs://BUCKET_NAME/FILENAME.

   Projects

   gcloud asset export \
       --project=PROJECT_ID \
       --billing-project=BILLING_PROJECT_ID \
       --content-type=CONTENT_TYPE \
       --snapshot-time="SNAPSHOT_TIME" \
       --output-path="gs://BUCKET_NAME/FILENAME"
   

   Provide the following values:

   • PROJECT_ID: The ID of the project whose metadata you want to export.

   • BILLING_PROJECT_ID: Optional. The project ID that the default Cloud Asset Inventory service agent is in that has permission to write to your Cloud Storage bucket. Not required if you've already switched to the billing project with gcloud config set project. Read more about billing projects.

   • CONTENT_TYPE: Optional. The content type of the asset to export. If no content type is specified, only the asset name is returned. Read more about content types.

   • SNAPSHOT_TIME: Optional. The time at which you want to take a snapshot of your assets. The value must be the current time or a time no more than 35 days in the past. When not provided, a snapshot is taken at the current time. For information on time formats, see gcloud topic datetimes.

   • BUCKET_NAME: The name of the Cloud Storage bucket to write to.

   • FILENAME: The file in your Cloud Storage bucket to write to.

   Folders

   gcloud asset export \
       --folder=FOLDER_ID \
       --billing-project=BILLING_PROJECT_ID \
       --content-type=CONTENT_TYPE \
       --snapshot-time="SNAPSHOT_TIME" \
       --output-path="gs://BUCKET_NAME/FILENAME"
   

   Provide the following values:

   • FOLDER_ID: The ID of the folder whose metadata you want to export.

     How to find a Google Cloud folder ID

     Console

     To find a Google Cloud folder ID, complete the following steps:

     1. Go to the Google Cloud console.

        Go to the Google Cloud console

     2. Click the switcher box in the menu bar.
     3. Click the Select from box, and then select your organization.
     4. Search for your folder name. The folder ID is shown next to the folder name.

     gcloud CLI

     You can retrieve a Google Cloud folder ID that's located at the organization level with the following command:

     gcloud resource-manager folders list \
         --organization=$(gcloud organizations describe ORGANIZATION_NAME \
           --format="value(name.segment(1))") \
         --filter='"DISPLAY_NAME":"TOP_LEVEL_FOLDER_NAME"' \
         --format="value(ID)"

     Where TOP_LEVEL_FOLDER_NAME can be a full or partial string match. Remove the --format option to see further information about the found folders.

     To get the ID of a folder within another folder, list the subfolders:

     gcloud resource-manager folders list --folder=FOLDER_ID

   • BILLING_PROJECT_ID: Optional. The project ID that the default Cloud Asset Inventory service agent is in that has permission to write to your Cloud Storage bucket. Not required if you've already switched to the billing project with gcloud config set project. Read more about billing projects.

   • CONTENT_TYPE: Optional. The content type of the asset to export. If no content type is specified, only the asset name is returned. Read more about content types.

   • SNAPSHOT_TIME: Optional. The time at which you want to take a snapshot of your assets. The value must be the current time or a time no more than 35 days in the past. When not provided, a snapshot is taken at the current time. For information on time formats, see gcloud topic datetimes.

   • BUCKET_NAME: The name of the Cloud Storage bucket to write to.

   • FILENAME: The file in your Cloud Storage bucket to write to.

   Organizations

   gcloud asset export \
       --organization=ORGANIZATION_ID \
       --billing-project=BILLING_PROJECT_ID \
       --content-type=CONTENT_TYPE \
       --snapshot-time="SNAPSHOT_TIME" \
       --output-path="gs://BUCKET_NAME/FILENAME"
   

   Provide the following values:

   • ORGANIZATION_ID: The ID of the organization whose metadata you want to export.

     How to find a Google Cloud organization ID

     Console

     To find a Google Cloud organization ID, complete the following steps:

     1. Go to the Google Cloud console.

        Go to the Google Cloud console

     2. Click the switcher box in the menu bar.
     3. Click the Select from box, and then select your organization.
     4. Click the All tab. The organization ID is shown next to the organization name.

     gcloud CLI

     You can retrieve a Google Cloud organization ID with the following command:

     gcloud organizations describe ORGANIZATION_NAME --format="value(name.segment(1))"

   • BILLING_PROJECT_ID: Optional. The project ID that the default Cloud Asset Inventory service agent is in that has permission to write to your Cloud Storage bucket. Not required if you've already switched to the billing project with gcloud config set project. Read more about billing projects.

   • CONTENT_TYPE: Optional. The content type of the asset to export. If no content type is specified, only the asset name is returned. Read more about content types.

   • SNAPSHOT_TIME: Optional. The time at which you want to take a snapshot of your assets. The value must be the current time or a time no more than 35 days in the past. When not provided, a snapshot is taken at the current time. For information on time formats, see gcloud topic datetimes.

   • BUCKET_NAME: The name of the Cloud Storage bucket to write to.

   • FILENAME: The file in your Cloud Storage bucket to write to.

3. Optional: To check the status of the export, run the following command. It is displayed in the gcloud CLI after running the export command.

   gcloud asset operations describe OPERATION_PATH
   

Viewing the asset snapshot

To view the asset snapshot after you've exported it to Cloud Storage, complete the following steps.

1. Go to the Cloud Storage Buckets page in the Google Cloud console.

   Go to Buckets

2. Open the file where you exported your metadata.

The export file lists the assets and their resource names.

What's next

• Learn more about Cloud Asset Inventory.

• Learn how to get the history of assets.