Export asset metadata by using Cloud Asset Inventory

Export asset metadata by using Cloud Asset Inventory

This quickstart shows you how to export asset metadata at a point in time using Cloud Asset Inventory and the Google Cloud CLI gcloud asset commands.

Before you begin

Before you can start working with Cloud Asset Inventory, you must enable the Cloud Asset Inventory API, the Google Cloud CLI, and assign permissions. The Google Cloud CLI interacts with Cloud Asset Inventory and other Google Cloud services. Learn more about the gcloud CLI.

Enabling the Cloud Asset Inventory API and Google Cloud CLI

1. Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.

2. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

3. Enable the required API.

   Enable the API

4. Install and initialize the Google Cloud CLI.

5. In the Google Cloud console, on the project selector page, select or create a Google Cloud project.

   Note: If you don't plan to keep the resources that you create in this procedure, create a project instead of selecting an existing project. After you finish these steps, you can delete the project, removing all resources associated with the project.

   Go to project selector

6. Enable the required API.

   Enable the API

7. Install and initialize the Google Cloud CLI.

Configuring permissions

To call the Cloud Asset Inventory API, you must first configure permissions.

Searching assets

1. To search resource metadata, run the following gcloud asset search-all-resources command.

    gcloud asset search-all-resources \
       --scope SCOPE \
       --query QUERY \
       --asset-types ASSET_TYPES,… \
       --order-by ORDER_BY \
       --read-mask READ_MASK
   

   Where all of the following flags are optional:

   • (Optional) SCOPE: A scope can be a project, a folder, or an organization. The search is limited to the Google Cloud resources within this scope. The caller must be granted the cloudasset.assets.searchAllResources permission on the desired scope. If not specified, the configured project property will be used. To find the configured project, run: gcloud config get-value project. To change the setting, run: gcloud config set project PROJECT_ID.

     The allowed values are:

     • projects/PROJECT_ID (e.g., "projects/foo-bar")
     • projects/PROJECT_NUMBER (e.g., "projects/12345678")
     • folders/FOLDER_NUMBER (e.g., "folders/1234567")
     • organizations/ORGANIZATION_NUMBER (e.g., "organizations/123456")

   • (Optional) QUERY: The query statement. See how to construct a query for more information. If not specified or empty, it will search all the resources within the specified scope.

     Examples:

     • name:Important to find Google Cloud resources whose name contains "Important" as a word.
     • name=Important to find the Google Cloud resource whose name is exactly "Important".
     • displayName:Impor* to find Google Cloud resources whose display name contains "Impor" as a prefix.
     • location:us-west* to find Google Cloud resources whose location contains both "us" and "west" as prefixes.
     • labels:prod to find Google Cloud resources whose labels contain "prod" as a key or value.
     • labels.env:prod to find Google Cloud resources that have a label "env" and its value is "prod".
     • labels.env:* to find Google Cloud resources that have a label "env".
     • kmsKey:key to find Google Cloud resources encrypted with a customer-managed encryption key whose name contains "key" as a word.
     • state:ACTIVE to find Google Cloud resources whose state contains "ACTIVE" as a word.
     • NOT state:ACTIVE to find Google Cloud resources whose state doesn't contain "ACTIVE" as a word.
     • createTime<1609459200 or createTime<2021-01-01 or createTime<"2021-01-01T00:00:00" to find Google Cloud resources that were created before "2021-01-01 00:00:00 UTC". 1609459200 is the epoch timestamp of "2021-01-01 00:00:00 UTC" in seconds.
     • updateTime>1609459200 or updateTime>2021-01-01 or updateTime>"2021-01-01T00:00:00" to find Google Cloud resources that were updated after "2021-01-01 00:00:00 UTC". 1609459200 is the epoch timestamp of "2021-01-01 00:00:00 UTC" in seconds.
     • project:12345 to find Google Cloud resources that belong to a Google Cloud project with number 12345. Project id is not supported.
     • folders:(123 or 456) to find Google Cloud resources that are a descendant of Google Cloud folder with numbers 123 or 456.
     • organization:123 to find Google Cloud resources that are a descendant of Google Cloud organization with number 123.
     • parentFullResourceName:ImportantName to find Google Cloud resources whose parent's name contains ImportantName.
     • parentAssetType:Project to find Google Cloud resources whose parent's asset type contains Project.
     • Important to find Google Cloud resources that contain "Important" as a word in any of the searchable fields.
     • Impor* to find Google Cloud resources that contain "Impor" as a prefix in any of the searchable fields.
     • Important location:(us-west1 OR global) to find Google Cloud resources that contain "Important" as a word in any of the searchable fields and are also located in the "us-west1" region or the "global" location.

   • (Optional) ASSET_TYPES: A list of asset types that this request searches for. If empty, it will search all the searchable asset types. Regular expressions are also supported. For example:

     • "compute.googleapis.com.*" snapshots resources whose asset type starts with "compute.googleapis.com".
     • ".*Instance" snapshots resources whose asset type ends with "Instance".
     • ".*Instance.*" snapshots resources whose asset type contains "Instance".

     See RE2 for all supported regular expression syntax. If the regular expression does not match any supported asset type, an INVALID_ARGUMENT error will be returned.

   • (Optional) ORDER_BY: A comma-separated list of fields specifying the sorting order of the results. The default order is ascending. Add " DESC" after the field name to indicate descending order. Redundant space characters are ignored. Example: "location DESC, name". Only singular primitive fields in the response are sortable:

     • name
     • assetType
     • project
     • displayName
     • description
     • location
     • kmsKey
     • createTime
     • updateTime
     • state
     • parentFullResourceName
     • parentAssetType

     All the other fields such as repeated fields (e.g., networkTags), map fields (e.g., labels), and struct fields (e.g., additionalAttributes) are not supported.

   • (Optional) READ_MASK: A comma-separated list of fields specifying which fields to be returned in the results. If not specified, all fields except versionedResources will be returned. If only "*" is specified, all fields will be returned. Examples: "name,location", "name,versionedResources", "*".

   To learn more about how to search resources, see Searching resources.

2. To search IAM policies, run the following gcloud asset search-all-iam-policies command.

    gcloud asset search-all-iam-policies \
       --scope SCOPE \
       --query QUERY \
       --asset-types ASSET_TYPES,… \
       --order-by ORDER_BY
   

   Where:

   • (Optional) SCOPE: A scope can be a project, a folder, or an organization. The search is limited to the Identity and Access Management (IAM) policies within this scope. The caller must be granted the cloudasset.assets.searchAllIamPolicies permission on the desired scope. If not specified, the configured project property will be used. To find the configured project, run: gcloud config get-value project. To change the setting, run: gcloud config set project PROJECT_ID.

     The allowed values are:

     • projects/PROJECT_ID (e.g., "projects/foo-bar")
     • projects/PROJECT_NUMBER (e.g., "projects/12345678")
     • folders/FOLDER_NUMBER (e.g., "folders/1234567")
     • organizations/ORGANIZATION_NUMBER (e.g., "organizations/123456")

   • (Optional) QUERY: The query statement. See how to construct a query for more information. If not specified or empty, it will search all the IAM policies within the specified scope. Note that the query string is compared against each IAM policy binding, including its principals, roles, and IAM conditions. The returned IAM policies will only contain the bindings that match your query. To learn more about the IAM policy structure, see IAM policy doc.

     Examples:

     • policy:[e-mail] to find IAM policy bindings that specify user "[e-mail]".
     • policy:roles/compute.admin to find IAM policy bindings that specify the Compute Admin role.
     • policy:comp* to find IAM policy bindings that contain "comp" as a prefix of any word in the binding.
     • policy.role.permissions:storage.buckets.update to find IAM policy bindings that specify a role containing the "storage.buckets.update" permission. Note that if callers don't have iam.roles.get access to a role's included permissions, policy bindings that specify this role will be dropped from the search results.
     • policy.role.permissions:upd* to find IAM policy bindings that specify a role containing "upd" as a prefix of any word in the role permission. Note that if callers don't have iam.roles.get access to a role's included permissions, policy bindings that specify this role will be dropped from the search results.
     • resource:organizations/123456 to find IAM policy bindings that are set on "organizations/123456".
     • resource=//cloudresourcemanager.googleapis.com/projects/myproject to find IAM policy bindings that are set on the project named "myproject".
     • Important to find IAM policy bindings that contain "Important" as a word in any of the searchable fields (except for the included permissions).
     • resource:(instance1 OR instance2) policy:amy to find IAM policy bindings that are set on resources "instance1" or "instance2" and also specify user "amy".
     • roles:roles/compute.admin to find IAM policy bindings that specify the Compute Admin role.
     • memberTypes:user to find IAM policy bindings that contain the "user" principal type.

   • (Optional) ASSET_TYPES: A list of asset types that the Identity and Access Management policies are attached to. If empty, it will search the Identity and Access Management policies that are attached to all the searchable asset types. Regular expressions are also supported. For example:

     • "compute.googleapis.com.*" snapshots IAM policies attached to asset type starts with "compute.googleapis.com".
     • ".*Instance" snapshots IAM policies attached to asset type ends with "Instance".
     • ".*Instance.*" snapshots IAM policies attached to asset type contains "Instance".

     See RE2 for all supported regular expression syntax. If the regular expression does not match any supported asset type, an INVALID_ARGUMENT error will be returned.

   • (Optional) ORDER_BY: A comma-separated list of fields specifying the sorting order of the results. The default order is ascending. Add " DESC" after the field name to indicate descending order. Redundant space characters are ignored. Example: "assetType DESC, resource". Only singular primitive fields in the response are sortable:

     • resource
     • assetType
     • project

     All the other fields such as repeated fields (e.g., folders) and non-primitive fields (e.g., policy) are not supported.

   To learn more about how to search IAM policies, see Searching IAM policies.

Exporting an asset snapshot to Cloud Storage

To export all the asset metadata at a given timestamp to a Cloud Storage file, complete the following steps.

Note: The Cloud Storage bucket you use to store exported metadata must be in the project where Cloud Asset Inventory API is enabled and from which you're running the export.

1. Create a new bucket if your project doesn't have an existing Cloud Storage bucket that is available to store exported data.

2. To export asset metadata within your project, run the following command. This command stores the exported snapshot in a Cloud Storage bucket at gs://YOUR_BUCKET/NEW_FILE.

   gcloud asset export \
      --content-type CONTENT_TYPE \
      --project PROJECT_ID \
      --snapshot-time SNAPSHOT_TIME \
      --output-path "gs://YOUR_BUCKET/NEW_FILE"
   

   Where:

   • (Optional) CONTENT_TYPE: The content type of the asset to export. If no content type is specified, only the asset name will be returned. Example types include resource, which exports resource metadata; iam-policy, which exports the IAM policy for each child asset of the specified resource, and so on. Other options include org-policy, access-policy, and os-inventory. See the gcloud asset export optional flags section for more information.
   • PROJECT_ID: The ID of the project that is having its metadata exported. This project can be either the project where Cloud Asset Inventory API is enabled and from which you're running the export, or a different project.
   • (Optional) SNAPSHOT_TIME: The value must be the current time or a time in the past at which you want to take a snapshot of your assets. By default, a snapshot is taken at the current time. See gcloud topic datetimes for information on time formats, for example: --snapshot-time 2021-05-25T10:49:41Z Note: If you specify a snapshot time, it cannot exceed 35 days in the past.

3. To export the assets of an organization or folder, you can use one of the following flags in place of the --project flag:

   • --organization=ORGANIZATION_ID
   • --folder=FOLDER_ID

4. (Optional) To check the status of the export, run the following command. It is displayed in the gcloud CLI after running the export command.

   gcloud asset operations describe projects/PROJECT_ID/operations/ExportAssets/CONTENT_TYPE/OPERATION_NUMBER
   

Viewing an asset snapshot

To view an asset snapshot after you've exported it to Cloud Storage, complete the following steps.

1. Go to the Cloud Storage Browser page.
   Open the Cloud Storage Browser page

2. Open the file where you exported your metadata.

The export file lists the assets and their resource names.

What's next

• Learn more about Cloud Asset Inventory.
• Learn how to get the history of assets.