Frequently asked questions and troubleshooting

Is Cloud Asset Inventory a global service?

Yes. Cloud Asset API has no dependence on location. It has a global endpoint, which serves the metadata of all supported regional and global assets in Cloud Asset Inventory. Cloud Asset API can be accessed in any zone.

What kind of data consistency does Cloud Asset Inventory provide?

Cloud Asset Inventory provides eventual consistency on current data and best-effort consistency on historical data. Despite the low chance in practice, it is possible for Cloud Asset Inventory to miss some updates of an asset in the past.

Why don't I have permission to use the Cloud Asset API?

An error is returned if you don't have permission to export assets or get the history on an organization, project, or folder.

For example, if you don't have permission, running the following command:

curl -X POST \
     -H "X-Goog-User-Project: BILLING_PROJECT_ID" \
     -H "Authorization: Bearer $(gcloud auth print-access-token)" \
     -H "Content-Type: application/json" \
     -d '{
          "outputConfig": {
            "gcsDestination": {
              "uri": "gs://BUCKET_NAME/FILENAME"
            }
          }
         }' \
         https://cloudasset.googleapis.com/v1/projects/PROJECT_ID:exportAssets

Returns the following error:

{
  "error": {
    "code": 403,
    "message": "The caller does not have permission",
    "status": "PERMISSION_DENIED",
    "details": [
      {
        "@type": "type.googleapis.com/google.rpc.DebugInfo",
        "detail": "[ORIGINAL ERROR] generic::permission_denied: Request
        denied by Cloud IAM."
      }
    ]
  }
}

To address this issue, request access from your project, folder, or organization admin. Depending on the assets you are trying to export or get history for, you need one of the following roles or other roles that include the required Cloud Asset API permissions:

  • cloudasset.viewer

  • cloudasset.owner

For more information on roles and permissions, see Understanding roles.

For more information on access control options for Cloud Asset APIs, see Access Control.

Why are my exports returning a permission denied error?

Unless you specify otherwise, Cloud Asset Inventory uses the default Cloud Asset Inventory service account in the active project to manage resources like Pub/Sub topics, Cloud Storage buckets, and BigQuery tables. This service account is created the first time you call the Cloud Asset Inventory API from a project, and by default has permission to manage these resources so long as they are located in the same project.

You can receive permission denied errors in the following situations:

  • When using the REST API, which doesn't set an active project and so Cloud Asset Inventory doesn't know which service account to use.

  • When using the gcloud CLI from a different project than where the Pub/Sub topic, Cloud Storage bucket, or BigQuery table is located. This means that the active project's default Cloud Asset Inventory service account is used to perform the task (if it exists), and might not have permissions to write to the other project's resources.

To make sure the correct service account is used when making requests to export to Pub/Sub topics, Cloud Storage buckets, or BigQuery tables, you can specify the project ID that contains the correct default Cloud Asset Inventory service account. If you are exporting from one project to another, you also need to grant specific roles to the service account.

gcloud

For the gcloud CLI, add the --billing-project flag to your command to specify the project ID that contains the correct service account:

--billing-project=BILLING_PROJECT_ID

Alternatively, you can set the billing project before you run commands with the gcloud CLI. First, check if the billing project is different from the core project:

gcloud config list

Then if you need to, set the billing project:

gcloud config set billing/quota_project BILLING_PROJECT_ID

Provide the following values:

  • BILLING_PROJECT_ID: A project ID that has the Cloud Asset Inventory API is enabled, and a service account with permissions to manage your target Pub/Sub topic, Cloud Storage bucket, or BigQuery table.

REST

For the REST API, add the X-Goog-User-Project header to specify the project ID that contains the correct service account. When using curl, you set the header with the -H flag:

-H "X-Goog-User-Project: BILLING_PROJECT_ID"

Provide the following values:

  • BILLING_PROJECT_ID: A project ID that has the Cloud Asset Inventory API is enabled, and a service account with permissions to manage your target Pub/Sub topic, Cloud Storage bucket, or BigQuery table.

Export asset metadata from one project to another

To export asset metadata from one project, PROJECT_A, to another, PROJECT_B, you must give the default Cloud Asset Inventory service account in PROJECT_A access to the resources in PROJECT_B. This enables two things:

  • You can export asset metadata from PROJECT_A into a Pub/Sub topic, Cloud Storage bucket, or BigQuery table located in PROJECT_B.

  • You can use PROJECT_A to export asset metadata from PROJECT_B into a Pub/Sub topic, Cloud Storage bucket, or BigQuery table located in PROJECT_B.

To export asset metadata from one project into another, complete the following instructions:

  1. Make sure that the Cloud Asset Inventory API is enabled in the project you want to run your request from, PROJECT_A.

  2. Make at least one call to the Cloud Asset Inventory API in PROJECT_A to create the default Cloud Asset Inventory service account. Alternatively, you can create it manually:

    gcloud beta services identity create \
        --service=cloudasset.googleapis.com \
        --project=PROJECT_A_ID
    gcloud projects add-iam-policy-binding PROJECT_A_ID \
        --member=serviceAccount:service-PROJECT_A_NUMBER@gcp-sa-cloudasset.iam.gserviceaccount.com \
        --role=roles/cloudasset.serviceAgent
    

    How to find a Google Cloud project number

    Console

    To find a Google Cloud project number, complete the following steps:

    1. Go to the Dashboard page in the Google Cloud console.

      Go to Dashboard

    2. Click the switcher box in the menu bar.
    3. Select your organization from the Select from box, and then search for your project name.
    4. Click the project name to switch to that project. The project number is shown in the Project info card.

    gcloud CLI

    You can retrieve a Google Cloud project number with the following command:

    gcloud projects describe PROJECT_ID --format="value(projectNumber)"

  3. Grant the correct permissions to the service account.

    • To publish to a feed through Pub/Sub, grant the roles/pubsub.publisher role to the service account on the topic:

      gcloud pubsub topics add-iam-policy-binding projects/PROJECT_B_ID/topics/TOPIC_ID \
          --member=serviceAccount:service-PROJECT_A_NUMBER@gcp-sa-cloudasset.iam.gserviceaccount.com \
          --role=roles/pubsub.publisher
      
    • To write to a Cloud Storage bucket, grant the roles/storage.admin role to the service account on the bucket:

      gsutil iam ch \
        serviceAccount:service-PROJECT_A_NUMBER@gcp-sa-cloudasset.iam.gserviceaccount.com:objectCreator \
        gs://BUCKET_NAME
      
    • To write to a BigQuery table, grant the roles/bigquery.dataEditor and roles/bigquery.user roles to the service account on the project:

      gcloud projects add-iam-policy-binding PROJECT_B_ID \
          --member=serviceAccount:service-PROJECT_A_NUMBER@gcp-sa-cloudasset.iam.gserviceaccount.com \
          --role=roles/bigquery.user
      gcloud projects add-iam-policy-binding PROJECT_B_ID \
          --member=serviceAccount:service-PROJECT_A_NUMBER@gcp-sa-cloudasset.iam.gserviceaccount.com \
          --role=roles/bigquery.dataEditor
      

When making Cloud Asset Inventory requests, make sure to specify PROJECT_A as the project you want to use. To do so for the gcloud CLI, set the --billing-project flag to PROJECT_A_ID. For REST, set the X-Goog-User-Project header to PROJECT_A_ID.

Why is the Cloud Asset API result stale?

Data freshness in the Cloud Asset API is on a best-effort basis. While almost all asset updates are available to clients in minutes, in rare cases it's possible the result of the Cloud Asset API methods won't include the most recent asset updates.

Why are temporary files output after running ExportAssets?

The ExportAssets operation might create temporary files in the output folder. Don't remove these temporary files while the operation is in progress. After the operation is complete, the temporary files are removed automatically.

If the temporary files remain, you can safely remove them after the ExportAssets operation is complete.

Why is my Google Cloud CLI or Cloud Shell credential rejected?

If a user project in a request is sent to cloudasset.googleapis.com from the Google Cloud CLI or Cloud Shell, you receive an error message like the following:

Your application has authenticated using end user credentials from the
Google Cloud CLI or Cloud Shell which are not supported by the
cloudasset.googleapis.com. We recommend that most server applications
use service accounts instead. For more information about service accounts
and how to use them in your application, see
https://cloud.google.com/docs/authentication/.

To fix this issue, set the user project to the Cloud Asset API-enabled user's project ID. This can be done by specifying HTTP header X-Goog-User-Project in the HTTP request.

If you're using curl, this can be done by adding the following parameter:

-H "X-Goog-User-Project: PROJECT_ID"

If you're using the gcloud CLI, specify flag --billing-project PROJECT_ID along with the gcloud asset command, or use the following command:

gcloud config set billing/quota_project PROJECT_ID

Why do I see different ancestors for the same assets?

When calling the Cloud Asset API to get different metadata types, such as RESOURCE metadata and IAM POLICY metadata for the same asset, it is possible that the ancestors field is inconsistent across content types. This is because there are different data ingestion schedules for each content type, and until the ingestion process is complete, they might be inconsistent. Check the update_time field to ensure that the asset has the most up-to-date information.

Please contact us if the inconsistency lasts for more than 24 hours.

How frequently should I call the ExportAssets API?

We recommend calling the ExportAssets API for the same project, folder, or organization in a sequential manner; For example, issue the second call after the previous one completes. If you want to capture asset updates in real time, consider using real-time notifications.

Receive duplicate asset updates

After setting up real-time notifications, it is possible for you to receive duplicate asset updates in your Pub/Sub topic. This is caused by an automatic attempt to retry the delivery, as Pub/Sub does not guarantee at-least-once delivery.

Why didn't I receive notifications for project deletions?

When you shut down a project, you have 30 days to undo the operation. The deleted field in the notification is not set until the project is permanently deleted. To monitor projects that are pending deletion, you can set a feed with a condition on the project's lifecycleState, for example temporal_asset.asset.resource.data.lifecycleState == "DELETE_REQUESTED".

How do I retrieve a resource's JSON representation with the SearchAllResources API?

By default, SearchAllResources returns the following standard fields when read_mask isn't specified:

  • name

  • assetType

  • project

  • folders

  • organization

  • displayName

  • description

  • location

  • labels

  • networkTags

  • kmsKeys

  • createTime

  • updateTime

  • state

  • additionalAttributes

  • parentFullResourceName

  • parentAssetType

If you want to retrieve all fields in the resource metadata in addition to the fields listed above, you can specify the read_mask flag (--read-mask in gcloud) in the search request.

A read_mask is a list of fields, separated by commas, that you want returned in your results. Some fields are too large, such as versionedResources and attachedResources, and so are not included in the results by default. To include these fields, you can specify them in the read_mask; or use "*" to include all available fields. Examples of read_mask values include: "name,location", "name,versionedResources", and "*".

Here's a gcloud example:

gcloud asset search-all-resources \
    --scope=organizations/123456 \
    --query="state=RUNNING" \
    --asset-types=compute.googleapis.com/Instance \
    --read-mask="name,versionedResources"