This page describes how to create and manage inventory report configurations and generated inventory reports, so you can get a high level summary of all your objects' metadata in a given bucket. For an overview of inventory reports, see the Storage Insights inventory reports overview documentation.
Before you begin
Get required roles
In order to get the required permissions for creating and managing inventory reports, ask your administrator to grant you the following IAM roles on the project or source and destination buckets with which you'll be managing inventory reports.
To create and manage inventory report configurations:
roles/storage.admin
on the source bucket and destination bucketroles/storageinsights.admin
on the project
To read and download inventory reports:
roles/storage.objectViewer
on the destination bucketroles/storageinsights.viewer
on the project
These predefined roles contain the permissions required to create and manage inventory reports and configurations. To see the exact permissions that are required, expand the Required permissions section:
Required permissions
To create and manage inventory report configurations:
storage.buckets.get
on the source bucketstorage.objects.list
on the source bucketstorage.buckets.getObjectInsights
on the source bucketstorage.buckets.get
on the destination bucketstorage.objects.create
on the destination bucketstorageinsights.reportConfigs.delete
on the projectstorageinsights.reportConfigs.get
on the projectstorageinsights.reportConfigs.create
on the projectstorageinsights.reportConfigs.list
on the projectstorageinsights.reportConfigs.update
on the project
To read and download inventory reports:
storage.objects.get
on the destination bucketstorageinsights.reportDetails.get
on the projectstorageinsights.reportDetails.list
on the project
You might also be able to get these permissions with other predefined roles. To see which roles are associated with which permissions, refer to IAM roles for Cloud Storage.
For instructions on using roles to control access to buckets, see Use IAM. For instructions on using roles to control access to projects, see Manage access.
Enable the Storage Insights API
Console
To enable the storageinsights.googleapis.com
API, follow the
instructions in Enabling services.
Command line
To enable the Storage Insights API in your current project, run the following command:
gcloud services enable storageinsights.googleapis.com
For more details on how to enable services for a Google Cloud project, see Enabling and disabling services.
REST APIs
JSON API
Use the Google Cloud console or the Google Cloud CLI to enable the Storage Insights API.
Create an inventory report configuration
Console
To create an inventory report configuration, complete the following steps:
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the bucket you'd like to use as the source bucket.
On the Bucket details page, click the Inventory reports tab.
Click Create report configuration.
In the Identify your report configuration section, create a display name for the inventory report configuration. The display name can contain up to 256 characters.
In the Choose metadata fields section, select the metadata fields you want to include in your inventory reports.
Click Continue.
In the Choose a file format section, select the file format in which you'd like to generate inventory reports.
In the Choose a destination bucket section, select the bucket you'd like to use as a destination bucket.
In the Enter destination path (optional) section, you can optionally specify a destination path where inventory reports will be generated.
In the Scheduling options section, specify the frequency, start date, and end date for generating reports.
Click Create.
The Report configuration details page is displayed. The metadata of generated inventory reports will appear on the page.
Command line
To create an inventory report configuration, run the
gcloud storage insights inventory-reports create
command:gcloud storage insights inventory-reports create SOURCE_BUCKET_URL \ --csv-separator=SEPARATOR \ --csv-delimiter=DELIMITER \ --[no]-csv-header \ --parquet \ --display-name=DISPLAY_NAME \ --destination=DESTINATION_PATH \ --metadata-fields=METADATA_FIELD \ --schedule-starts=START_DATE \ --schedule-repeats=FREQUENCY \ --schedule-repeats-until=END_DATE
Replace:
SOURCE_BUCKET_URL
with the URL of the source bucket. For example,gs://my_example_source_bucket
.SEPARATOR
with the character used to separate the records in the inventory report CSV file. Must be either\n
or\r\n
. Default value is\n
. Optional. If the--csv-separator
flag is used,--parquet
cannot be used.DELIMITER
with the delimiter that separates the fields in the inventory report CSV file. The value can include one character and cannot be the same value asSEPARATOR
. Default value is,
. Optional. If--csv-delimiter
is used,--parquet
cannot be used.--[no]-csv-header
with the flag that indicates whether or not headers are included in the inventory report CSV file. Use--csv-header
to include headers and--no-csv-header
to exclude headers. If either flag is used,--parquet
cannot be used.DISPLAY_NAME
with the editable name of the inventory report configuration. Optional.--parquet
with the flag that generates inventory reports in Apache Parquet format instead of CSV. If used,--csv-delimiter
,--csv-separator
, and--[no-]csv-header
cannot be used.DESTINATION_PATH
with the bucket or folder in the bucket where inventory reports will be generated. For example,gs://my_example_destination_bucket
orgs://my_example_destination_bucket/path/to/inventory/report
.When you specify a path to a folder within a bucket, you can use keywords in the path, which are substituted with their corresponding values when the report is generated. This enables you to generate reports in a Hive partitioned format so you can load or query the data in BigQuery without additional processing.
METADATA_FIELD
with a comma-separated list of metadata fields you want included in the inventory report.START_DATE
with the UTC date on which you want to start generating inventory reports. For example,2022-01-15
.FREQUENCY
with how often you want inventory reports to be generated. Values aredaily
orweekly
.END_DATE
with the UTC date after which you want to stop generating inventory reports. Must be a value after theSTART_DATE
. For example, if you specify2022-02-15
, inventory reports will no longer be generated as of February 16, 2022.
Client libraries
For more information, see the
Cloud Storage Java API
reference documentation.
To authenticate to Cloud Storage, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Java
REST APIs
JSON API
To create an inventory report configuration, complete the following steps:
Have gcloud CLI installed and initialized, in order to generate an access token for the
Authorization
header.Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the
Authorization
header.Create a JSON file that contains the following information, where all fields are required unless noted otherwise:
{ "display_name": "DISPLAY_NAME", "frequency_options": { "frequency": "FREQUENCY", "start_date": "START_DATE", "end_date": "END_DATE", }, "csv_options": { "record_separator": "RECORD_SEPARATOR", "delimiter": "DELIMITER", "header_required": HEADER_REQUIRED }, "object_metadata_report_options": { "metadata_fields": [ "METADATA_FIELD", ... ], "storage_filters": { "bucket": "SOURCE_BUCKET_NAME" }, "storage_destination_options": { "bucket": "DESTINATION_BUCKET_NAME", "destination_path": "DESTINATION_PATH" } } }
Replace:
DISPLAY_NAME
with the editable display name of the inventory report configuration. Can contain up to 256 characters.FREQUENCY
with the frequency for generating inventory reports. Values areDAILY
orWEEKLY
.START_DATE
with the UTC date on which you want to start generating inventory reports. Cannot be the current date or any past date. Value must be an object containing the keysday
,month
, andyear
. For example:{"day": 15, "month": 8, "year": 2022}
.END_DATE
with the UTC date after which you want to stop generating inventory reports. Value must be an object containing the keysday
,month
, andyear
. For example, if you specify{"day": 15, "month": 9, "year": 2022}
, inventory reports are no longer generated as of September 16, 2022.RECORD_SEPARATOR
with the character used to separate the records in the inventory report CSV file. Must be either\n
or\r\n
. Default value is\n
. This field is optional.DELIMITER
with the delimiter that separates the fields in the inventory report CSV file. The value can include one character and cannot be the same as theRECORD_SEPARATOR
. Default value is,
. This field is optional.HEADER_REQUIRED
with the boolean that indicates whether or not headers are included in the CSV file. This field is optional.SOURCE_BUCKET_NAME
with the name of the source bucket that contains the objects you want to generate inventory reports for. For example,my_example_bucket
.METADATA_FIELD
with a comma-separated list of metadata fields you want included in the inventory report.DESTINATION_BUCKET_NAME
with the name of the destination bucket where generated inventory reports are generated and stored. For example,my_example_destination_bucket
.DESTINATION_PATH
with the path in the destination bucket where generated inventory reports are generated. This field is optional.When you specify a path to a folder within a bucket, you can use keywords in the path, which are substituted with their corresponding values when the report is generated. This enables you to generate reports in a Hive partitioned format so you can load or query the data in BigQuery without additional processing.
For example, the following code sample creates an inventory report configuration named "Example inventory report configuration" that generates reports at a weekly frequency:
{ "display_name": "Example inventory report configuration", "frequency_options": { "frequency": "WEEKLY", "start_date": { "day": 15, "month": 8, "year": 2022 }, "end_date": { "day": 15, "month": 9, "year": 2022 }, }, "csv_options": { "record_separator": "\n", "delimiter": ",", "header_required": true }, "object_metadata_report_options": { "metadata_fields": [ "project", "name", "bucket" ], "storage_filters": { "bucket": "example_source_bucket" }, "storage_destination_options": { "bucket": "example_destination_bucket" } } }
To apply the inventory report configuration, use
cURL
to call the JSON API with anInsert
ReportConfig request:curl -X POST --data-binary @JSON_FILE_NAME \ "https://storageinsights.googleapis.com/v1/projects/YOUR_PROJECT/locations/LOCATION/reportConfigs" \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --header "Accept: application/json" \ --header "Content-Type: application/json" \
Replace:
JSON_FILE_NAME
with the path to the JSON file you created in the previous step.YOUR_PROJECT
with the name of the project in which your source and destination buckets reside.LOCATION
with the location of the source and destination buckets. For example,us-west1
.
Grant required roles to the service agent
A project-level
service agent
is automatically created the first time you set up an inventory report
configuration. The service agent follows the naming format
service-PROJECT_NUMBER@gcp-sa-storageinsights.iam.gserviceaccount.com
and
appears on the
IAM page of the Google Cloud console
when you select the Include Google-provided role grants checkbox.
To enable Storage Insights to generate and write inventory reports, ask your administrator to grant the service agent the following IAM roles:
roles/storage.insightsCollectorService
on the source bucket, which includes thestorage.buckets.getObjectInsights
andstorage.buckets.get
permissionsroles/storage.objectCreator
on the destination bucket, which includes thestorage.objects.create
permission
For instructions on granting roles, see Use IAM.
You can grant the roles/storage.insightsCollectorService
role by using
the Google Cloud console or the Google Cloud CLI. For example:
gcloud storage buckets add-iam-policy-binding SOURCE_BUCKET_URL \ --member=serviceAccount:service-PROJECT_NUMBER@gcp-sa-storageinsights.iam.gserviceaccount.com \ --role=roles/storage.insightsCollectorService
Once the service agent is granted the required permissions, it takes up to 24 hours to write the first inventory report to the destination bucket.
Edit an inventory report configuration
Once an inventory report configuration has been created, you can modify certain properties of the configuration.
Console
To edit an inventory report configuration, follow the steps below:
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the source bucket that contains the inventory report configuration you want to edit.
In your source bucket's Bucket details page, click the Inventory reports tab.
Click the configuration name of the inventory report configuration you want to edit.
In the Report configuration details page that appears, edit the properties you desire by using the
Edit button.
Command line
To edit an inventory report configuration, follow the steps below:
To find the name of the inventory report configuration you want to edit, list all the inventory configurations in the source bucket by using the
gcloud storage insights inventory-reports list
command:gcloud storage insights inventory-reports list \ --source=SOURCE_BUCKET \ --filter=EXPRESSION \ --page-size=SIZE \ --sort-by=FIELD \ --format="yaml(name)"
Replace:
SOURCE_BUCKET
with the URL of the source bucket that contains the inventory report configuration.EXPRESSION
with a boolean filter to apply to each resource item to be listed. If the expression evaluates True, the item is listed. For more details and examples of filter expressions, run$ gcloud topic filters
.SIZE
with the maximum number of resources per page. The default is 50.FIELD
with a comma-separated list of resource field key names to sort by. The default order is ascending. Prefix a field with~
for descending order on that field.
Edit the inventory report configuration fields you want to update by using the
gcloud storage insights inventory-reports update
command. The following example updates the inventory report configuration to generate inventory reports daily in Apache Parquet format:gcloud storage insights inventory-reports update CONFIG_NAME \ --parquet \ --schedule-repeats="daily"
Client libraries
For more information, see the
Cloud Storage Java API
reference documentation.
To authenticate to Cloud Storage, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Java
REST APIs
JSON API
To edit an inventory report configuration, complete the following steps:
Have gcloud CLI installed and initialized, in order to generate an access token for the
Authorization
header.Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the
Authorization
header.Get the name of the inventory report configuration you want to edit by using
cURL
to call the JSON API with aGet
ReportConfig request:curl --request GET \ "https://storageinsights.googleapis.com/v1/projects/YOUR_PROJECT/locations/LOCATION/reportConfigs?" \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --header "Accept: application/json" \ --header "Content-Type: application/json" \
Replace:
YOUR_PROJECT
with the name of the project in which your source and destination buckets reside.LOCATION
with the location of the source and destination buckets. For example,us-west1
.
If you want to retrieve all inventory report configurations in a specific source bucket, append the
filter
query parameter to your request:curl --request GET \ "https://storageinsights.googleapis.com/v1/projects/YOUR_PROJECT/locations/LOCATION/reportConfigs?filter=objectMetadataReportOptions.storageFilters.bucket=BUCKET_NAME" \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --header "Accept: application/json" \ --header "Content-Type: application/json" \
Replace:
YOUR_PROJECT
with the name of the project in which your source and destination buckets reside.LOCATION
with the location of the source and destination buckets. For example,us-west1
.BUCKET_NAME
with the name of the source bucket that contains the inventory report configurations you want to edit.
Create a JSON file that includes changes to the properties you want to edit. The following example updates the inventory report configuration to generate inventory reports daily in Apache Parquet format:
{ "ReportConfig": { "frequency_options": { "frequency": "DAILY" }, "parquet_options": { } }
To apply the inventory report configuration, use
cURL
to call the JSON API with aPatch
ReportConfig request:curl --request PATCH \ "https://storageinsights.googleapis.com/v1/projects/YOUR_PROJECT/locations/LOCATION/reportConfigs/REPORT_CONFIG_UUID" \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --header "Accept: application/json" \ --header "Content-Type: application/json" \ --data-binary "@JSON_FILE_NAME.json" \
Replace:
YOUR_PROJECT
with the name of the project in which your source and destination buckets reside.LOCATION
with the location of the source and destination buckets. For example,us-west1
.REPORT_CONFIG_UUID
with the auto-generated UUID of the inventory report configuration.
List inventory report configurations
Console
To list the inventory report configurations in a source bucket, complete the steps below:
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the source bucket that contains the inventory report configurations you want to view.
On the Bucket details page, click the Inventory reports tab.
Inventory report configurations in the source bucket appear.
Command line
You can list all the inventory configurations in a source bucket by
using the gcloud storage insights inventory-reports list
command:
gcloud storage insights inventory-reports list \ --source=SOURCE_BUCKET \ --filter=EXPRESSION \ --page-size=SIZE \ --sort-by=FIELD \ --format="yaml(name)"
Replace:
SOURCE_BUCKET
with the URL of the source bucket that contains the inventory report configuration.EXPRESSION
with a boolean filter to apply to each resource item to be listed. If the expression evaluates True, the item is listed. For more details and examples of filter expressions, run$ gcloud topic filters
.SIZE
with the maximum number of resources per page. The default is 50.FIELD
with a comma-separated list of resource field key names to sort by. The default order is ascending. Prefix a field with~
for descending order on that field.
Client libraries
For more information, see the
Cloud Storage Java API
reference documentation.
To authenticate to Cloud Storage, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Java
REST APIs
JSON API
Have gcloud CLI installed and initialized, in order to generate an access token for the
Authorization
header.Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the
Authorization
header.You can list all inventory report configurations in a project for a given location by using a request to list inventory report configurations:
curl --request GET \ "https://storageinsights.googleapis.com/v1/projects/YOUR_PROJECT/locations/LOCATION/reportConfigs?" \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --header "Accept: application/json" \ --header "Content-Type: application/json" \
Replace:
YOUR_PROJECT
with the name of the project in which your source and destination buckets reside.LOCATION
with the location of the source and destination buckets. For example,us-west1
.
If you want to list all inventory report configurations in a specific source bucket, append the
filter
query parameter to your request:curl --request GET \ "https://storageinsights.googleapis.com/v1/projects/YOUR_PROJECT/locations/LOCATION/reportConfigs?filter=objectMetadataReportOptions.storageFilters.bucket=BUCKET_NAME" \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --header "Accept: application/json" \ --header "Content-Type: application/json" \
Replace:
YOUR_PROJECT
with the name of the project in which your source and destination buckets reside.LOCATION
with the location of the source and destination buckets. For example,us-west1
.BUCKET_NAME
with the name of the source bucket that contains the inventory report configurations you want to list.
Download an inventory report
Console
Download individual reports
Inventory reports are generated and stored as objects in the destination bucket, so you can download them as you would download a regular object.
To download an inventory report, complete the following steps:
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the destination bucket that contains the inventory report you want to download.
On the Bucket details page, make sure the Objects tab is selected.
Click
Download associated with the inventory report you want to download.
If you don't know the destination bucket, you can also download an inventory report through the inventory report configuration from which it was generated:
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the source bucket containing the inventory report configuration that generated the report you want to download.
On the Bucket details page, click the configuration name of the inventory report configuration.
On the Report configuration details page that appears, navigate to the Inventory report history section, then click the destination object path of the inventory report you want to download.
The Bucket details page appears for the destination bucket that contains the inventory report.
Click
Download associated with the inventory report you want to download.
Download report shards
To download an inventory report that's been split into one or more shards, complete the following steps:
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the destination bucket you specified when you created the inventory report configuration.
On the Bucket details page, check for the presence of a manifest file. The presence of a manifest file indicates that all the shards of an inventory report have been generated.
An example manifest file name is
fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:00_manifest.json
.In the destination bucket, click
Download associated with the manifest file. Note the names of the shard files you want to download from thereport_shards_file_names
field.In the destination bucket, click
Download associated with the shard files you want to download.
Command line
Download individual reports
To download an inventory report, complete the following steps:
To list all the inventory reports that have been generated by an inventory report configuration and retrieve their
REPORT_DETAIL_ID
, use thegcloud storage insights inventory-reports details list
command:gcloud storage insights inventory-reports details list CONFIG_NAME \ --filter=EXPRESSION \ --page-size=SIZE \ --sort-by=FIELD
Replace:
CONFIG_NAME
with the unique name of the inventory report configuration, in the formatprojects/PROJECT/locations/LOCATION/reportConfigs/REPORT_CONFIG_UUID
.EXPRESSION
with a boolean filter to apply to each resource item to be listed. If the expression evaluates True, then that item is listed. For more details and examples of filter expressions, run$ gcloud topic filters
.SIZE
with the maximum number of resources per page. The default is 50.FIELD
with a comma-separated list of resource field key names to sort by. The default order is ascending. Prefix a field with~
for descending order on that field.
If successful, the command returns output similar to the following:
REPORT_DETAIL_ID SNAPSHOT_TIME Report_2023-04-10T00-00 2023-04-10T00:53:03Z Report_2023-04-12T00-00 2023-04-12T00:52:54Z Report_2023-04-05T00-00 2023-04-05T00:53:01Z
To download an inventory report, you need to first retrieve the
reportPathPrefix
property of the ReportDetail object. To get thereportPathPrefix
of a report, use thegcloud storage insights inventory-reports details describe
command:gcloud storage insights inventory-reports details describe REPORT_DETAIL_NAME
Replace
REPORT_DETAIL_NAME
with the name of the inventory report in the formatprojects/PROJECT/locations/LOCATION/reportConfigs/REPORT_CONFIG_UUID/reportDetails/REPORT_DETAIL_ID
.
Download report shards
To download an inventory report that's been split into one or more shards, complete the following steps:
Download the inventory report manifest file by using the
gcloud storage cp
command:gcloud storage cp gs://BUCKET_NAME/MANIFEST_FILE_NAME DOWNLOAD_PATH
Replace:
BUCKET_NAME
with the name of the destination bucket.MANIFEST_FILE_NAME
with the name of the manifest file in the destination bucket, in the following naming convention:REPORT_CONFIG_UUID_TARGET_DATETIME_manifest.json
Where:
REPORT_CONFIG_UUID
is the auto-generated UUID of the inventory report configuration that generated the report shards you want to download.TARGET_DATETIME
is the datetime at which the inventory report was generated.
For example,
fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:00_manifest.json
.DOWNLOAD_PATH
with the path to your file system where you want to save the inventory report to. For example,./example_report.csv
.
To download a shard file, use the
gcloud storage cp
command:gcloud storage cp gs://BUCKET_NAME/SHARD_FILE_NAME DOWNLOAD_PATH
Replace:
BUCKET_NAME
with the name of the destination bucket.SHARD_FILE_NAME
with the URL-encoded name of the shard file you want to download. For example,fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:54_0.csv
.DOWNLOAD_PATH
with the path to your file system where you want to save the inventory report to. For example,./example_report.csv
.
Client libraries
For more information, see the
Cloud Storage Java API
reference documentation.
To authenticate to Cloud Storage, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Java
REST APIs
JSON API
Download individual reports
To download an inventory report, complete the following steps:
Have gcloud CLI installed and initialized, in order to generate an access token for the
Authorization
header.Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the
Authorization
header.List all the inventory reports generated by an inventory report configuration by using
cURL
to call the JSON API with a request to list inventory report details:curl --request GET \ "https://storageinsights.googleapis.com/v1/projects/YOUR_PROJECT/locations/LOCATION/reportConfigs/REPORT_CONFIG_UUID/reportDetails/" \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --header "Accept: application/json" \ --header "Content-Type: application/json" \
Replace:
YOUR_PROJECT
with the name of the project in which your source and destination buckets reside.LOCATION
with the location of the source and destination buckets. For example,us-west1
.REPORT_CONFIG_UUID
with the auto-generated UUID of the inventory report configuration that generated the report you want to download.
To get the details of an individual report, use
cURL
to call the JSON API with aGet
ReportDetails request:curl --request GET \ "https://storageinsights.googleapis.com/v1/projects/YOUR_PROJECT/locations/LOCATION/reportConfigs/REPORT_CONFIG_UUID/reportDetails/REPORT_DETAIL_ID" \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --header "Accept: application/json" \ --header "Content-Type: application/json" \
Replace:
YOUR_PROJECT
with the name of the project in which your source and destination buckets reside.LOCATION
with the location of the source and destination buckets. For example,us-west1
.REPORT_CONFIG_UUID
with the auto-generated UUID of the inventory report configuration that generated the report you want to download.REPORT_DETAIL_ID
with the name of the inventory report you want to download.
Download report shards
To download an inventory report that's been split into one or more shards, complete the following steps:
Have gcloud CLI installed and initialized, in order to generate an access token for the
Authorization
header.Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the
Authorization
header.To download the manifest file of the inventory report, use
cURL
to call the JSON API with aGET
Object request:curl -X GET \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ --header "Accept: application/json" \ --header "Content-Type: application/json" \ "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o/MANIFEST_FILE_NAME?alt=media" \
Replace:
BUCKET_NAME
with the name of the destination bucket.MANIFEST_FILE_NAME
with the name of the manifest file in the destination bucket, in the following naming convention:REPORT_CONFIG_UUID_TARGET_DATETIME_manifest.json
Where:
REPORT_CONFIG_UUID
is the auto-generated UUID of the inventory report configuration that generated the report shards you want to download.TARGET_DATETIME
is the date on which the inventory report was generated.
For example,
fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:00_manifest.json
.The
report_shards_file_names
field of the manifest file contains the names of the inventory report shards you can download.To download the inventory report shard files, use
curl
to call the Cloud Storage JSON API with aGet
Object request:curl -X GET \ --header "Authorization: Bearer $(gcloud auth print-access-token)" \ -o "DOWNLOAD_PATH" \ "https://storage.googleapis.com/storage/v1/b/BUCKET_NAME/o/SHARD_FILE_NAME?alt=media"
Replace:
DOWNLOAD_PATH
with the path to your local file system where you want to save your object. For example,Desktop/dog.png
.BUCKET_NAME
with the name of the destination bucket containing the inventory report. For example,my-bucket
.SHARD_FILE_NAME
with the URL-encoded name of the shard file you want to download. For example,fc95c52f-157a-494f-af4a-d4a53a69ba66_2022-11-30T00:54_0.csv
.
Delete an inventory report configuration
Console
To delete an inventory report configuration, complete the following steps:
- In the Google Cloud console, go to the Cloud Storage Buckets page.
In the list of buckets, click the name of the source bucket that contains the inventory report configuration you want to delete.
In your source bucket's Bucket details page, click the Inventory reports tab.
Click the configuration name of the inventory report configuration you want to delete.
In the Report configuration details page that appears, click
Delete.
Command line
To delete an inventory report configuration, use the
gcloud storage insights inventory-reports delete
command:
gcloud storage insights inventory-reports delete CONFIG_NAME --force
Replace CONFIG_NAME
with the unique name of the
inventory report configuration, in the format
projects/PROJECT/locations/LOCATION/reportConfigs/REPORT_CONFIG_UUID
.
The --force
flag deletes the metadata for all inventory reports
generated by the given inventory report configuration. The
inventory report objects themselves will not be deleted.
Client libraries
For more information, see the
Cloud Storage Java API
reference documentation.
To authenticate to Cloud Storage, set up Application Default Credentials.
For more information, see
Set up authentication for a local development environment.
Java
REST APIs
JSON API
To delete an inventory report configuration, complete the following steps:
Have gcloud CLI installed and initialized, in order to generate an access token for the
Authorization
header.Alternatively, you can create an access token using the OAuth 2.0 Playground and include it in the
Authorization
header.To delete an inventory report configuration, use
cURL
to call the JSON API with aDelete
ReportConfig request:curl --request DELETE \ 'https://storageinsights.googleapis.com/v1/projects/YOUR_PROJECT/locations/LOCATION/reportConfigs/REPORT_CONFIG_UUID?force=true' \ --header 'Authorization: Bearer $(gcloud auth print-access-token)' \ --header 'Accept: application/json' \ --header 'Content-Type: application/json' \
Replace:
YOUR_PROJECT
with the name of the project in which your source and destination buckets reside.LOCATION
with the location of the source and destination buckets. For example,us-west1
.REPORT_CONFIG_UUID
with the auto-generated UUID of the inventory report configuration you want to delete.
The force
query parameter deletes the metadata for all
inventory reports generated by the given inventory report configuration.
The inventory report objects themselves will not be deleted.
Advanced: Generate inventory reports in a Hive partitioned format
When you create an inventory report configuration, you can specify a destination path, which is a location in your destination bucket where inventory reports will be generated. This enables you to generate inventory reports in a Hive partitioned format.
You specify a destination path by using placeholder keywords.
Keywords in the destination path are substituted with their corresponding values
when the inventory report is generated. For example, the destination
path
config={{report-config-id}}/date={{date}}
can resolve to config=1A34-F2E456-12B456-1C3D/date=2022-05-20
.
Destination path keywords
You can specify the keywords
report-config-id
, date
, or datetime
, enclosed within {{ and
}}, where:
report-config-id
is the UUID of the inventory report configuration.date
is the date on which the inventory report is generated. Follows the ISO 8601 format.datetime
is the date and time on which the inventory report is generated. Follows the ISO 8601 format.
Keywords can be specified by using any of the following tools:
When using the Google Cloud console, specify keywords in the Enter destination path (optional) section. This section appears when you create an inventory report configuration.
When using the Google Cloud CLI, specify placeholder keywords by using the
--destination
flag.When using the JSON API, specify placeholder keywords in the
storage_destination_options.destination_path
field of the ReportConfig object.
Access control and security
The following recommendations and considerations relate to the security and usage of inventory reports and configurations:
We recommend that users with the
roles/storage.admin
role also have thestorageinsights.reportConfigs.*
permissions so they can maintain control over the usage of all resources in their buckets, such as the reading of objects and their metadata.We recommend that you limit access to your inventory report resources by granting the
storageinsights.reportConfigs.*
permissions only to necessary individuals.Once an inventory report configuration has been created, it continues to generate inventory reports even if the user who created the configuration no longer has the required permissions. To stop generating inventory reports, you can edit the end date in the inventory report configuration or delete the configuration entirely.
What's next
- If you encounter issues with generating inventory reports, refer to Troubleshooting for help.