Understand and manage data at scale with Cloud Storage Insights

With Cloud Storage, organizations can scale cloud storage usage from terabytes (TBs) to exabytes (EBs) without being constrained by infrastructure or scalability. But, when data volume increases, it gets harder to efficiently discover and manage that data, which can cause problems such as security flaws and unforeseen storage expenditures.

To address these challenges, we’re excited to announce the availability of the Storage Insights inventory report, which offers comprehensive and actionable insight into your data assets stored in Cloud Storage buckets. The Storage Insights inventory report includes metadata about your objects, such as their storage class, size, ETag, and content type, among other details. You can import these inventory reports into analytics products such as BigQuery for further analysis and use this information to audit and validate your objects, analyze your storage expenses, and ensure data security and compliance. The inventory report can also help identify objects that are no longer needed and that can be removed or moved to a lower storage class to optimize your storage costs.

Using Storage Insights

The Storage Insights inventory report is a scheduled and high-performance alternative to the Object list API. It is a bucket-level feature that can be configured using various methods, such as the API, gCloud CLI, SDKs, or the Google Cloud console. The Inventory report is generated by taking rolling snapshots of your buckets, which has no impact on your production performance.

You can configure the inventory report to run daily or weekly and define the start and end dates. You can also choose to save the report in file formats such as CSV or Parquet, which are easily accessible and can be further analyzed using powerful tools such as BigQuery.

What are customers saying about Storage Insights?

"Google Cloud Storage Insights has provided tremendous value to our organization! Insights gives us usage patterns for our Cloud Storage resources that allow us to make real-time, data-driven decisions for improving our business operations. Using the information produced by Insights, we've been able to quickly and easily create custom alerts, notifications, and interactive dashboards that keep us informed about our storage needs, such as optimizing our storage costs without compromising the performance or availability of our data.” - Sachin GopalaKrishnan, VP, Data and Marketing, at Priceline

“ShareChat and Moj are the leading social media and short-form video platforms in India. Over time, massive amounts of data had accumulated in our Cloud Storage buckets, making it increasingly difficult to gain insights and make informed decisions. Storage Insights has proved to be a game-changer here, providing us with valuable insights into our storage usage and patterns, allowing us to identify, reclassify or eliminate unused objects, and save significant costs. By using the inventory report and the powerful analytics capabilities of BigQuery, we were able to efficiently analyze billions of objects within 24-48 hours and at a fraction of cost as compared to any other alternative. Without Storage Insights, it would have taken us months to complete such a task.” - Chhaya Sharma, Director of Engineering, at Sharechat

Get started today

All Cloud Storage customers can configure Storage Insights on new and existing buckets and receive actionable insights to efficiently manage their data at scale. The Storage Insights inventory report is priced based on the number of objects it scans at each run. For every one million objects in the report, there will be a charge according to the pricing published here. 

For more information and detailed instructions on how to use Storage Insights, refer to the documentation here.