Treat Google Cloud Storage like a file system with our new PowerShell provider
Jim Przybylinski and Chris Smith
Google Cloud Storage is pretty amazing. It offers near-infinite capacity, up to 99.95% availability and fees as low as $0.007GB/month. But storing data in the cloud has always had one drawback: you need to use specialized tools like gsutil to browse or access it. You can’t just treat Cloud Storage like a really, really, really big hard disk. That is, until now.
Navigating Cloud Storage with Cloud Tools for PowerShellThe latest release of Cloud Tools for PowerShell (included with the Cloud SDK for Windows) includes a PowerShell provider for Cloud Storage. PowerShell providers are a slick feature of Windows PowerShell that allows you to treat a data source as if it were a file system, to do things like browse the system registry or interact with a SQL Server instance. With a PowerShell provider for Cloud Storage, you can now use commands like
cd, dir, copy, del,or even
catto navigate and manipulate your data in Cloud Storage.
To use the provider for Cloud Storage, first load the
GoogleCloud PowerShell module by using any of its cmdlets, PowerShell’s lightweight commands. Then just
cd into the
gs:\ drive. You can now explore your data like you would any local disk. To see what buckets you have available in Cloud Storage, just type
dir. The provider will use whatever credentials you have configured for the Cloud SDK (see gcloud init).
PS C:\> Import-Module GoogleCloud
WARNING: The names of some imported commands from the module 'GoogleCloud' include unapproved verbs that might make
them less discoverable. To find the commands with unapproved verbs, run the Import-Module command again with the
Verbose parameter. For a list of approved verbs, type Get-Verb.
PS C:\> cd gs:\PS gs:\> dir | Select Name
To navigate your buckets and search for a specific object, just keep using
dir (which are aliases for the
Set-Location and Get-ChildItem cmdlets respectively.) Note that just like the regular file system provider, you can use tab-completion for file and folder names.
Populating Google Cloud StorageThe following code snippet shows how to create a new bucket using
mkdirand use the
Set-Contentcmdlet to create a new object. Notice that Get-Content takes an object name relative to the current folder in Google Cloud Storage, e.g.
PS gs:\> mkdir gootoso-test-bucket | Out-Null
PS gs:\> Set-Content gs:\gootoso-test-bucket\folder\file.txt `
-Value "Hello, GCS!"
PS gs:\> Test-Path gs:\gootoso-test-bucket\folder\file.txt
PS gs:\> cd .\gootoso-test-bucket\folder
PS gs:\gootoso-test-bucket\folder> cat file.txt
Of course you could do the same thing with the existing PowerShell cmdlets for Cloud Storage such as
Copy-GcsObject and so on. But being able to use common commands like
cd in the PowerShell provider provides a much more natural and productive experience.
Mixing Cmdlets and the PowerShell ProviderSince the PowerShell provider returns the same objects as other Cloud Storage cmdlets, you can intermix commands. For example:
PS gs:\gootoso-test-bucket\folder> $objs = dir
PS gs:\gootoso-test-bucket\folder> $objs.GetType().FullName
PS gs:\gootoso-test-bucket\folder> $objs | Read-GcsObject
PS gs:\gootoso-test-bucket\folder> Write-GcsObject -Object $objs -Contents "update"
PS gs:\> Remove-GcsBucket -Name gootoso-test-bucket
All of the objects returned are strongly typed, defined in the C# client library for the Cloud Storage API. That means you can use PowerShell’s particularly powerful pipelining features to access properties on the returned objects, for things like sorting and filtering.
This snippet shows how to get the largest file in the blog-posts Bucket, for any object under the images folder.
PS gs:\> cd gs:\blog-posts\images
PS gs:\blog-posts\images> $objects = dir -Recurse
PS gs:\blog-posts\images> $objects |
Sort-Object Size -Descending |
Select-Object -First 1 -Property Name,TimeCreated,Size
In short, the PowerShell provider for Cloud Storage simplifies a lot of tasks, so give it a whirl and try it for yourself. For more information on the provider as well as other PowerShell cmdlets, check out the PowerShell documentation.
Google Cloud Tools for PowerShell, including the new provider for Cloud Storage, is in beta. If you have any feedback on the cmdlet design, documentation, or have any other issues, please report it on GitHub. The code is open-source too, so pull requests are also welcome.