This page shows how to get started with the Cloud Client Libraries for the BigQuery Storage API. Read more about the client libraries for Cloud APIs, including the older Google APIs Client Libraries, in Client Libraries Explained.
Installing the client library
Go
For more information, see Setting Up a Go Development Environment.
go get -u cloud.google.com/go/bigquery
Java
For more information, see Setting Up a Java Development Environment.
If you are using Maven, add
the following to your pom.xml
file. For more information about
BOMs, see The Google Cloud Platform Libraries BOM.
If you are using Gradle, add the following to your dependencies:
If you are using sbt, add the following to your dependencies:
If you're using IntelliJ or Eclipse, you can add client libraries to your project using the following IDE plugins:
The plugins provide additional functionality, such as key management for service accounts. Refer to each plugin's documentation for details.
Python
For more information, see Setting Up a Python Development Environment.
pip install --upgrade google-cloud-bigquery-storage
Setting up authentication
To run the client library, you must first set up authentication by creating a service account and setting an environment variable. Complete the following steps to set up authentication. For other ways to authenticate, see the GCP authentication documentation.
Cloud Console
Create a service account:
-
In the Cloud Console, go to the Create service account page.
Go to Create service account - Select a project.
-
In the Service account name field, enter a name. The Cloud Console fills in the Service account ID field based on this name.
In the Service account description field, enter a description. For example,
Service account for quickstart
. - Click Create.
-
Click the Select a role field.
Under Quick access, click Basic, then click Owner.
- Click Continue.
-
Click Done to finish creating the service account.
Do not close your browser window. You will use it in the next step.
Create a service account key:
- In the Cloud Console, click the email address for the service account that you created.
- Click Keys.
- Click Add key, then click Create new key.
- Click Create. A JSON key file is downloaded to your computer.
- Click Close.
Command line
You can run the following commands using the Cloud SDK on your local machine, or in Cloud Shell.
-
Create the service account. Replace NAME with a name for the service account.
gcloud iam service-accounts create NAME
-
Grant permissions to the service account. Replace PROJECT_ID with your project ID.
gcloud projects add-iam-policy-binding PROJECT_ID --member="serviceAccount:NAME@PROJECT_ID.iam.gserviceaccount.com" --role="roles/owner"
-
Generate the key file. Replace FILE_NAME with a name for the key file.
gcloud iam service-accounts keys create FILE_NAME.json --iam-account=NAME@PROJECT_ID.iam.gserviceaccount.com
Provide authentication credentials to your application code by
setting the environment variable GOOGLE_APPLICATION_CREDENTIALS
.
Replace [PATH] with the file path of the JSON file that contains your
service account key. This variable only applies to your current shell session,
so if you open a new session, set the variable again.
Linux or macOS
export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
For example:
export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/my-key.json"
Windows
With PowerShell:
$env:GOOGLE_APPLICATION_CREDENTIALS="[PATH]"
For example:
$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\my-key.json"
With command prompt:
set GOOGLE_APPLICATION_CREDENTIALS=[PATH]
Using the client library
The following example shows basic interactions with the BigQuery Storage API.
Go
To use this sample, prepare your machine for Go development, and complete the BigQuery Storage API quickstart. For more information, see the BigQuery Storage API Go API reference documentation.
Java
Before trying this sample, follow the Java setup instructions in Setting Up a Java Development Environment. For more information, see the BigQuery Storage API Java API reference documentation.
Python
Before trying this sample, follow the Python setup instructions in Setting Up a Python Development Environment. For more information, see the BigQuery Storage API Python API reference documentation.
Additional resources
Go
Java
Python
What's next?
For users of the pandas and the pandas-gbq integration to BigQuery, see the tutorial Downloading BigQuery data to pandas using the BigQuery Storage API for more information about leveraging the storage API.