Before you begin
- Sign in to your Google Cloud account. If you're new to Google Cloud, create an account to evaluate how our products perform in real-world scenarios. New customers also get $300 in free credits to run, test, and deploy workloads.
-
In the Google Cloud Console, on the project selector page, select or create a Google Cloud project.
- Enable the BigQuery API.
-
Create a service account:
-
In the Cloud Console, go to the Create service account page.
Go to Create service account - Select a project.
-
In the Service account name field, enter a name. The Cloud Console fills in the Service account ID field based on this name.
In the Service account description field, enter a description. For example,
Service account for quickstart
. - Click Create.
-
Click the Select a role field.
Under Quick access, click Basic, then click Owner.
- Click Continue.
-
Click Done to finish creating the service account.
Do not close your browser window. You will use it in the next step.
-
-
Create a service account key:
- In the Cloud Console, click the email address for the service account that you created.
- Click Keys.
- Click Add key, then click Create new key.
- Click Create. A JSON key file is downloaded to your computer.
- Click Close.
-
Set the environment variable
GOOGLE_APPLICATION_CREDENTIALS
to the path of the JSON file that contains your service account key. This variable only applies to your current shell session, so if you open a new session, set the variable again.
Install the client library
C#
For more on setting up your C# development environment, refer to the C# Development Environment Setup Guide.
Install-Package Google.Cloud.BigQuery.V2 -Pre
Go
go mod init YOUR_MODULE_NAME go get cloud.google.com/go/bigquery
Java
For more on setting up your Java development environment, refer to the Java Development Environment Setup Guide.
If you are using Maven, add
the following to your pom.xml
file. For more information about
BOMs, see The Google Cloud Platform Libraries BOM.
If you are using Gradle, add the following to your dependencies:
If you are using sbt, add the following to your dependencies:
If you're using IntelliJ or Eclipse, you can add client libraries to your project using the following IDE plugins:
The plugins provide additional functionality, such as key management for service accounts. Refer to each plugin's documentation for details.
Node.js
For more on setting up your Node.js development environment, refer to the Node.js Development Environment Setup Guide.
npm install --save @google-cloud/bigquery
PHP
composer require google/cloud-bigquery
Python
For more on setting up your Python development environment, refer to the Python Development Environment Setup Guide.
pip install --upgrade google-cloud-bigquery
Ruby
For more on setting up your Ruby development environment, refer to the Ruby Development Environment Setup Guide.
gem install google-cloud-bigquery
Import the libraries
C#
For more information, see the BigQuery C# API reference documentation.
Go
For more information, see the BigQuery Go API reference documentation.
Java
For more information, see the BigQuery Java API reference documentation.
Node.js
For more information, see the BigQuery Node.js API reference documentation.
PHP
For more information, see the BigQuery PHP API reference documentation.
Python
For more information, see the BigQuery Python API reference documentation.
Ruby
For more information, see the BigQuery Ruby API reference documentation.
Initialize a BigQuery client
C#
Use the BigQueryClient.Create() function to create the BigQuery client.
Go
Use the bigquery.NewClient() function to create the BigQuery client.
Java
Use the BigQueryOptions.getDefaultInstance() function to use the default authentication options. Use the BigQueryOptions.getService() function to create the BigQuery client.
Node.js
Instantiate the BigQuery class to create the BigQuery client.
PHP
Instantiate the BigQueryClient class to create the BigQuery client.
Python
Instantiate the bigquery.Client class to create the BigQuery client.
Ruby
Use the Google::Cloud::Bigquery.new function to create the BigQuery client.
Running queries
Query the Stack Overflow public dataset
to find the most viewed questions tagged with google-bigquery
.
SELECT CONCAT( 'https://stackoverflow.com/questions/', CAST(id as STRING)) as url, view_count FROM `bigquery-public-data.stackoverflow.posts_questions` WHERE tags like '%google-bigquery%' ORDER BY view_count DESC LIMIT 10
This query uses standard SQL syntax, which is described in the query reference guide. The client libraries default to standard SQL syntax. See Switching SQL dialects to change SQL dialects.
Running the query
Query using the authenticated BigQuery client.
C#
Define a query string and use the client.ExecuteQuery() function to submit the query and get the results.
Go
Use the bigquery.Query() function to define a query and Query.Read() function to submit the query and get the results.
Java
Define the query with a QueryJobConfiguration instance. Start the query job with the BigQuery.create() method.
Node.js
Use the BigQuery.query() method to start the query.
PHP
Create a query configuration and use the BigQueryClient.startQuery() method to start the query.
Python
Use the Client.query() method to start the query.
Ruby
Use the Google::Cloud::Bigquery::Project.query function to start a query and wait for the results.
For more examples of running BigQuery queries, see:
- Querying data overview
- How to run interactive and batch queries
- How to write query results to a permanent table
Displaying the query result
Display the query results.
C#
Go
Use the RowIterator.Next() function to load each row into a struct pointer.
Java
Iterate over the QueryResponse to get all the rows in the results. The iterator automatically handles pagination. Each FieldList exposes the columns by numeric index or column name.
Node.js
Query results are returned as a list of rows, where each row is a dictionary.
PHP
Call the Job.queryResults() method to wait for the query to finish. Each row in the query results is an associative array.
Python
Iterate over the RowIterator to get all the rows in the results. The iterator automatically handles pagination. Each Row exposes the columns by numeric index, column name, or as Python attributes.
Ruby
The Google::Cloud::Bigquery::Data class exposes each row as a hash.
Learn more about working with data rows in BigQuery:
Complete source code
Here is the complete source code for the sample.
C#
Go
Java
Node.js
PHP
Python
Ruby
How did it go?
What's next
Find out more about our BigQuery API Client Libraries.