BigQuery Client Libraries

This page shows how to get started with the new Cloud Client Libraries for the Google BigQuery API. Read more about the client libraries for Cloud APIs, including the older Google APIs Client Libraries, in Client Libraries Explained.

Installing the client library

C#

Install-Package Google.Cloud.BigQuery.V2 -Pre

Go

go get -u cloud.google.com/go/bigquery

Java

If you are using Maven, add this to your pom.xml file:
<dependency>
  <groupId>com.google.cloud</groupId>
  <artifactId>google-cloud-bigquery</artifactId>
  <version>1.31.0</version>
</dependency>
If you are using Gradle, add this to your dependencies:
compile 'com.google.cloud:google-cloud-bigquery:1.31.0'
If you are using SBT, add this to your dependencies:
libraryDependencies += "com.google.cloud" % "google-cloud-bigquery" % "1.31.0"

Node.js

npm install --save @google-cloud/bigquery

PHP

composer require google/cloud-bigquery

Python

For more on setting up your Python development environment, refer to Python Development Environment Setup Guide.
pip install --upgrade google-cloud-bigquery

Ruby

gem install google-cloud-bigquery

Setting up authentication

To run the client library, you must first set up authentication by creating a service account and setting an environment variable.

GCP Console

  1. Go to the Create service account key page in the GCP Console.

    Go to the Create Service Account Key page
  2. From the Service account drop-down list, select New service account.
  3. Enter a name into the Service account name field.
  4. From the Role drop-down list, select Project > Owner.

    Note: The Role field authorizes your service account to access resources. You can view and change this field later using GCP Console. If you are developing a production application, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.
  5. Click Create. A JSON file that contains your key downloads to your computer.

Command line

You can run the following commands using the Cloud SDK on your local machine, or within Cloud Shell.

  1. Create the service account. Replace [NAME] with your desired service account name.

    gcloud iam service-accounts create [NAME]
  2. Grant permissions to the service account. Replace [PROJECT_ID] with your project ID.

    gcloud projects add-iam-policy-binding [PROJECT_ID] --member "serviceAccount:[NAME]@[PROJECT_ID].iam.gserviceaccount.com" --role "roles/owner"
    Note: The Role field authorizes your service account to access resources. You can view and change this field later using GCP Console. If you are developing a production application, specify more granular permissions than Project > Owner. For more information, see granting roles to service accounts.
  3. Generate the key file. Replace [FILE_NAME] with a name for the key file.

    gcloud iam service-accounts keys create [FILE_NAME].json --iam-account [NAME]@[PROJECT_ID].iam.gserviceaccount.com

Provide authentication credentials to your application code by setting the environment variable GOOGLE_APPLICATION_CREDENTIALS. Replace [PATH] with the file path of the JSON file that contains your service account key, and [FILE_NAME] with the filename.

Linux or macOS

export GOOGLE_APPLICATION_CREDENTIALS="[PATH]"

For example:

export GOOGLE_APPLICATION_CREDENTIALS="/home/user/Downloads/[FILE_NAME].json"

Windows

With PowerShell:

$env:GOOGLE_APPLICATION_CREDENTIALS="[PATH]"

For example:

$env:GOOGLE_APPLICATION_CREDENTIALS="C:\Users\username\Downloads\[FILE_NAME].json"

With command prompt:

set GOOGLE_APPLICATION_CREDENTIALS=[PATH]

Using the client library

The following example shows how to use the client library.

C#

Read more in the C# API Reference Documentation for the Google BigQuery API Client Library.

using System;
using Google.Cloud.BigQuery.V2;

namespace GoogleCloudSamples
{
    public class Program
    {
        public static void Main(string[] args)
        {
            // Your Google Cloud Platform project ID
            string projectId = "YOUR-PROJECT-ID";

            // Instantiates a client
            BigQueryClient client = BigQueryClient.Create(projectId);

            // The id for the new dataset
            string datasetId = "my_new_dataset";

            // Creates the dataset
            BigQueryDataset dataset = client.CreateDataset(datasetId);

            Console.WriteLine($"Dataset {dataset.FullyQualifiedId} created.");
        }
    }
}

Go

Read more in the Go API Reference Documentation for the Google BigQuery API Client Library.

// Sample bigquery-quickstart creates a Google BigQuery dataset.
package main

import (
	"fmt"
	"log"

	// Imports the Google Cloud BigQuery client package.
	"cloud.google.com/go/bigquery"
	"golang.org/x/net/context"
)

func main() {
	ctx := context.Background()

	// Sets your Google Cloud Platform project ID.
	projectID := "YOUR_PROJECT_ID"

	// Creates a client.
	client, err := bigquery.NewClient(ctx, projectID)
	if err != nil {
		log.Fatalf("Failed to create client: %v", err)
	}

	// Sets the name for the new dataset.
	datasetName := "my_new_dataset"

	// Creates the new BigQuery dataset.
	if err := client.Dataset(datasetName).Create(ctx, &bigquery.DatasetMetadata{}); err != nil {
		log.Fatalf("Failed to create dataset: %v", err)
	}

	fmt.Printf("Dataset created\n")
}

Java

Read more in the Java API Reference Documentation for the Google BigQuery API Client Library.

// Imports the Google Cloud client library
import com.google.cloud.bigquery.BigQuery;
import com.google.cloud.bigquery.BigQueryOptions;
import com.google.cloud.bigquery.Dataset;
import com.google.cloud.bigquery.DatasetInfo;

public class QuickstartSample {
  public static void main(String... args) throws Exception {
    // Instantiate a client. If you don't specify credentials when constructing a client, the
    // client library will look for credentials in the environment, such as the
    // GOOGLE_APPLICATION_CREDENTIALS environment variable.
    BigQuery bigquery = BigQueryOptions.getDefaultInstance().getService();

    // The name for the new dataset
    String datasetName = "my_new_dataset";

    // Prepares a new dataset
    Dataset dataset = null;
    DatasetInfo datasetInfo = DatasetInfo.newBuilder(datasetName).build();

    // Creates the dataset
    dataset = bigquery.create(datasetInfo);

    System.out.printf("Dataset %s created.%n", dataset.getDatasetId().getDataset());
  }
}

Node.js

Read more in the Node.js API Reference Documentation for the Google BigQuery API Client Library.

// Imports the Google Cloud client library
const BigQuery = require('@google-cloud/bigquery');

// Your Google Cloud Platform project ID
const projectId = 'YOUR_PROJECT_ID';

// Creates a client
const bigquery = new BigQuery({
  projectId: projectId,
});

// The name for the new dataset
const datasetName = 'my_new_dataset';

// Creates the new dataset
bigquery
  .createDataset(datasetName)
  .then(results => {
    const dataset = results[0];

    console.log(`Dataset ${dataset.id} created.`);
  })
  .catch(err => {
    console.error('ERROR:', err);
  });

PHP

Read more in the PHP API Reference Documentation for the Google BigQuery API Client Library.

# Includes the autoloader for libraries installed with composer
require __DIR__ . '/vendor/autoload.php';

# Imports the Google Cloud client library
use Google\Cloud\BigQuery\BigQueryClient;

# Your Google Cloud Platform project ID
$projectId = 'YOUR_PROJECT_ID';

# Instantiates a client
$bigquery = new BigQueryClient([
    'projectId' => $projectId
]);

# The name for the new dataset
$datasetName = 'my_new_dataset';

# Creates the new dataset
$dataset = $bigquery->createDataset($datasetName);

echo 'Dataset ' . $dataset->id() . ' created.';

Python

Read more in the Python API Reference Documentation for the Google BigQuery API Client Library.

# Imports the Google Cloud client library
from google.cloud import bigquery

# Instantiates a client
bigquery_client = bigquery.Client()

# The name for the new dataset
dataset_id = 'my_new_dataset'

# Prepares a reference to the new dataset
dataset_ref = bigquery_client.dataset(dataset_id)
dataset = bigquery.Dataset(dataset_ref)

# Creates the new dataset
dataset = bigquery_client.create_dataset(dataset)

print('Dataset {} created.'.format(dataset.dataset_id))

Ruby

Read more in the Ruby API Reference Documentation for the Google BigQuery API Client Library.

# Imports the Google Cloud client library
require "google/cloud/bigquery"

# Your Google Cloud Platform project ID
project_id = "YOUR_PROJECT_ID"

# Instantiates a client
bigquery = Google::Cloud::Bigquery.new project: project_id

# The name for the new dataset
dataset_name = "my_new_dataset"

# Creates the new dataset
dataset = bigquery.create_dataset dataset_name

puts "Dataset #{dataset.dataset_id} created."

Additional Resources

Third party BigQuery client libraries

In addition to the Google-supported client libraries listed in the tables above, a set of third-party libraries are available.

Language Library
Python pandas-gbq
R bigrquery
Scala spark-bigquery

What's next?

Was this page helpful? Let us know how we did:

Send feedback about...