Loading Data from Cloud Storage

This page describes how to load data from Google Cloud Storage into BigQuery.

Before you begin

To load data into BigQuery, you must set up billing and ensure that you have read access to the data source and write access to the destination table.

  1. Sign in to your Google account.

    If you don't already have one, sign up for a new account.

  2. Select or create a Cloud Platform Console project.

    Go to the Projects page

  3. Enable billing for your project.

    Enable billing

  4. Ensure that you have read access to your data source. If you are loading content from Google Cloud Storage, and you are an owner of the project that contains your data source, you probably have read access.

    To set READ access on a Cloud Storage object, see Creating and Managing Access Control Lists (ACLs) in the Cloud Storage documentation.

  5. Ensure that you have write access to your destination table. If you are the owner of the dataset that contains your destination table, you probably have write access.

    To set write access to a dataset in BigQuery:

    1. Go to the BigQuery web UI.
      Go to the BigQuery web UI
    2. In the navigation, hover on a dataset ID and click the down arrow icon down arrow icon image next to the ID and click Share dataset.
    3. Add a person and give that person edit access, then click Save changes.

Loading data from Google Cloud Storage

BigQuery supports loading data from these storage classes:

  • Multi-Regional
  • Regional
  • Nearline
  • Coldline
  • Standard
  • Durable Reduced Availability

To load data from Google Cloud Storage:

Web UI

  1. Go to the BigQuery web UI.
    Go to the BigQuery web UI

  2. In the navigation, hover on a dataset ID and click the down arrow icon down arrow icon image next to the ID and click Create new table.

  3. Under Source Data, select Google Cloud Storage for the Location.

  4. Specify the location of the source data using the path gs://[BUCKET]/[OBJECT].

  5. Under Destination Table, enter a value for the destination table name.

  6. In the Schema section, input the table's schema.

  7. Click the Create Table button.


Use the bq load command and include a Cloud Storage URI for the source argument:



  • [DATASET].[TABLE_NAME] is a fully qualified table name, where [DATASET] represents an existing dataset.
  • [PATH_TO_SOURCE] is a fully-qualified Cloud Storage URI.
  • [SCHEMA] is a valid schema.

For example, the following examples all load data from Cloud Storage:

bq load ds.new_tbl gs://mybucket/info.csv ./info_schema.json
bq load ds.small gs://mybucket/small.csv name:integer,value:string
bq load ds.small gs://mybucket/small.csv field1,field2,field3


This sample uses the Google Cloud Client Library for Python.

def load_data_from_gcs(dataset_name, table_name, source):
    bigquery_client = bigquery.Client()
    dataset = bigquery_client.dataset(dataset_name)
    table = dataset.table(table_name)
    job_name = str(uuid.uuid4())

    job = bigquery_client.load_table_from_storage(
        job_name, table, source)



    print('Loaded {} rows into {}:{}.'.format(
        job.output_rows, dataset_name, table_name))

def wait_for_job(job):
    while True:
        if job.state == 'DONE':
            if job.error_result:
                raise RuntimeError(job.error_result)


This sample uses the Google Cloud Client Library for PHP.

use Google\Cloud\ServiceBuilder;
use Google\Cloud\ExponentialBackoff;

 * @param string $projectId  The Google project ID.
 * @param string $datasetId  The BigQuery dataset ID.
 * @param string $tableId    The BigQuery table ID.
 * @param string $bucketName The Cloud Storage bucket Name.
 * @param string $objectName The Cloud Storage object Name.
function import_from_storage($projectId, $datasetId, $tableId, $bucketName, $objectName)
    // determine the import options from the object name
    $options = [];
    if ('.backup_info' === substr($objectName, -12)) {
        $options['jobConfig'] = ['sourceFormat' => 'DATASTORE_BACKUP'];
    } elseif ('.json' === substr($objectName, -5)) {
        $options['jobConfig'] = ['sourceFormat' => 'NEWLINE_DELIMITED_JSON'];
    // instantiate the bigquery table service
    $builder = new ServiceBuilder([
        'projectId' => $projectId,
    $bigQuery = $builder->bigQuery();
    $dataset = $bigQuery->dataset($datasetId);
    $table = $dataset->table($tableId);
    // load the storage object
    $storage = $builder->storage();
    $object = $storage->bucket($bucketName)->object($objectName);
    // create the import job
    $job = $table->loadFromStorage($object, $options);
    // poll the job until it is complete
    $backoff = new ExponentialBackoff(10);
    $backoff->execute(function () use ($job) {
        print('Waiting for job to complete' . PHP_EOL);
        if (!$job->isComplete()) {
            throw new Exception('Job has not yet completed', 500);
    // check if the job has errors
    if (isset($job->info()['status']['errorResult'])) {
        $error = $job->info()['status']['errorResult']['message'];
        printf('Error running job: %s' . PHP_EOL, $error);
    } else {
        print('Data imported successfully' . PHP_EOL);


This sample uses the Google Cloud Client Library for Node.js.

var Storage = require('@google-cloud/storage');

function importFileFromGCS (datasetId, tableId, bucketName, fileName, callback) {
  var bigquery = BigQuery();
  var storage = Storage();

  var table = bigquery.dataset(datasetId).table(tableId);
  var file = storage.bucket(bucketName).file(fileName);

  // Import the file from Google Cloud Storage
  // See https://googlecloudplatform.github.io/google-cloud-node/#/docs/bigquery/latest/bigquery/table?method=import
  table.import(file, function (err, job, apiResponse) {
    if (err) {
      return callback(err);

    console.log('Started job: %s', job.id);
      .on('error', callback)
      .on('complete', function (metadata) {
        console.log('Completed job: %s', job.id);
        return callback(null, metadata, apiResponse);

For more information about loading data with a POST request, see loading data with a POST request

What's next

Send feedback about...

BigQuery Documentation