Managing BigQuery Jobs

After you submit a BigQuery job, you can view job data, list jobs, cancel a job, or rerun a job.

When a job is submitted, it can be in one of three states:

  • PENDING — scheduled
  • RUNNING
  • DONE — reported as SUCCESS or FAILURE (if the job completed with errors)

Viewing job data

You can view job data and metadata by using the web UI, CLI, and API. This data includes details such as the job type, the job state, and the user who ran the job.

Required permissions

In order to get job data and metadata, you must have bigquery.jobs.get permissions. The following project-level, predefined IAM role includes bigquery.jobs.get permissions:

If you grant an account the bigquery.admin role, the user can view all job data in the project no matter who submitted the job.

The following roles are granted bigquery.jobs.get permissions for self-created jobs. These users can only view job data for jobs they submit:

For more information on IAM roles and permissions in BigQuery, see access control.

Viewing jobs

To view job data:

Web UI

  1. In the navigation pane, click Job History.

  2. In the Recent Jobs section, click the job to view the details.

CLI

Issue the bq show command with the -j flag and the job_id parameter.

The following command requests information about a job:

bq show -j [JOB_ID]

Where:

  • [JOB_ID] is the ID of the job you're getting.

For example, the following command gets summary information about job bquijob_123x456_123y123z123c running in myproject:

bq show -j myproject:bquijob_123x456_123y123z123c

The output looks like the following:

 Job Type    State      Start Time      Duration      User Email       Bytes Processed   Bytes Billed   Billing Tier   Labels


extract SUCCESS 06 Jul 11:32:10 0:01:41 user@example.com

To see full job details, enter:

bq show --format=prettyjson -j myproject:bquijob_123x456_789y123z456c

The output looks like the following:

{
  "configuration": {
    "extract": {
      "compression": "NONE",
      "destinationUri": "[URI removed]",
      "destinationUris": [
        "[URI removed]"
      ],
      "sourceTable": {
        "datasetId": "github_repos",
        "projectId": "bigquery-public-data",
        "tableId": "commits"
      }
    }
  },
  "etag": "\"[etag removed]\"",
  "id": "myproject:bquijob_123x456_789y123z456c",
  "jobReference": {
    "jobId": "bquijob_123x456_789y123z456c",
    "projectId": "[Project ID removed]"
  },
  "kind": "bigquery#job",
  "selfLink": "https://www.googleapis.com/bigquery/v2/projects/federated-testing/jobs/bquijob_123x456_789y123z456c",
  "statistics": {
    "creationTime": "1499365894527",
    "endTime": "1499365894702",
    "startTime": "1499365894702"
  },
  "status": {
    "errorResult": {
      "debugInfo": "[Information removed for readability]",
      "message": "Operation cannot be performed on a nested schema. Field: author",
      "reason": "invalid"
    },
    "errors": [
      {
        "message": "Operation cannot be performed on a nested schema. Field: author",
        "reason": "invalid"
      }
    ],
    "state": "DONE"
  },
  "user_email": "user@example.com"
}

API

Call jobs.get and provide the jobId and projectId parameters.

Listing jobs in a project

Your project maintains your job history for all jobs created in the past six months. To request automatic deletion of jobs that are more than 50 days old, contact support.

You can view your BigQuery job history via the Google Cloud Platform Console, the CLI, or the API. This history includes jobs that are in the RUNNING state and jobs that are DONE (indicated by reporting the state as SUCCESS or FAILURE).

Required permissions

In order to list jobs, you must have bigquery.jobs.list permissions. The following project-level, predefined IAM roles include bigquery.jobs.list permissions:

If you grant an account the bigquery.admin or bigquery.user role, the user can list all jobs in the project's job history no matter who submitted the job.

The following role is granted bigquery.jobs.list permissions for self-created jobs. These users can only list jobs they submit:

For more information on IAM roles and permissions in BigQuery, see access control.

Listing jobs

To list jobs in a project:

Web UI

  1. In the navigation pane, click Job History.

  2. In the Recent Jobs section, your jobs are listed by creation time with the most recent jobs at the top.

CLI

Issue the bq ls command with the -j flag. The -j flag lists all accessible jobs in a project. To limit the results, use the -n flag.

The following command lists all jobs in a project:

bq ls -j [PROJECT_ID]

The following command lists the most recent jobs in a project:

bq ls -j -n [INTEGER] [PROJECT_ID]

Where:

  • [PROJECT_ID] is the ID of the project that contains the listed jobs.
  • [INTEGER] is an integer that indicates the number of jobs returned.

For example, the following command lists the 10 most recent jobs in myproject:

bq ls -j -n 10 myproject-12312324

API

Call jobs.list and provide the projectId parameter.

Cancelling jobs

You can cancel a RUNNING or PENDING job in the web UI, CLI, or API. However, not all job types can be cancelled. If the job cannot be cancelled, an error is returned.

Even if the job can be cancelled, success is not guaranteed. The job might have completed by the time the cancel request is submitted, or the job might be in a stage where it cannot be cancelled.

Required permissions

In order to cancel a job, you must have bigquery.jobs.update permissions. The following project-level, predefined IAM role includes bigquery.jobs.update permissions:

If you grant an account the bigquery.admin role, the user can cancel any eligible job, no matter who submitted it.

The following roles can cancel self-created jobs. These users can only cancel jobs they submit:

For more information on IAM roles and permissions in BigQuery, see access control.

Cancelling a job

To cancel a job:

Web UI

  1. In the navigation pane, click Job History.

  2. In the Recent Jobs section, click the job you're cancelling. The most recent jobs appear at the top of the list.

  3. In the job details, click Cancel Job.

    Cancel job

CLI

Issue the bq cancel command with the job_id parameter. You can request cancellation and return immediately by using the --nosync flag. By default, cancellation requests wait for completion.

The following command requests job cancellation and waits for completion:

bq cancel [JOB_ID]

The following command requests job cancellation and returns immediately:

bq --nosync cancel [JOB_ID]

Where:

  • [JOB_ID] is the ID of the job you're cancelling.

For example, the following command cancels job bquijob_123x456_123y123z123c running in myproject and waits for completion:

bq cancel myproject:bquijob_123x456_123y123z123c

API

Call jobs.cancel and provide the jobId and projectId parameters.

Repeating a job

It is not possible to rerun a job using the same job ID. Instead you create a new job with the same configuration. When you submit the new job in the web UI or CLI, a new job ID is assigned. When you submit the job using the API or client libraries, you must generate a new job ID.

Required permissions

In order to run a job, you must have bigquery.jobs.create permissions. The following project-level, predefined IAM roles include bigquery.jobs.create permissions:

For more information on IAM roles and permissions in BigQuery, see access control.

Rerunning a job

To repeat a job:

Web UI

To repeat a query job:

  1. In the navigation pane, click Query History.

  2. In the Queries section, to the right of the query, click Open Query.

  3. Click Run Query.

To repeat a load job:

  1. In the navigation pane, click Job History.

  2. In the Recent Jobs section, click the job you want to repeat. The most recent jobs appear at the top of the list.

  3. In the job details, click Repeat Load Job.

CLI

Issue your command again and BigQuery automatically generates a job with a new job ID.

API

There is no single-call method to repeat a job; if you want to repeat a specific job:

  1. Call jobs.get to retrieve the resource for the job to repeat.

  2. Remove the id, status, and statistics field. Change the jobId field to a new value generated by your client code. Change any other fields as necessary.

  3. Call jobs.insert with the modified resource and the new job ID to start the new job.

Send feedback about...