After you submit a BigQuery job, you can view job data, list jobs, cancel a job, or rerun a job.
When a job is submitted, it can be in one of three states:
PENDING
: The job is scheduled and waiting to begin running.RUNNING
: The job is in progress.DONE
: The job is completed. In some cases, BigQuery reports this state asSUCCESS
(if the job completed without errors) orFAILURE
(if the job completed with errors).
Viewing job data
You can view job data and metadata by using the Cloud Console, the
bq
command-line tool, and the API. This data includes details such as the job type, the
job state, and the user who ran the job.
Required permissions
At a minimum, in order to get job data and metadata, you must be granted
bigquery.jobs.get
permissions. The following predefined IAM
role includes bigquery.jobs.get
permissions:
bigquery.admin
If you grant an account the bigquery.admin
role, the user can view all job
data in the project no matter who submitted the job.
The following roles are granted bigquery.jobs.get
permissions for self-created
jobs. These users can only view job data for jobs they submit:
bigquery.user
bigquery.jobUser
For more information on IAM roles and permissions in BigQuery, see Access control.
Viewing information about jobs
To view information about a job:
Console
In the navigation pane, click Job history, or click Query history to see information about query jobs.
Click Personal history to view details about your jobs. Click Project history to view details about all jobs run in the project.
bq
Issue the bq show
command with the -j
flag and a job ID.
When you supply the job ID, you can use the fully qualified ID or the short form. For example, job IDs listed in the Cloud Console are fully qualified, that is, they include the project and location:
my-project-1234:US.bquijob_123x456_123y123z123c
Job IDs in the command-line tool are listed using the short form. Project ID and location are not included:
bquijob_123x456_123y123z123c
To specify the job location, supply the --location
flag and set the value
to your location. This flag is optional
if you use the fully qualified job ID. If you include the --location
flag and you're using the fully qualified job ID, the --location
flag is
ignored.
The following command requests information about a job:
bq --location=location show -j job_id
Replace the following:
location
is optional. Location is the name of the location where the job runs. For example, if you are using BigQuery in the Tokyo region, set the flag's value toasia-northeast1
. You can set a default value for the location using the.bigqueryrc
file.job_id
is the ID of the job.
Examples:
The following command gets summary information about job
US.bquijob_123x456_123y123z123c
running in myproject
.
bq show -j myproject:US.bquijob_123x456_123y123z123c
The output looks like the following:
Job Type State Start Time Duration User Email Bytes Processed Bytes Billed Billing Tier Labels ---------- --------- ----------------- ---------- ------------------- ----------------- -------------- -------------- -------- extract SUCCESS 06 Jul 11:32:10 0:01:41 user@example.com
To see full job details, enter:
bq show --format=prettyjson -j myproject:US.bquijob_123x456_789y123z456c
The output looks like the following:
{ "configuration": { "extract": { "compression": "NONE", "destinationUri": "[URI removed]", "destinationUris": [ "[URI removed]" ], "sourceTable": { "datasetId": "github_repos", "projectId": "bigquery-public-data", "tableId": "commits" } } }, "etag": "\"[etag removed]\"", "id": "myproject:bquijob_123x456_789y123z456c", "jobReference": { "jobId": "bquijob_123x456_789y123z456c", "projectId": "[Project ID removed]" }, "kind": "bigquery#job", "selfLink": "https://bigquery.googleapis.com/bigquery/v2/projects/federated-testing/jobs/bquijob_123x456_789y123z456c", "statistics": { "creationTime": "1499365894527", "endTime": "1499365894702", "startTime": "1499365894702" }, "status": { "errorResult": { "debugInfo": "[Information removed for readability]", "message": "Operation cannot be performed on a nested schema. Field: author", "reason": "invalid" }, "errors": [ { "message": "Operation cannot be performed on a nested schema. Field: author", "reason": "invalid" } ], "state": "DONE" }, "user_email": "user@example.com" }
API
Call jobs.get and provide
the jobId
and projectId
parameters. (Optional) Supply the location
parameter and set the value to the location
where the job runs. This parameter is optional if you use the
fully qualified job ID that includes the location, for example,
my-project-1234:US.bquijob_123x456_123y123z123c
.
Go
Before trying this sample, follow the Go setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Go API reference documentation.
Java
Before trying this sample, follow the Java setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Java API reference documentation.
Node.js
Before trying this sample, follow the Node.js setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Node.js API reference documentation.
Python
Before trying this sample, follow the Python setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Python API reference documentation.
Listing jobs in a project
Your project maintains your job history for all jobs created in the past six months.
You can view your BigQuery job history in the following ways:
- Using the Google Cloud Console.
- Using the
bq
command-line tool. - Calling the
jobs.list
API method. - Using the client libraries.
The job history includes jobs that are in the RUNNING
state and jobs that are
DONE
(indicated by reporting the state as SUCCESS
or FAILURE
).
Required permissions
At a minimum, to list jobs, you must be granted bigquery.jobs.list
permissions. The following predefined IAM roles include
bigquery.jobs.list
permissions:
bigquery.user
bigquery.admin
The following role is granted bigquery.jobs.list
permissions only for
self-created jobs. Entities granted this role can only list jobs they submit:
bigquery.jobUser
When you are granted bigquery.jobs.list
permissions, you can list all jobs in
a project, but the details and metadata are redacted for jobs submitted by other
users. bigquery.jobs.list
permissions lets you see full details for
self-created jobs.
To list all jobs, including details for jobs created by other users, you must
be granted bigquery.jobs.listAll
permissions. Only the bigquery.admin
and
bigquery.resourceAdmin
roles have bigquery.jobs.listAll
permissions.
For more information about IAM roles and permissions in BigQuery, see Predefined roles and permissions.
Listing jobs
When you list jobs in a project, you do not need to provide a location. Currently, jobs are listed for all locations.
To list jobs in a project:
Console
In the navigation pane, click Job History.
In the Personal history section, your jobs are listed by creation time with the most recent jobs at the top. The list includes jobs only for the current user. To see all jobs for a project, click Project history. (If you aren't the project owner you may not have permission to view all jobs for a project.)
bq
Issue the bq ls
command with one of the following flags:
-j
is used to identify jobs as the resource to list.--all
or-a
lists jobs from all users. To see full (unredacted) details for all jobs, you must havebigquery.jobs.listAll
permissions.--min_creation_time
is used to list jobs after a supplied timestamp value.--max_creation_time
is used to list jobs before a supplied timestamp value.-n
limits the results. By default, you are limited to 100,000 results.
bq ls -j -a \ --min_creation_time integer1 \ --max_creation_time integer2 \ -n integer3 \ project_id
Replace the following:
integer1
is an integer that represents a timestamp.integer2
is an integer that represents a timestamp.integer3
is an integer that indicates the number of jobs returned.project_id
is the ID of the project that contains the jobs that you're listing. If you set a default project, you do not need to provide theproject_id
parameter.
Examples:
The following command lists all jobs for the current user. Running this
command requires bigquery.jobs.list
permissions.
bq ls -j myproject
The following command lists all jobs for all users. Running this command
requires bigquery.jobs.listAll
permissions.
bq ls -j -a myproject
The following command lists the 10 most recent jobs in myproject
:
bq ls -j -a -n 10 myproject
The following command lists all jobs submitted before Oct 18, 2018, at
4:04:53 PM. This timestamp (in milliseconds) is equivalent to the following
integer value: 1539903893000
.
bq ls -j --max_creation_time 1539903893000
API
Call jobs.list and provide
the projectId
parameter. To list jobs for all users, set the allUsers
parameter to true
. Setting allUsers
to true
requires
bigquery.jobs.listAll
permissions.
Go
Before trying this sample, follow the Go setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Go API reference documentation.
Java
Before trying this sample, follow the Java setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Java API reference documentation.
Node.js
Before trying this sample, follow the Node.js setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Node.js API reference documentation.
Python
Before trying this sample, follow the Python setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Python API reference documentation.
Canceling jobs
You can cancel a RUNNING
or PENDING
job in the following ways:
- Using the Cloud Console.
- Using the
bq
command-line tool. - Using the
BQ.JOBS.CANCEL
system procedure in a SQL query. - By calling the
jobs.cancel
API method. - Using the client libraries.
Note that not all job types can be canceled. If the job cannot be canceled, an error is returned.
Even if the job can be canceled, success is not guaranteed. The job might have completed by the time the cancel request is submitted, or the job might be in a stage where it cannot be canceled.
Required permissions
At a minimum, to cancel a job, you must be granted bigquery.jobs.update
permissions. The following predefined IAM role includes
bigquery.jobs.update
permissions:
bigquery.admin
If you grant an account the bigquery.admin
role, the user can cancel any
eligible job, no matter who submitted it.
The following roles can cancel self-created jobs. These users can only cancel jobs they submit:
bigquery.user
bigquery.jobUser
For more information about IAM roles and permissions in BigQuery, see Predefined roles and permissions.
Canceling a job
To cancel a job:
Console
In the navigation pane, click Job history.
In the Personal history section, click the job you're canceling. The most recent jobs appear at the top of the list.
In the job details, click Cancel job.
SQL
Use the BQ.JOBS.CANCEL
system procedure:
CALL BQ.JOBS.CANCEL('job_id');
Replace job_id with the ID of the job you're canceling.
If you are in a different project but in the same region as the job you want to cancel, you must also include the project ID:
CALL BQ.JOBS.CANCEL('project_id.job_id');
Replace the following:
project_id
is the ID of the project that contains the job that you're canceling.job_id
is the ID of the job that you're canceling.
The procedure returns immediately, and BigQuery cancels the job shortly afterward. If the job has already succeeded or failed, the procedure has no effect.
bq
Issue the bq cancel
command with the job_id
parameter. You can request
cancellation and return immediately by using the --nosync
flag. By
default, cancellation requests wait for completion.
When you supply the job ID, you can use the fully qualified ID or the short form. For example, job IDs listed in the Cloud Console are fully qualified, that is, they include the project and location:
my-project-1234:US.bquijob_123x456_123y123z123c
Job IDs in the bq
command-line tool are listed using the short form. Project ID and
location are not included:
bquijob_123x456_123y123z123c
To specify the job location, supply the --location
flag and set the value
to your location. This flag is optional
if you use the fully qualified job ID. If you include the --location
flag and you're using the fully qualified job ID, the --location
flag is
ignored.
The following command requests job cancellation and waits for completion. If
the fully qualified job ID is supplied, the --location
flag is ignored:
bq --location=location cancel job_id
The following command requests job cancellation and returns immediately. If
the fully qualified job ID is supplied, the --location
flag is ignored:
bq --location=location --nosync cancel job_id
Replace the following:
location
is optional. Location is the name of the location where the job runs. For example, if you are using BigQuery in the Tokyo region, set the flag's value toasia-northeast1
. You can set a default value for the location using the.bigqueryrc
file.job_id
is the ID of the job that you're canceling. If you copy the job ID from the Cloud Console, the project ID and location are included in the job ID. For example,my-project-1234:US.bquijob_123x456_123y123z123c
.
Examples:
The following command cancels job
my-project-1234:US.bquijob_123x456_123y123z123c
running in
the US
multi-region location in my-project-1234
and waits for
completion. Because the fully qualified job ID is used, the location flag is
not supplied.
bq cancel my-project-1234:US.bquijob_123x456_123y123z123c
The following command cancels job bquijob_123x456_123y123z123c
running in
the US
multi-region location in my-project-1234
and waits for
completion. Because the short form of the job ID is used, the --location
flag is supplied.
bq --location=US cancel bquijob_123x456_123y123z123c
The following command cancels job bquijob_123x456_123y123z123c
running in
the US
multi-region location in my-project-1234
and returns immediately.
Because the fully qualified job ID is used, the --location
flag is
not supplied.
bq --nosync cancel my-project-1234:US.bquijob_123x456_123y123z123c
API
Call jobs.cancel and provide
the jobId
and projectId
parameters. Supply the location
parameter and set the value to the location
where the job runs.
Go
Before trying this sample, follow the Go setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Go API reference documentation.
Java
Before trying this sample, follow the Java setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Java API reference documentation.
Node.js
Before trying this sample, follow the Node.js setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Node.js API reference documentation.
Python
Before trying this sample, follow the Python setup instructions in the BigQuery Quickstart Using Client Libraries. For more information, see the BigQuery Python API reference documentation.
Repeating a job
It is not possible to rerun a job using the same job ID. Instead, you create a
new job with the same configuration. When you submit the new job in the
Cloud Console or the bq
command-line tool, a new job ID is assigned. When you
submit the job using the API or client libraries, you must generate a new job
ID.
Required permissions
At a minimum, to run a job, you must be granted bigquery.jobs.create
permissions. The following predefined IAM roles include
bigquery.jobs.create
permissions:
bigquery.user
bigquery.jobUser
bigquery.admin
For more information about IAM roles and permissions in BigQuery, see Predefined roles and permissions.
Rerunning a job
To repeat a job:
Console
To repeat a query job:
In the navigation pane, click Query history.
In the Personal history or Project history section, click the query you want to rerun, and then click Open query in editor.
Click Run.
To repeat a load job:
In the navigation pane, click Job history.
In the Personal history or Project history section, click the job you want to repeat. The most recent jobs appear at the top of the list.
In the job details, click Repeat load job.
bq
Issue your command again and BigQuery automatically generates a job with a new job ID.
API
There is no single-call method to repeat a job; if you want to repeat a specific job:
Call
jobs.get
to retrieve the resource for the job to repeat.Remove the id, status, and statistics field. Change the jobId field to a new value generated by your client code. Change any other fields as necessary.
Call
jobs.insert
with the modified resource and the new job ID to start the new job.