Method: googleapis.dataflow.v1b3.projects.jobs.create

Creates a Cloud Dataflow job. To create a job, we recommend using projects.locations.jobs.create with a regional endpoint. Using projects.jobs.create is not recommended, as your job will always start in us-central1.

This method waits—the workflow execution is paused—until the operation is complete, fails, or times out. The default timeout value is 1800 seconds (30 minutes) and can be changed to a maximum value of 31536000 seconds (one year) for long-running operations using the connector_params field. See the Connectors reference.

The connector uses polling to monitor the long-running operation, which might generate additional billable steps. For more information about retries and long-running operations, refer to Understand connectors.

The polling policy for the long-running operation can be configured. To set the connector-specific parameters (connector_params), refer to Invoke a connector call.

Arguments

Parameters
projectId

string

Required. The ID of the Cloud Platform project that the job belongs to.

location

string

The regional endpoint that contains this job.

replaceJobId

string

Deprecated. This field is now in the Job message.

view

enum

The level of information requested in response.

Enum type. Can be one of the following:
JOB_VIEW_UNKNOWN The job view to return isn't specified, or is unknown. Responses will contain at least the JOB_VIEW_SUMMARY information, and may contain additional information.
JOB_VIEW_SUMMARY Request summary information only: Project ID, Job ID, job name, job type, job status, start/end time, and Cloud SDK version details.
JOB_VIEW_ALL Request all information available for this job.
JOB_VIEW_DESCRIPTION Request summary info and limited job description data for steps, labels and environment.
body

object (Job)

Required.

Raised exceptions

Exceptions
ConnectionError In case of a network problem (such as DNS failure or refused connection).
HttpError If the response status is >= 400 (excluding 429 and 503).
TimeoutError If a long-running operation takes longer to finish than the specified timeout limit.
TypeError If an operation or function receives an argument of the wrong type.
ValueError If an operation or function receives an argument of the right type but an inappropriate value. For example, a negative timeout.
OperationError If the long-running operation finished unsuccessfully.
ResponseTypeError If the long-running operation returned a response of the wrong type.

Response

If successful, the response contains an instance of Job.

Subworkflow snippet

Some fields might be optional or required. To identify required fields, refer to the API documentation.

YAML

- create:
    call: googleapis.dataflow.v1b3.projects.jobs.create
    args:
        projectId: ...
        location: ...
        replaceJobId: ...
        view: ...
        body:
            clientRequestId: ...
            createTime: ...
            createdFromSnapshotId: ...
            currentState: ...
            currentStateTime: ...
            environment:
                clusterManagerApiService: ...
                dataset: ...
                debugOptions:
                    enableHotKeyLogging: ...
                experiments: ...
                flexResourceSchedulingGoal: ...
                internalExperiments: ...
                sdkPipelineOptions: ...
                serviceAccountEmail: ...
                serviceKmsKeyName: ...
                serviceOptions: ...
                tempStoragePrefix: ...
                userAgent: ...
                version: ...
                workerPools: ...
                workerRegion: ...
                workerZone: ...
            executionInfo:
                stages: ...
            id: ...
            jobMetadata:
                bigTableDetails: ...
                bigqueryDetails: ...
                datastoreDetails: ...
                fileDetails: ...
                pubsubDetails: ...
                sdkVersion:
                    sdkSupportStatus: ...
                    version: ...
                    versionDisplayName: ...
                spannerDetails: ...
            labels: ...
            location: ...
            name: ...
            pipelineDescription:
                displayData: ...
                executionPipelineStage: ...
                originalPipelineTransform: ...
            projectId: ...
            replaceJobId: ...
            replacedByJobId: ...
            requestedState: ...
            satisfiesPzs: ...
            stageStates: ...
            startTime: ...
            steps: ...
            stepsLocation: ...
            tempFiles: ...
            transformNameMapping: ...
            type: ...
    result: createResult

JSON

[
  {
    "create": {
      "call": "googleapis.dataflow.v1b3.projects.jobs.create",
      "args": {
        "projectId": "...",
        "location": "...",
        "replaceJobId": "...",
        "view": "...",
        "body": {
          "clientRequestId": "...",
          "createTime": "...",
          "createdFromSnapshotId": "...",
          "currentState": "...",
          "currentStateTime": "...",
          "environment": {
            "clusterManagerApiService": "...",
            "dataset": "...",
            "debugOptions": {
              "enableHotKeyLogging": "..."
            },
            "experiments": "...",
            "flexResourceSchedulingGoal": "...",
            "internalExperiments": "...",
            "sdkPipelineOptions": "...",
            "serviceAccountEmail": "...",
            "serviceKmsKeyName": "...",
            "serviceOptions": "...",
            "tempStoragePrefix": "...",
            "userAgent": "...",
            "version": "...",
            "workerPools": "...",
            "workerRegion": "...",
            "workerZone": "..."
          },
          "executionInfo": {
            "stages": "..."
          },
          "id": "...",
          "jobMetadata": {
            "bigTableDetails": "...",
            "bigqueryDetails": "...",
            "datastoreDetails": "...",
            "fileDetails": "...",
            "pubsubDetails": "...",
            "sdkVersion": {
              "sdkSupportStatus": "...",
              "version": "...",
              "versionDisplayName": "..."
            },
            "spannerDetails": "..."
          },
          "labels": "...",
          "location": "...",
          "name": "...",
          "pipelineDescription": {
            "displayData": "...",
            "executionPipelineStage": "...",
            "originalPipelineTransform": "..."
          },
          "projectId": "...",
          "replaceJobId": "...",
          "replacedByJobId": "...",
          "requestedState": "...",
          "satisfiesPzs": "...",
          "stageStates": "...",
          "startTime": "...",
          "steps": "...",
          "stepsLocation": "...",
          "tempFiles": "...",
          "transformNameMapping": "...",
          "type": "..."
        }
      },
      "result": "createResult"
    }
  }
]