API JobGroups Create v4

This is the latest version of the APIs.

Create a jobGroup, which launches the specified job as the authenticated user.

The request specification depends on one of the following conditions:

  • Dataset has already had a job run against it and just needs to be re-run.
  • Dataset has not had a job run, or the job definition needs to be re-specified.

NOTE: Applying runtime overrides to jobs through the APIs is not supported in your product.

Version: v4

Required Permissions

Access to the APIs must be enabled by the project owner for the project.

NOTE: You must be a project owner to create access tokens.

NOTE: Each request to the Google Cloud Platform must include authentication credentials. See API Authentication.

Request

Request Type: POST

Endpoint:

/v4/jobGroups

Request Body - Run job:

To run a job, you just specify the recipe identifier (wrangledDataset.id). If the job is successful, all defined outputs are generated, as defined in the outputobject, publications, and writeSettings objects associated with the recipe.

{
  "wrangledDataset": {
    "id": 7
  }
}

Request Body - Specify job:

If you must change some outputs or other settings for the specific job, you can insert these changes in the overrides section of the request. Below, the running environment (photon), profiling option, and writeSettings for the job are modified for this execution:

{
  "wrangledDataset": {
    "id": 1
  },
  "overrides": {
    "execution": "photon",
    "profiler": false,
    "writeSettings": [
      {
        "path": "<path_to_output_file>",
        "action": "create",
        "format": "csv",
        "compression": "none",
        "header": false,
        "asSingleFile": false
      }
    ]
  },
  "ranfrom": "ui"
}

Request Body - Specify job with parameter overrides:

If you have created a dataset with parameters, you can specify overrides for parameter values during execution through the APIs. Through this method, you can iterate job executions across all matching sources of a parameterized dataset.

  • You can also apply runtime parameters using the same request structure. For more information on creating datasets with parameters, see Overview of Parameterization.

In the following, the runParameters overrides have been specified for the varRegion and the varReferenceType variables.

{
    "wrangledDataset": {
        "id": 33
    },
    "overrides": {
      "writesettings": [
      {
        "path": "<path_to_output_file>",
        "action": "create",
        "format": "csv",
        "compression": "none",
        "header": false,
        "asSingleFile": false
      }
      ]
    },
    "runParameters": {
      "overrides": {
        "data": [{
          "key": "Region",
          "value": "02"
        },
        {
          "key": "varReferenceType",
          "value": "PROD"
        }
       ]
      }
    },
  "ranfrom": "ui"
}

Response

Response Status Code - Success: 201 - Created

Response Body Example:

{
    "sessionId": "79276c31-c58c-4e79-ae5e-fed1a25ebca1",
    "reason": "JobStarted",
    "jobGraph": {
        "vertices": [
            21,
            22
        ],
        "edges": [
            {
                "source": 21,
                "target": 22
            }
        ]
    },
    "id": 9,
    "jobs": {
        "data": [
            {
                "id": 21
            },
            {
                "id": 22
            }
        ]
    }
}

Reference

Request Reference:

PropertyDescription
wrangledDataset
(required) Internal identifier for the object whose results you wish to generate. The recipes of all preceding datasets on which this dataset depends are executed as part of the job.
ranfrom

(optional) If this value is set to null, then the job does not show up in the Job Details page.

NOTE: This property is not available in your product.

See Job Details Page.

Response reference:

PropertyDescription
reasonCurrent state of the job group at time of API call. Since this call creates the job group, this value is always Job started in the response to this call.
sessionIdSession identifier for the job group.
idInternal identifier of the job group.
jobGraphInternal identifiers of the internal objects executed for the job.
jobsInternal identifiers of the jobs within the job group that were executed as part of this run. Jobs are listed in order of execution.