REST Resource: projects.locations.metadataJobs

Resource: MetadataJob

A metadata job resource.

JSON representation 
{
  "name": string,
  "uid": string,
  "createTime": string,
  "updateTime": string,
  "labels": {
    string: string,
    ...
  },
  "type": enum (Type),
  "status": {
    object (Status)
  },

  // Union field spec can be only one of the following:
  "importSpec": {
    object (ImportJobSpec)
  }
  // End of list of possible types for union field spec.

  // Union field result can be only one of the following:
  "importResult": {
    object (ImportJobResult)
  }
  // End of list of possible types for union field result.
}
Fields
name

string

Output only. The name of the resource that the configuration is applied to, in the format projects/{project_number}/locations/{locationId}/metadataJobs/{metadataJobId}.
uid

string

Output only. A system-generated, globally unique ID for the metadata job. If the metadata job is deleted and then re-created with the same name, this ID is different.
createTime

string (Timestamp format)

Output only. The time when the metadata job was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. The time when the metadata job was updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
labels

map (key: string, value: string)

Optional. User-defined labels.

An object containing a list of "key": value pairs. Example: { "name": "wrench", "mass": "1.3kg", "count": "3" }.
type

enum (Type)
status

object (Status)

Union field spec.

spec can be only one of the following:
importSpec

object (ImportJobSpec)

Import job specification.

Union field result.

result can be only one of the following:
importResult

object (ImportJobResult)

Output only. Import job result.

Type

Metadata job type.

Enums
TYPE_UNSPECIFIED Unspecified.
IMPORT Import job.

ImportJobSpec

Job specification for a metadata import job

JSON representation 
{
  "sourceStorageUri": string,
  "sourceCreateTime": string,
  "scope": {
    object (ImportJobScope)
  },
  "entrySyncMode": enum (SyncMode),
  "aspectSyncMode": enum (SyncMode),
  "logLevel": enum (LogLevel)
}
Fields
sourceStorageUri

string

Optional. The URI of a Cloud Storage bucket or folder (beginning with gs:// and ending with /) that contains the metadata import files for this job.

A metadata import file defines the values to set for each of the entries and aspects in a metadata job. For more information about how to create a metadata import file and the file requirements, see Metadata import file.

You can provide multiple metadata import files in the same metadata job. The bucket or folder must contain at least one metadata import file, in JSON Lines format (either .json or .jsonl file extension).

In FULL entry sync mode, don't save the metadata import file in a folder named SOURCE_STORAGE_URI/deletions/.

Caution: If the metadata import file contains no data, all entries and aspects that belong to the job's scope are deleted.
sourceCreateTime

string (Timestamp format)

Optional. The time when the process that created the metadata import files began.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
scope

object (ImportJobScope)

Required. A boundary on the scope of impact that the metadata import job can have.
entrySyncMode

enum (SyncMode)

Required. The sync mode for entries. Only FULL mode is supported for entries. All entries in the job's scope are modified. If an entry exists in Dataplex but isn't included in the metadata import file, the entry is deleted when you run the metadata job.
aspectSyncMode

enum (SyncMode)

Required. The sync mode for aspects. Only INCREMENTAL mode is supported for aspects. An aspect is modified only if the metadata import file includes a reference to the aspect in the updateMask field and the aspectKeys field.
logLevel

enum (LogLevel)

Optional. The level of logs to write to Cloud Logging for this job.

Debug-level logs provide highly-detailed information for troubleshooting, but their increased verbosity could incur additional costs that might not be merited for all jobs.

If unspecified, defaults to INFO.

ImportJobScope

A boundary on the scope of impact that the metadata import job can have.

JSON representation 
{
  "entryGroups": [
    string
  ],
  "entryTypes": [
    string
  ],
  "aspectTypes": [
    string
  ]
}
Fields
entryGroups[]

string

Required. The entry group that is in scope for the import job, specified as a relative resource name in the format projects/{project_number_or_id}/locations/{locationId}/entryGroups/{entryGroupId}. Only entries that belong to the specified entry group are affected by the job.

Must contain exactly one element. The entry group and the job must be in the same location.
entryTypes[]

string

Required. The entry types that are in scope for the import job, specified as relative resource names in the format projects/{project_number_or_id}/locations/{locationId}/entryTypes/{entryTypeId}. The job modifies only the entries that belong to these entry types.

If the metadata import file attempts to modify an entry whose type isn't included in this list, the import job is halted before modifying any entries or aspects.

The location of an entry type must either match the location of the job, or the entry type must be global.
aspectTypes[]

string

Optional. The aspect types that are in scope for the import job, specified as relative resource names in the format projects/{project_number_or_id}/locations/{locationId}/aspectTypes/{aspectTypeId}. The job modifies only the aspects that belong to these aspect types.

If the metadata import file attempts to modify an aspect whose type isn't included in this list, the import job is halted before modifying any entries or aspects.

The location of an aspect type must either match the location of the job, or the aspect type must be global.

SyncMode

Specifies how the entries and aspects in a metadata job are updated.

Enums
SYNC_MODE_UNSPECIFIED Sync mode unspecified.
FULL All resources in the job's scope are modified. If a resource exists in Dataplex but isn't included in the metadata import file, the resource is deleted when you run the metadata job. Use this mode to perform a full sync of the set of entries in the job scope.
INCREMENTAL Only the entries and aspects that are explicitly included in the metadata import file are modified. Use this mode to modify a subset of resources while leaving unreferenced resources unchanged.

LogLevel

The level of logs to write to Cloud Logging for this job.

Enums
LOG_LEVEL_UNSPECIFIED Log level unspecified.
DEBUG

Debug-level logging. Captures detailed logs for each import item. Use debug-level logging to troubleshoot issues with specific import items. For example, use debug-level logging to identify resources that are missing from the job scope, entries or aspects that don't conform to the associated entry type or aspect type, or other misconfigurations with the metadata import file.

Depending on the size of your metadata job and the number of logs that are generated, debug-level logging might incur additional costs.
INFO Info-level logging. Captures logs at the overall job level. Includes aggregate logs about import items, but doesn't specify which import item has an error.

ImportJobResult

Results from a metadata import job.

JSON representation 
{
  "deletedEntries": string,
  "updatedEntries": string,
  "createdEntries": string,
  "unchangedEntries": string,
  "recreatedEntries": string,
  "updateTime": string
}
Fields
deletedEntries

string (int64 format)

Output only. The total number of entries that were deleted.
updatedEntries

string (int64 format)

Output only. The total number of entries that were updated.
createdEntries

string (int64 format)

Output only. The total number of entries that were created.
unchangedEntries

string (int64 format)

Output only. The total number of entries that were unchanged.
recreatedEntries

string (int64 format)

Output only. The total number of entries that were recreated.
updateTime

string (Timestamp format)

Output only. The time when the status was updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

Status

Metadata job status.

JSON representation 
{
  "state": enum (State),
  "message": string,
  "completionPercent": integer,
  "updateTime": string
}
Fields
state

enum (State)

Output only. State of the metadata job.
message

string

Output only. Message relating to the progression of a metadata job.
completionPercent

integer

Output only. Progress tracking.
updateTime

string (Timestamp format)

Output only. The time when the status was updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

State

State of a metadata job.

Enums
STATE_UNSPECIFIED State unspecified.
QUEUED The job is queued.
RUNNING The job is running.
CANCELING The job is being canceled.
CANCELED The job is canceled.
SUCCEEDED The job succeeded.
FAILED The job failed.
SUCCEEDED_WITH_ERRORS The job completed with some errors.

Methods

cancel

 Cancels a metadata job.

create

 Creates a metadata job.

get

 Gets a metadata job.

list

 Lists metadata jobs.