REST Resource: projects.locations.workflows

Stay organized with collections Save and categorize content based on your preferences.

Resource: MigrationWorkflow

A migration workflow which specifies what needs to be done for an EDW migration.

JSON representation 
{
  "name": string,
  "displayName": string,
  "tasks": {
    string: {
      object (MigrationTask)
    },
    ...
  },
  "state": enum (State),
  "createTime": string,
  "lastUpdateTime": string
}
Fields
name

string

Output only. Immutable. The unique identifier for the migration workflow. The ID is server-generated.

Example: projects/123/locations/us/workflows/345
displayName

string

The display name of the workflow. This can be set to give a workflow a descriptive name. There is no guarantee or enforcement of uniqueness.
tasks

map (key: string, value: object (MigrationTask))

The tasks in a workflow in a named map. The name (i.e. key) has no meaning and is merely a convenient way to address a specific task in a workflow.
state

enum (State)

Output only. That status of the workflow.
createTime

string (Timestamp format)

Time when the workflow was created.
lastUpdateTime

string (Timestamp format)

Time when the workflow was last updated.

MigrationTask

A single task for a migration which has details about the configuration of the task.

JSON representation 
{
  "id": string,
  "type": string,
  "state": enum (State),
  "processingError": {
    object (ErrorInfo)
  },
  "createTime": string,
  "lastUpdateTime": string,
  "orchestrationResult": {
    object (MigrationTaskOrchestrationResult)
  },

  // Union field task_details can be only one of the following:
  "assessmentTaskDetails": {
    object (AssessmentTaskDetails)
  },
  "translationTaskDetails": {
    object (TranslationTaskDetails)
  }
  // End of list of possible types for union field task_details.
}
Fields
id

string

Output only. Immutable. The unique identifier for the migration task. The ID is server-generated.
type

string

The type of the task. This must be a supported task type.
state

enum (State)

Output only. The current state of the task.
processingError

object (ErrorInfo)

Output only. An explanation that may be populated when the task is in FAILED state.
createTime

string (Timestamp format)

Time when the task was created.
lastUpdateTime

string (Timestamp format)

Time when the task was last updated.
orchestrationResult

object (MigrationTaskOrchestrationResult)

Output only. Additional information about the orchestration.
Union field task_details. The details of the task. task_details can be only one of the following:
assessmentTaskDetails

object (AssessmentTaskDetails)

Task configuration for Assessment.
translationTaskDetails

object (TranslationTaskDetails)

Task configuration for Batch/Offline SQL Translation.

AssessmentTaskDetails

Assessment task config.

JSON representation 
{
  "inputPath": string,
  "outputDataset": string,
  "querylogsPath": string,
  "dataSource": string
}
Fields
inputPath

string

Required. The Cloud Storage path for assessment input files.
outputDataset

string

Required. The BigQuery dataset for output.
querylogsPath

string

Optional. An optional Cloud Storage path to write the query logs (which is then used as an input path on the translation task)
dataSource

string

Required. The data source or data warehouse type (eg: TERADATA/REDSHIFT) from which the input data is extracted.

TranslationTaskDetails

The translation task config to capture necessary settings for a translation task and subtask.

JSON representation 
{
  "inputPath": string,
  "outputPath": string,
  "filePaths": [
    {
      object (TranslationFileMapping)
    }
  ],
  "schemaPath": string,
  "fileEncoding": enum (FileEncoding),
  "identifierSettings": {
    object (IdentifierSettings)
  },
  "specialTokenMap": {
    string: enum (TokenType),
    ...
  },
  "filter": {
    object (Filter)
  },
  "translationExceptionTable": string,

  // Union field language_options can be only one of the following:
  "teradataOptions": {
    object (TeradataOptions)
  },
  "bteqOptions": {
    object (BteqOptions)
  }
  // End of list of possible types for union field language_options.
}
Fields
inputPath

string

The Cloud Storage path for translation input files.
outputPath

string

The Cloud Storage path for translation output files.
filePaths[]

object (TranslationFileMapping)

Cloud Storage files to be processed for translation.
schemaPath

string

The Cloud Storage path to DDL files as table schema to assist semantic translation.
fileEncoding

enum (FileEncoding)

The file encoding type.
identifierSettings

object (IdentifierSettings)

The settings for SQL identifiers.
specialTokenMap

map (key: string, value: enum (TokenType))

The map capturing special tokens to be replaced during translation. The key is special token in string. The value is the token data type. This is used to translate SQL query template which contains special token as place holder. The special token makes a query invalid to parse. This map will be applied to annotate those special token with types to let parser understand how to parse them into proper structure with type information.
filter

object (Filter)

The filter applied to translation details.
translationExceptionTable

string

Specifies the exact name of the bigquery table ("dataset.table") to be used for surfacing raw translation errors. If the table does not exist, we will create it. If it already exists and the schema is the same, we will re-use. If the table exists and the schema is different, we will throw an error.
Union field language_options. The language specific settings for the translation task. language_options can be only one of the following:
teradataOptions

object (TeradataOptions)

The Teradata SQL specific settings for the translation task.
bteqOptions

object (BteqOptions)

The BTEQ specific settings for the translation task.

TeradataOptions

Teradata SQL specific translation task related settings.

BteqOptions

BTEQ translation task related settings.

JSON representation 
{
  "projectDataset": {
    object (DatasetReference)
  },
  "defaultPathUri": string,
  "fileReplacementMap": {
    string: string,
    ...
  }
}
Fields
projectDataset

object (DatasetReference)

Specifies the project and dataset in BigQuery that will be used for external table creation during the translation.
defaultPathUri

string

The Cloud Storage location to be used as the default path for files that are not otherwise specified in the file replacement map.
fileReplacementMap

map (key: string, value: string)

Maps the local paths that are used in BTEQ scripts (the keys) to the paths in Cloud Storage that should be used in their stead in the translation (the value).

DatasetReference

Reference to a BigQuery dataset.

JSON representation 
{
  "datasetId": string,
  "projectId": string
}
Fields
datasetId

string

A unique ID for this dataset, without the project name. The ID must contain only letters (a-z, A-Z), numbers (0-9), or underscores (_). The maximum length is 1,024 characters.
projectId

string

The ID of the project containing this dataset.

TranslationFileMapping

Mapping between an input and output file to be translated in a subtask.

JSON representation 
{
  "inputPath": string,
  "outputPath": string
}
Fields
inputPath

string

The Cloud Storage path for a file to translation in a subtask.
outputPath

string

The Cloud Storage path to write back the corresponding input file to.

FileEncoding

The file encoding types.

Enums
FILE_ENCODING_UNSPECIFIED File encoding setting is not specified.
UTF_8 File encoding is UTF_8.
ISO_8859_1 File encoding is ISO_8859_1.
US_ASCII File encoding is US_ASCII.
UTF_16 File encoding is UTF_16.
UTF_16LE File encoding is UTF_16LE.
UTF_16BE File encoding is UTF_16BE.

IdentifierSettings

Settings related to SQL identifiers.

JSON representation 
{
  "outputIdentifierCase": enum (IdentifierCase),
  "identifierRewriteMode": enum (IdentifierRewriteMode)
}
Fields
outputIdentifierCase

enum (IdentifierCase)

The setting to control output queries' identifier case.
identifierRewriteMode

enum (IdentifierRewriteMode)

Specifies the rewrite mode for SQL identifiers.

IdentifierCase

The identifier case type.

Enums
IDENTIFIER_CASE_UNSPECIFIED The identifier case is not specified.
ORIGINAL Identifiers' cases will be kept as the original cases.
UPPER Identifiers will be in upper cases.
LOWER Identifiers will be in lower cases.

IdentifierRewriteMode

The SQL identifier rewrite mode.

Enums
IDENTIFIER_REWRITE_MODE_UNSPECIFIED SQL Identifier rewrite mode is unspecified.
NONE SQL identifiers won't be rewrite.
REWRITE_ALL All SQL identifiers will be rewrite.

TokenType

The special token data type.

Enums
TOKEN_TYPE_UNSPECIFIED Token type is not specified.
STRING Token type as string.
INT64 Token type as integer.
NUMERIC Token type as numeric.
BOOL Token type as boolean.
FLOAT64 Token type as float.
DATE Token type as date.
TIMESTAMP Token type as timestamp.

Filter

The filter applied to fields of translation details.

JSON representation 
{
  "inputFileExclusionPrefixes": [
    string
  ]
}
Fields
inputFileExclusionPrefixes[]

string

The list of prefixes used to exclude processing for input files.

State

Possible states of a migration task.

Enums
STATE_UNSPECIFIED The state is unspecified.
PENDING The task is waiting for orchestration.
ORCHESTRATING The task is assigned to an orchestrator.
RUNNING The task is running, i.e. its subtasks are ready for execution.
PAUSED Tha task is paused. Assigned subtasks can continue, but no new subtasks will be scheduled.
SUCCEEDED The task finished successfully.
FAILED The task finished unsuccessfully.

MigrationTaskOrchestrationResult

Additional information from the orchestrator when it is done with the task orchestration.

JSON representation 
{
  "assessmentDetails": {
    object (AssessmentOrchestrationResultDetails)
  }
}
Fields
assessmentDetails

object (AssessmentOrchestrationResultDetails)

Details specific to assessment task types.

AssessmentOrchestrationResultDetails

Details for an assessment task orchestration result.

JSON representation 
{
  "outputTablesSchemaVersion": string
}
Fields
outputTablesSchemaVersion

string

Optional. The version used for the output table schemas.

State

Possible migration workflow states.

Enums
STATE_UNSPECIFIED Workflow state is unspecified.
DRAFT Workflow is in draft status, i.e. tasks are not yet eligible for execution.
RUNNING Workflow is running (i.e. tasks are eligible for execution).
PAUSED Workflow is paused. Tasks currently in progress may continue, but no further tasks will be scheduled.
COMPLETED Workflow is complete. There should not be any task in a non-terminal state, but if they are (e.g. forced termination), they will not be scheduled.

Methods

create

 Creates a migration workflow.

delete

 Deletes a migration workflow by name.

get

 Gets a previously created migration workflow.

list

 Lists previously created migration workflow.

start

 Starts a previously created migration workflow.