Index
MigrationService(interface)AzureSynapseDialect(message)BigQueryDialect(message)CreateMigrationWorkflowRequest(message)DB2Dialect(message)DeleteMigrationWorkflowRequest(message)Dialect(message)ErrorDetail(message)ErrorLocation(message)GcsReportLogMessage(message)GetMigrationSubtaskRequest(message)GetMigrationWorkflowRequest(message)GreenplumDialect(message)HiveQLDialect(message)ListMigrationSubtasksRequest(message)ListMigrationSubtasksResponse(message)ListMigrationWorkflowsRequest(message)ListMigrationWorkflowsResponse(message)Literal(message)MigrationSubtask(message)MigrationSubtask.State(enum)MigrationTask(message)MigrationTask.State(enum)MigrationTaskResult(message)MigrationWorkflow(message)MigrationWorkflow.State(enum)MySQLDialect(message)NameMappingKey(message)NameMappingKey.Type(enum)NameMappingValue(message)NetezzaDialect(message)ObjectNameMapping(message)ObjectNameMappingList(message)OracleDialect(message)Point(message)PostgresqlDialect(message)PrestoDialect(message)RedshiftDialect(message)ResourceErrorDetail(message)SQLServerDialect(message)SQLiteDialect(message)SnowflakeDialect(message)SourceEnv(message)SourceEnvironment(message)SourceSpec(message)SourceTargetMapping(message)SparkSQLDialect(message)StartMigrationWorkflowRequest(message)TargetSpec(message)TeradataDialect(message)TeradataDialect.Mode(enum)TimeInterval(message)TimeSeries(message)TranslationConfigDetails(message)TranslationDetails(message)TranslationTaskResult(message)TypedValue(message)VerticaDialect(message)
MigrationService
Service to handle EDW migrations.
| CreateMigrationWorkflow |
|---|
|
Creates a migration workflow.
|
| DeleteMigrationWorkflow |
|---|
|
Deletes a migration workflow by name.
|
| GetMigrationSubtask |
|---|
|
Gets a previously created migration subtask.
|
| GetMigrationWorkflow |
|---|
|
Gets a previously created migration workflow.
|
| ListMigrationSubtasks |
|---|
|
Lists previously created migration subtasks.
|
| ListMigrationWorkflows |
|---|
|
Lists previously created migration workflow.
|
| StartMigrationWorkflow |
|---|
|
Starts a previously created migration workflow. I.e., the state transitions from DRAFT to RUNNING. This is a no-op if the state is already RUNNING. An error will be signaled if the state is anything other than DRAFT or RUNNING.
|
AzureSynapseDialect
This type has no fields.
The dialect definition for Azure Synapse.
BigQueryDialect
This type has no fields.
The dialect definition for BigQuery.
CreateMigrationWorkflowRequest
Request to create a migration workflow resource.
| Fields | |
|---|---|
parent |
Required. The name of the project to which this migration workflow belongs. Example: |
migration_workflow |
Required. The migration workflow to create. |
DB2Dialect
This type has no fields.
The dialect definition for DB2.
DeleteMigrationWorkflowRequest
A request to delete a previously created migration workflow.
| Fields | |
|---|---|
name |
Required. The unique identifier for the migration workflow. Example: |
Dialect
The possible dialect options for translation.
| Fields | |
|---|---|
Union field dialect_value. The possible dialect options that this message represents. dialect_value can be only one of the following: |
|
bigquery_dialect |
The BigQuery dialect |
hiveql_dialect |
The HiveQL dialect |
redshift_dialect |
The Redshift dialect |
teradata_dialect |
The Teradata dialect |
oracle_dialect |
The Oracle dialect |
sparksql_dialect |
The SparkSQL dialect |
snowflake_dialect |
The Snowflake dialect |
netezza_dialect |
The Netezza dialect |
azure_synapse_dialect |
The Azure Synapse dialect |
vertica_dialect |
The Vertica dialect |
sql_server_dialect |
The SQL Server dialect |
postgresql_dialect |
The Postgresql dialect |
presto_dialect |
The Presto dialect |
mysql_dialect |
The MySQL dialect |
db2_dialect |
DB2 dialect |
sqlite_dialect |
SQLite dialect |
greenplum_dialect |
Greenplum dialect |
ErrorDetail
Provides details for errors, e.g. issues that where encountered when processing a subtask.
| Fields | |
|---|---|
location |
Optional. The exact location within the resource (if applicable). |
error_info |
Required. Describes the cause of the error with structured detail. |
ErrorLocation
Holds information about where the error is located.
| Fields | |
|---|---|
line |
Optional. If applicable, denotes the line where the error occurred. A zero value means that there is no line information. |
column |
Optional. If applicable, denotes the column where the error occurred. A zero value means that there is no columns information. |
GcsReportLogMessage
A record in the aggregate CSV report for a migration workflow
| Fields | |
|---|---|
severity |
Severity of the translation record. |
category |
Category of the error/warning. Example: SyntaxError |
file_path |
The file path in which the error occurred |
filename |
The file name in which the error occurred |
source_script_line |
Specifies the row from the source text where the error occurred (0 based, -1 for messages without line location). Example: 2 |
source_script_column |
Specifies the column from the source texts where the error occurred. (0 based, -1 for messages without column location) example: 6 |
message |
Detailed message of the record. |
script_context |
The script context (obfuscated) in which the error occurred |
action |
Category of the error/warning. Example: SyntaxError |
effect |
Effect of the error/warning. Example: COMPATIBILITY |
object_name |
Name of the affected object in the log message. |
GetMigrationSubtaskRequest
A request to get a previously created migration subtasks.
| Fields | |
|---|---|
name |
Required. The unique identifier for the migration subtask. Example: |
read_mask |
Optional. The list of fields to be retrieved. |
GetMigrationWorkflowRequest
A request to get a previously created migration workflow.
| Fields | |
|---|---|
name |
Required. The unique identifier for the migration workflow. Example: |
read_mask |
The list of fields to be retrieved. |
GreenplumDialect
This type has no fields.
The dialect definition for Greenplum.
HiveQLDialect
This type has no fields.
The dialect definition for HiveQL.
ListMigrationSubtasksRequest
A request to list previously created migration subtasks.
| Fields | |
|---|---|
parent |
Required. The migration task of the subtasks to list. Example: |
read_mask |
Optional. The list of fields to be retrieved. |
page_size |
Optional. The maximum number of migration tasks to return. The service may return fewer than this number. |
page_token |
Optional. A page token, received from previous When paginating, all other parameters provided to |
filter |
Optional. The filter to apply. This can be used to get the subtasks of a specific tasks in a workflow, e.g. |
ListMigrationSubtasksResponse
Response object for a ListMigrationSubtasks call.
| Fields | |
|---|---|
migration_subtasks[] |
The migration subtasks for the specified task. |
next_page_token |
A token, which can be sent as |
ListMigrationWorkflowsRequest
A request to list previously created migration workflows.
| Fields | |
|---|---|
parent |
Required. The project and location of the migration workflows to list. Example: |
read_mask |
The list of fields to be retrieved. |
page_size |
The maximum number of migration workflows to return. The service may return fewer than this number. |
page_token |
A page token, received from previous When paginating, all other parameters provided to |
ListMigrationWorkflowsResponse
Response object for a ListMigrationWorkflows call.
| Fields | |
|---|---|
migration_workflows[] |
The migration workflows for the specified project / location. |
next_page_token |
A token, which can be sent as |
Literal
Literal data.
| Fields | |
|---|---|
relative_path |
Required. The identifier of the literal entry. |
Union field literal_data. The literal SQL contents. literal_data can be only one of the following: |
|
literal_string |
Literal string data. |
literal_bytes |
Literal byte data. |
MigrationSubtask
A subtask for a migration which carries details about the configuration of the subtask. The content of the details should not matter to the end user, but is a contract between the subtask creator and subtask worker.
| Fields | |
|---|---|
name |
Output only. Immutable. The resource name for the migration subtask. The ID is server-generated. Example: |
task_id |
The unique ID of the task to which this subtask belongs. |
type |
The type of the Subtask. The migration service does not check whether this is a known type. It is up to the task creator (i.e. orchestrator or worker) to ensure it only creates subtasks for which there are compatible workers polling for Subtasks. |
state |
Output only. The current state of the subtask. |
processing_error |
Output only. An explanation that may be populated when the task is in FAILED state. |
resource_error_details[] |
Output only. Provides details to errors and issues encountered while processing the subtask. Presence of error details does not mean that the subtask failed. |
resource_error_count |
The number or resources with errors. Note: This is not the total number of errors as each resource can have more than one error. This is used to indicate truncation by having a |
create_time |
Time when the subtask was created. |
last_update_time |
Time when the subtask was last updated. |
metrics[] |
The metrics for the subtask. |
State
Possible states of a migration subtask.
| Enums | |
|---|---|
STATE_UNSPECIFIED |
The state is unspecified. |
ACTIVE |
The subtask is ready, i.e. it is ready for execution. |
RUNNING |
The subtask is running, i.e. it is assigned to a worker for execution. |
SUCCEEDED |
The subtask finished successfully. |
FAILED |
The subtask finished unsuccessfully. |
PAUSED |
The subtask is paused, i.e., it will not be scheduled. If it was already assigned,it might still finish but no new lease renewals will be granted. |
PENDING_DEPENDENCY |
The subtask is pending a dependency. It will be scheduled once its dependencies are done. |
MigrationTask
A single task for a migration which has details about the configuration of the task.
| Fields | |
|---|---|
id |
Output only. Immutable. The unique identifier for the migration task. The ID is server-generated. |
type |
The type of the task. This must be one of the supported task types: Translation_Teradata2BQ, Translation_Redshift2BQ, Translation_Bteq2BQ, Translation_Oracle2BQ, Translation_HiveQL2BQ, Translation_SparkSQL2BQ, Translation_Snowflake2BQ, Translation_Netezza2BQ, Translation_AzureSynapse2BQ, Translation_Vertica2BQ, Translation_SQLServer2BQ, Translation_Presto2BQ, Translation_MySQL2BQ, Translation_Postgresql2BQ, Translation_SQLite2BQ, Translation_Greenplum2BQ. |
state |
Output only. The current state of the task. |
processing_error |
Output only. An explanation that may be populated when the task is in FAILED state. |
create_time |
Time when the task was created. |
last_update_time |
Time when the task was last updated. |
resource_error_details[] |
Output only. Provides details to errors and issues encountered while processing the task. Presence of error details does not mean that the task failed. |
resource_error_count |
The number or resources with errors. Note: This is not the total number of errors as each resource can have more than one error. This is used to indicate truncation by having a |
metrics[] |
The metrics for the task. |
task_result |
Output only. The result of the task. |
total_processing_error_count |
Count of all the processing errors in this task and its subtasks. |
total_resource_error_count |
Count of all the resource errors in this task and its subtasks. |
Union field task_details. The details of the task. task_details can be only one of the following: |
|
translation_config_details |
Task configuration for CW Batch/Offline SQL Translation. |
translation_details |
Task details for unified SQL Translation. |
State
Possible states of a migration task.
| Enums | |
|---|---|
STATE_UNSPECIFIED |
The state is unspecified. |
PENDING |
The task is waiting for orchestration. |
ORCHESTRATING |
The task is assigned to an orchestrator. |
RUNNING |
The task is running, i.e. its subtasks are ready for execution. |
PAUSED |
Tha task is paused. Assigned subtasks can continue, but no new subtasks will be scheduled. |
SUCCEEDED |
The task finished successfully. |
FAILED |
The task finished unsuccessfully. |
MigrationTaskResult
The migration task result.
| Fields | |
|---|---|
Union field details. Details specific to the task type. details can be only one of the following: |
|
translation_task_result |
Details specific to translation task types. |
MigrationWorkflow
A migration workflow which specifies what needs to be done for an EDW migration.
| Fields | |
|---|---|
name |
Output only. Immutable. Identifier. The unique identifier for the migration workflow. The ID is server-generated. Example: |
display_name |
The display name of the workflow. This can be set to give a workflow a descriptive name. There is no guarantee or enforcement of uniqueness. |
tasks |
The tasks in a workflow in a named map. The name (i.e. key) has no meaning and is merely a convenient way to address a specific task in a workflow. |
state |
Output only. That status of the workflow. |
create_time |
Time when the workflow was created. |
last_update_time |
Time when the workflow was last updated. |
State
Possible migration workflow states.
| Enums | |
|---|---|
STATE_UNSPECIFIED |
Workflow state is unspecified. |
DRAFT |
Workflow is in draft status, i.e. tasks are not yet eligible for execution. |
RUNNING |
Workflow is running (i.e. tasks are eligible for execution). |
PAUSED |
Workflow is paused. Tasks currently in progress may continue, but no further tasks will be scheduled. |
COMPLETED |
Workflow is complete. There should not be any task in a non-terminal state, but if they are (e.g. forced termination), they will not be scheduled. |
MySQLDialect
This type has no fields.
The dialect definition for MySQL.
NameMappingKey
The potential components of a full name mapping that will be mapped during translation in the source data warehouse.
| Fields | |
|---|---|
type |
The type of object that is being mapped. |
database |
The database name (BigQuery project ID equivalent in the source data warehouse). |
schema |
The schema name (BigQuery dataset equivalent in the source data warehouse). |
relation |
The relation name (BigQuery table or view equivalent in the source data warehouse). |
attribute |
The attribute name (BigQuery column equivalent in the source data warehouse). |
Type
The type of the object that is being mapped.
| Enums | |
|---|---|
TYPE_UNSPECIFIED |
Unspecified name mapping type. |
DATABASE |
The object being mapped is a database. |
SCHEMA |
The object being mapped is a schema. |
RELATION |
The object being mapped is a relation. |
ATTRIBUTE |
The object being mapped is an attribute. |
RELATION_ALIAS |
The object being mapped is a relation alias. |
ATTRIBUTE_ALIAS |
The object being mapped is a an attribute alias. |
FUNCTION |
The object being mapped is a function. |
NameMappingValue
The potential components of a full name mapping that will be mapped during translation in the target data warehouse.
| Fields | |
|---|---|
database |
The database name (BigQuery project ID equivalent in the target data warehouse). |
schema |
The schema name (BigQuery dataset equivalent in the target data warehouse). |
relation |
The relation name (BigQuery table or view equivalent in the target data warehouse). |
attribute |
The attribute name (BigQuery column equivalent in the target data warehouse). |
NetezzaDialect
This type has no fields.
The dialect definition for Netezza.
ObjectNameMapping
Represents a key-value pair of NameMappingKey to NameMappingValue to represent the mapping of SQL names from the input value to desired output.
| Fields | |
|---|---|
source |
The name of the object in source that is being mapped. |
target |
The desired target name of the object that is being mapped. |
ObjectNameMappingList
Represents a map of name mappings using a list of key:value proto messages of existing name to desired output name.
| Fields | |
|---|---|
name_map[] |
The elements of the object name map. |
OracleDialect
This type has no fields.
The dialect definition for Oracle.
Point
A single data point in a time series.
| Fields | |
|---|---|
interval |
The time interval to which the data point applies. For |
value |
The value of the data point. |
PostgresqlDialect
This type has no fields.
The dialect definition for Postgresql.
PrestoDialect
This type has no fields.
The dialect definition for Presto.
RedshiftDialect
This type has no fields.
The dialect definition for Redshift.
ResourceErrorDetail
Provides details for errors and the corresponding resources.
| Fields | |
|---|---|
resource_info |
Required. Information about the resource where the error is located. |
error_details[] |
Required. The error details for the resource. |
error_count |
Required. How many errors there are in total for the resource. Truncation can be indicated by having an |
SQLServerDialect
This type has no fields.
The dialect definition for SQL Server.
SQLiteDialect
This type has no fields.
The dialect definition for SQLite.
SnowflakeDialect
This type has no fields.
The dialect definition for Snowflake.
SourceEnv
Represents the default source environment values for the translation.
| Fields | |
|---|---|
default_database |
The default database name to fully qualify SQL objects when their database name is missing. |
schema_search_path[] |
The schema search path. When SQL objects are missing schema name, translation engine will search through this list to find the value. |
metadata_store_dataset |
Optional. Expects a valid BigQuery dataset ID that exists, e.g., project-123.metadata_store_123. If specified, translation will search and read the required schema information from a metadata store in this dataset. If metadata store doesn't exist, translation will parse the metadata file and upload the schema info to a temp table in the dataset to speed up future translation jobs. |
SourceEnvironment
Represents the default source environment values for the translation.
| Fields | |
|---|---|
default_database |
The default database name to fully qualify SQL objects when their database name is missing. |
schema_search_path[] |
The schema search path. When SQL objects are missing schema name, translation engine will search through this list to find the value. |
metadata_store_dataset |
Optional. Expects a validQ BigQuery dataset ID that exists, e.g., project-123.metadata_store_123. If specified, translation will search and read the required schema information from a metadata store in this dataset. If metadata store doesn't exist, translation will parse the metadata file and upload the schema info to a temp table in the dataset to speed up future translation jobs. |
SourceSpec
Represents one path to the location that holds source data.
| Fields | |
|---|---|
encoding |
Optional. The optional field to specify the encoding of the sql bytes. |
Union field source. The specific source SQL. source can be only one of the following: |
|
base_uri |
The base URI for all files to be read in as sources for translation. |
literal |
Source literal. |
SourceTargetMapping
Represents one mapping from a source SQL to a target SQL.
| Fields | |
|---|---|
source_spec |
The source SQL or the path to it. |
target_spec |
The target SQL or the path for it. |
SparkSQLDialect
This type has no fields.
The dialect definition for SparkSQL.
StartMigrationWorkflowRequest
A request to start a previously created migration workflow.
| Fields | |
|---|---|
name |
Required. The unique identifier for the migration workflow. Example: |
TargetSpec
Represents one path to the location that holds target data.
| Fields | |
|---|---|
relative_path |
The relative path for the target data. Given source file |
TeradataDialect
The dialect definition for Teradata.
| Fields | |
|---|---|
mode |
Which Teradata sub-dialect mode the user specifies. |
Mode
The sub-dialect options for Teradata.
| Enums | |
|---|---|
MODE_UNSPECIFIED |
Unspecified mode. |
SQL |
Teradata SQL mode. |
BTEQ |
BTEQ mode (which includes SQL). |
TimeInterval
A time interval extending just after a start time through an end time. If the start time is the same as the end time, then the interval represents a single point in time.
| Fields | |
|---|---|
start_time |
Optional. The beginning of the time interval. The default value for the start time is the end time. The start time must not be later than the end time. |
end_time |
Required. The end of the time interval. |
TimeSeries
The metrics object for a SubTask.
| Fields | |
|---|---|
metric |
Required. The name of the metric. If the metric is not known by the service yet, it will be auto-created. |
value_type |
Required. The value type of the time series. |
metric_kind |
Optional. The metric kind of the time series. If present, it must be the same as the metric kind of the associated metric. If the associated metric's descriptor must be auto-created, then this field specifies the metric kind of the new descriptor and must be either |
points[] |
Required. The data points of this time series. When listing time series, points are returned in reverse time order. When creating a time series, this field must contain exactly one point and the point's type must be the same as the value type of the associated metric. If the associated metric's descriptor must be auto-created, then the value type of the descriptor is determined by the point's type, which must be |
TranslationConfigDetails
The translation config to capture necessary settings for a translation task and subtask.
| Fields | |
|---|---|
source_dialect |
The dialect of the input files. |
target_dialect |
The target dialect for the engine to translate the input to. |
source_env |
The default source environment values for the translation. |
request_source |
The indicator to show translation request initiator. |
target_types[] |
The types of output to generate, e.g. sql, metadata etc. If not specified, a default set of targets will be generated. Some additional target types may be slower to generate. See the documentation for the set of available target types. |
Union field source_location. The chosen path where the source for input files will be found. source_location can be only one of the following: |
|
gcs_source_path |
The Cloud Storage path for a directory of files to translate in a task. |
Union field target_location. The chosen path where the destination for output files will be found. target_location can be only one of the following: |
|
gcs_target_path |
The Cloud Storage path to write back the corresponding input files to. |
Union field output_name_mapping. The mapping of full SQL object names from their current state to the desired output. output_name_mapping can be only one of the following: |
|
name_mapping_list |
The mapping of objects to their desired output names in list form. |
TranslationDetails
The translation details to capture the necessary settings for a translation job.
| Fields | |
|---|---|
source_target_mapping[] |
The mapping from source to target SQL. |
target_base_uri |
The base URI for all writes to persistent storage. |
source_environment |
The default source environment values for the translation. |
target_return_literals[] |
The list of literal targets that will be directly returned to the response. Each entry consists of the constructed path, EXCLUDING the base path. Not providing a target_base_uri will prevent writing to persistent storage. |
target_types[] |
The types of output to generate, e.g. sql, metadata, lineage_from_sql_scripts, etc. If not specified, a default set of targets will be generated. Some additional target types may be slower to generate. See the documentation for the set of available target types. |
TranslationTaskResult
Translation specific result details from the migration task.
| Fields | |
|---|---|
translated_literals[] |
The list of the translated literals. |
report_log_messages[] |
The records from the aggregate CSV report for a migration workflow. |
TypedValue
A single strongly-typed value.
| Fields | |
|---|---|
Union field value. The typed value field. value can be only one of the following: |
|
bool_value |
A Boolean value: |
int64_value |
A 64-bit integer. Its range is approximately |
double_value |
A 64-bit double-precision floating-point number. Its magnitude is approximately |
string_value |
A variable-length string value. |
distribution_value |
A distribution value. |
VerticaDialect
This type has no fields.
The dialect definition for Vertica.