Preview

This product or feature is covered by the Pre-GA Offerings Terms of the Google Cloud Terms of Service. Pre-GA products and features might have limited support, and changes to pre-GA products and features might not be compatible with other pre-GA versions. For more information, see the launch stage descriptions.

See the supported connectors for Application Integration.

Connectors task

The Connectors task lets you seamlessly connect to the various Google Cloud services and other business applications from your integration using the out-of-the-box connectors available in Integration Connectors.

For the list of all the supported connectors for Application Integration, see Connector reference.

Before you begin

Ensure that you have an active connection created in Integration Connectors using the same Google Cloud project as your integration.

For information about how to create and manage connections, see Create and manage connections.

Configure the Connectors task

To configure a Connectors task, perform the following steps:

Note: Configuration details such as authentication will vary depending upon the connection type.

1. In the Google Cloud console, go to the Application Integration page.

   Go to Application Integration

2. In the navigation menu, click Integrations.

   The Integrations List page appears.

3. Select an existing integration or create a new integration by clicking Create integration.

   If you are creating a new integration:

   1. Enter a name and description in the Create Integration dialog.
   2. Select a Region for the integration from the list of supported regions.
   3. Click Create.

   This opens the integration in the integration designer.

4. In the integration designer navigation bar, click +Add a task/trigger > Tasks to view the list of available tasks.
5. Click and place the Connectors element in the integration designer.
6. Click the Connectors task element on the designer to view the Connectors task configuration pane.

   Optionally, click to edit the task name. This enables you to change the task name from the generic Connectors to a meaningful name for your integration.

7. Click Configure task, and do the following in the Configure connector task dialog:
1. Select a region. Select the region of the connection.

   The list of available connections for the selected region is displayed in the Connection section.

2. Select a connection.
3. In the Type column, select Entities or Actions.
   • If you select Entities, the list of supported entities is displayed in the Entity column. Select an entity followed by the Operation that you'd want to perform on that entity.
   • If you select Actions, the list of supported actions is displayed in the Action column. Select an action for the connection.
   For detailed information on entities and actions, see Entities, operations, and actions.
   Note: Not all connection types support Entities or Actions. If a connection type doesn't support any actions or entities, the corresponding Entities and Actions list will be empty.
4. Click Done to complete the connection configuration and close the pane.

The following image shows a sample layout of the Configure connector task dialog for an SFTP connection.

Configure task input and output variables

The Connectors task configuration pane displays Task Input and Task Output variables that are automatically generated based on the Entity and Operation or Action selected in the Configure connector task dialog. These variables are configurable and are accessible as inputs to the current task or as outputs to subsequent tasks, or conditionals in the current integration.

To configure the Task Input or Task Output variables, click the respective variable to open the Configure Variable pane and perform the following steps:

1. Enter the variable value in the Default Value field.
2. (Optional) Select Use as an input to integration or Use as an output to integration.
3. Click Save.

For more information about the input and output parameters of the Connectors task, see Configuration properties.

Entities, operations, and actions

All the Integration Connectors provide a layer of abstraction for the objects of the connected application. You can access an application's objects only through this abstraction. The abstraction is exposed to you as entities, operations, and actions.

Note: You can view the entities, operations, and actions of a connector in the Connectors task.

• Entity: An entity can be thought of as an object, or a collection of properties, in the connected application or service. The definition of an entity differs from a connector to a connector. For example, in a database connector, tables are the entities, in a file server connector, folders are the entities, and in a messaging system connector, queues are the entities.

  However, it is possible that a connector doesn't support or have any entities, in which case the Entities list will be empty. For example, the Pub/Sub connector doesn't have any entity, but only has the publishMessage action.

• Operation: An operation is the activity that you can perform on an entity. You can perform any of the following operations on an entity:
  • List
  • Get
  • Create
  • Update
  • Delete

  Selecting an entity from the available list, generates a list of operations available for the entity. For a detailed description of the operations, see the Connectors task's entity operations. However, it is possible that a connector doesn't support any entity operations, in which case the Operations list will be empty.

• Action: An action is a first class function that is made available to the integration through the connector interface. An action lets you make changes to an entity or entities, and vary from connector to connector. However, it is possible that a connector doesn't support any action, in which case the Actions list will be empty.

Entity operations

You can perform CRUD (Create, Read, Update, Delete) operations on the entities of a connector. Each of these entity operations has a different set of input and output parameters. The following table lists the input and output parameters of the various entity operations.

Operation name	Input parameters	Output parameters
List	listEntitiesPageSize listEntitiesPageToken listEntitiesSortByColumns filterClause	connectorOutputPayload listEntitiesNextPageToken
Get	entityId	connectorOutputPayload
	Note: You can use the Get operation only to fetch a single record from a table that has a primary key. Alternately, you can use the List operation with a filterClause.
Create	connectorInputPayload	connectorOutputPayload
Update	connectorInputPayload entityId filterClause	connectorOutputPayload
Delete	entityId filterClause	N/A

Input parameters

The following table describes the input parameters for the various entity operations.

Parameter name	Data type	Description
entityId	String	A unique identifier of the row that you want to access. Normally, the entityId is a primary key value of a table or a dataset. If you specify a value for the entityId and the table or dataset doesn't have a primary key column, integration reports a runtime error and the Connectors task fails. For example, to get a specific row from a MySQL table, the entityId will be a primary key value in the table. Similarly, to update a specific row in a BigQuery dataset, the entityId will be a primary key value in the dataset.
connectorInputPayload	JSON	The actual data to be added or updated in an entity. The following example shows the JSON snippet of a row data to be added in a table: { "employee_first_name": "John", "employee_emailID": "[e-mail]" } In this example, employee_first_name and employee_emailID are the column names with the corresponding values John and [e-mail].
filterClause	String	Restricts the result of the operations based on a condition. For more information about adding a filter clause, see Add a filter for an operation.
listEntitiesPageSize	Integer	Specifies the number of results that should be returned in a page. A page is a logical grouping of the records in a result set. The concept of a page is useful when you are expecting a large number of records in the result set. If the result set is large, the Connectors task might fail, as there is a limit on the data size that the Connectors task can process. By breaking down the result set into smaller chunks, you can avoid this issue. For example, if you are expecting 1000 records in your result set, you can set the listEntitiesPageSize to 100. So when the Connectors task runs for the first time, it returns the first 100 records, the next 100 records in the second run and so on. Note: The default page size is 25 and the maximum number of pages supported by the task is 200. Tip: Use the listEntitiesPageSize parameter in conjunction with the listEntitiesPageToken parameter to navigate through the pages.
listEntitiesPageToken	String	A page identifier (token) that lets you access a specific page. You can get the value of a page token from the listEntitiesNextPageToken output parameter. Because each page has a unique token, you have the flexibility to access any page you want in the result set. To understand the usage of this parameter, also read the description of the listEntitiesNextPageToken output parameter.
listEntitiesSortByColumns	String array	Column name(s) by which you want to sort the result set. If you want to sort the result set by more than one column, specify multiple column names as comma-separated values.

Output parameters

The following table describes the output parameters for the various entity operations.

Parameter name	Data type	Description
connectorOutputPayload	JSON	The output of an operation in JSON format.
listEntitiesNextPageToken	String	A system generated identifier for a page. You can think of the token as a pointer by which you can access a particular page of the result set. If you have broken down your result set into multiple pages by setting the listEntitiesPageSize parameter, you need a mechanism to navigate through the pages. The listEntitiesNextPageToken output parameter lets you do exactly that. Every time the Connectors task runs, the system generates a token for the next page and sets the listEntitiesNextPageToken's value to the newly generated token. You can then use this token to access the next page in the result set. To access the next page, you must set the listEntitiesPageToken input parameter to the next page's token value. For example, consider you have set the listEntitiesPageSize parameter to 2 and when then Connectors task runs for the first time, the listEntitiesNextPageToken is set to the ChoKC2VtcGxveWVlX2lkEgkRAAAAAAAA8D8YDw== token value. You can then set the listEntitiesPageToken input parameter to this token value to fetch the next page in the subsequent run of the Connectors task. If your result set has a large number of pages, you can consider using the For Each Loop task to repeatedly call the Connectors task and use the Data Mapping task to automatically assign token values to the listEntitiesPageToken input parameter after each run.

Filter clause for entity operations

You can restrict the records that are processed by the Connectors task using the Filter clause variable that is available as a Task Input. For example, in case of a delete operation, you can add a filter clause to delete records with a specific orderId.

Filter clause can be applied only for the following entity operations:

• List
• Delete
• Update

When you select any of these operations, the Task Input section of the Connectors task displays the Filter clause field automatically.

Add a filter clause

To add a filter clause, perform the following steps:

1. Click the Connectors task element on the designer to view the Connectors task configuration pane.
2. Expand the Task Input section and click the filterClause(Connectors) string variable.

   The Configure Variable dialog appears.

3. Enter the filter clause (following the clause syntax) in the Default Value field.
4. Click Save.

Filter clause syntax and examples

A filter clause has the following format:

<FIELD_NAME> <CONDITION> <FILTER_VALUE>

Examples

• OwnerId = '0053t000007941XAAQ'

• PoNumber < 2345

• OrderNumber = 00110 AND StatusCode = 'Draft'

• TotalAmount > 2500

• ShippingPostalCode = 94043 OR ShippingPostalCode = 77002

Important: In the filter clause, specify the string values within single quotes ( ' ) or double quotes ( " ) depending on the database you are using. For example, MySQL accepts both single and double quotes, whereas BigQuery accepts only single quotes.

Use of variables in filter clause

You can't directly use an integration variable in a filter clause. If you want to use an integration variable, you must first configure a Data Mapping task to create a mapping between the integration variable and the filter clause.

The following table shows a sample mapping between an integration variable and the filterClause(Connectors) variable:

Input	Output
PRIMARY_KEY_ID = ' .CONCAT(INTEGRATION_VARIABLE) .CONCAT(')	filterClause(Connectors)

Where, PRIMARY_KEY_ID = ' is entered as a Value in the input row.

Best practices

• For information about the usage limits that apply to the Connectors task, see Usage limits.
• For information about how to troubleshoot a failed Connectors task, see Use logs to troubleshoot and trace connector failures.

Retry on failure

You can configure various retry strategies to handle errors in a task. The retry strategies allow you to specify how to rerun the task or integration in case of an error. For more information, see Error handling strategies.