- Resource: IntegrationVersion
- JSON representation
- TaskConfig
- EventParameter
- FailurePolicy
- RetryStrategy
- NextTask
- NextTasksExecutionPolicy
- TaskExecutionStrategy
- SuccessPolicy
- FinalState
- JsonValidationOption
- TriggerConfig
- TriggerType
- IntegrationAlertConfig
- MetricType
- ThresholdType
- ThresholdValue
- CloudSchedulerConfig
- IntegrationParameter
- InOutType
- IntegrationState
- DatabasePersistencePolicy
- Methods
Resource: IntegrationVersion
The integration version definition.
JSON representation |
---|
{ "name": string, "description": string, "taskConfigs": [ { object ( |
Fields | |
---|---|
name |
Output only. Auto-generated primary key. |
description |
Optional. The integration description. |
taskConfigs[] |
Optional. Task configuration for the integration. It's optional, but the integration doesn't do anything without taskConfigs. |
triggerConfigs[] |
Optional. Trigger configurations. |
integrationParameters[] |
Optional. Parameters that are expected to be passed to the integration when an event is triggered. This consists of all the parameters that are expected in the integration execution. This gives the user the ability to provide default values, add information like PII and also provide data types of each parameter. |
state |
Output only. User should not set it as an input. |
snapshotNumber |
Optional. An increasing sequence that is set when a new snapshot is created. The last created snapshot can be identified by [workflowName, org_id latest(snapshotNumber)]. However, last created snapshot need not be same as the HEAD. So users should always use "HEAD" tag to identify the head. |
updateTime |
Output only. Auto-generated. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
lockHolder |
Optional. The edit lock holder's email address. Generated based on the End User Credentials/LOAS role of the user making the call. |
createTime |
Output only. Auto-generated. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
lastModifierEmail |
Optional. The last modifier's email address. Generated based on the End User Credentials/LOAS role of the user making the call. |
parentTemplateId |
Optional. The id of the template which was used to create this integrationVersion. |
userLabel |
Optional. A user-defined label that annotates an integration version. Typically, this is only set when the integration version is created. |
databasePersistencePolicy |
Optional. Flag to disable database persistence for execution data, including event execution info, execution export info, execution metadata index and execution param index. |
TaskConfig
The task configuration details. This is not the implementation of Task. There might be multiple TaskConfigs for the same Task.
JSON representation |
---|
{ "task": string, "taskId": string, "parameters": { string: { object ( |
Fields | |
---|---|
task |
Optional. The name for the task. |
taskId |
Required. The identifier of this task within its parent event config, specified by the client. This should be unique among all the tasks belong to the same event config. We use this field as the identifier to find next tasks (via field |
parameters |
Optional. The customized parameters the user can pass to this task. An object containing a list of |
failurePolicy |
Optional. Determines the number of times the task will be retried on failure and with what retry strategy. This is applicable for asynchronous calls to Eventbus alone (events.post To Queue, events.schedule etc.). |
synchronousCallFailurePolicy |
Optional. Determines the number of times the task will be retried on failure and with what retry strategy. This is applicable for synchronous calls to Eventbus alone (events.post). |
nextTasks[] |
Optional. The set of tasks that are next in line to be executed as per the execution graph defined for the parent event, specified by |
nextTasksExecutionPolicy |
Optional. The policy dictating the execution of the next set of tasks for the current task. |
taskExecutionStrategy |
Optional. The policy dictating the execution strategy of this task. |
displayName |
Optional. User-provided label that is attached to this TaskConfig in the UI. |
successPolicy |
Optional. Determines what action to take upon successful task completion. |
jsonValidationOption |
Optional. If set, overrides the option configured in the Task implementation class. |
description |
Optional. User-provided description intended to give additional business context about the task. |
taskTemplate |
Optional. Used to define task-template name if task is of type task-template |
EventParameter
This message is used for processing and persisting (when applicable) key value pair parameters for each event in the event bus.
JSON representation |
---|
{
"key": string,
"value": {
object ( |
Fields | |
---|---|
key |
Key is used to retrieve the corresponding parameter value. This should be unique for a given fired event. These parameters must be predefined in the integration definition. |
value |
Values for the defined keys. Each value can either be string, int, double or any proto message. |
FailurePolicy
Policy that defines the task retry logic and failure type. If no FailurePolicy is defined for a task, all its dependent tasks will not be executed (i.e, a retryStrategy
of NONE will be applied).
JSON representation |
---|
{
"retryStrategy": enum ( |
Fields | |
---|---|
retryStrategy |
Defines what happens to the task upon failure. |
maxRetries |
Required if retryStrategy is FIXED_INTERVAL or LINEAR/EXPONENTIAL_BACKOFF/RESTART_INTEGRATION_WITH_BACKOFF. Defines the number of times the task will be retried if failed. |
intervalTime |
Required if retryStrategy is FIXED_INTERVAL or LINEAR/EXPONENTIAL_BACKOFF/RESTART_INTEGRATION_WITH_BACKOFF. Defines the initial interval in seconds for backoff. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
RetryStrategy
The behavior when the taks failed.
Enums | |
---|---|
RETRY_STRATEGY_UNSPECIFIED |
UNSPECIFIED. |
IGNORE |
Ignores the failure of this task. The rest of the integration will be executed Assuming this task succeeded. |
NONE |
Causes a permanent failure of the task. However, if the last task(s) of event was successfully completed despite the failure of this task, it has no impact on the integration. |
FATAL |
Causes a permanent failure of the event. It is different from NONE because this will mark the event as FAILED by shutting down the event execution. |
FIXED_INTERVAL |
The task will be retried from the failed task onwards after a fixed delay. A max-retry count is required to be specified with this strategy. A jitter is added to each exponential interval so that concurrently failing tasks of the same type do not end up retrying after the exact same exponential interval. maxRetries and intervalInSeconds must be specified. |
LINEAR_BACKOFF |
The task will be retried from the failed task onwards after a fixed delay that linearly increases with each retry attempt. A jitter is added to each exponential interval so that concurrently failing tasks of the same type do not end up retrying after the exact same exponential interval. A max-retry count is required to be specified with this strategy. maxRetries and intervalInSeconds must be specified. |
EXPONENTIAL_BACKOFF |
The task will be retried after an exponentially increasing period of time with each failure. A jitter is added to each exponential interval so that concurrently failing tasks of the same type do not end up retrying after the exact same exponential interval. A max-retry count is required to be specified with this strategy. maxRetries and intervalInSeconds must be specified. |
RESTART_INTEGRATION_WITH_BACKOFF |
The entire integration will be restarted with the initial parameters that were set when the event was fired. A max-retry count is required to be specified with this strategy. maxRetries and intervalInSeconds must be specified. |
NextTask
The task that is next in line to be executed, if the condition specified evaluated to true.
JSON representation |
---|
{ "taskConfigId": string, "taskId": string, "condition": string, "displayName": string, "description": string } |
Fields | |
---|---|
taskConfigId |
ID of the next task. |
taskId |
Task number of the next task. |
condition |
Standard filter expression for this task to become an eligible next task. |
displayName |
User-provided label that is attached to this edge in the UI. |
description |
User-provided description intended to give additional business context about the task. |
NextTasksExecutionPolicy
Various policies for executing the next set of tasks.
Enums | |
---|---|
NEXT_TASKS_EXECUTION_POLICY_UNSPECIFIED |
Default. |
RUN_ALL_MATCH |
Execute all the tasks that satisfy their associated condition. |
RUN_FIRST_MATCH |
Execute the first task that satisfies the associated condition. |
TaskExecutionStrategy
Various policies to trigger the execution of this task.
Enums | |
---|---|
TASK_EXECUTION_STRATEGY_UNSPECIFIED |
Default. If the strategy is not set explicitly, it will default to WHEN_ALL_SUCCEED . |
WHEN_ALL_SUCCEED |
Wait until all of its previous tasks finished execution, then verify at least one of the edge conditions is met, and execute if possible. This should be considered as WHEN_ALL_TASKS_SUCCEED. |
WHEN_ANY_SUCCEED |
Start execution as long as any of its previous tasks finished execution and the corresponding edge condition is met (since we will execute if only that succeeding edge condition is met). |
WHEN_ALL_TASKS_AND_CONDITIONS_SUCCEED |
Wait until all of its previous tasks finished execution, then verify the all edge conditions are met and execute if possible. |
SuccessPolicy
Policy that dictates the behavior for the task after it completes successfully.
JSON representation |
---|
{
"finalState": enum ( |
Fields | |
---|---|
finalState |
State to which the execution snapshot status will be set if the task succeeds. |
FinalState
The state of execution.
Enums | |
---|---|
FINAL_STATE_UNSPECIFIED |
UNSPECIFIED. |
SUCCEEDED |
The default behavior, where successful tasks will be marked as SUCCEEDED. |
SUSPENDED |
Sets the state to SUSPENDED after executing. This is required for SuspensionTask; event execution will continue once the user calls ResolveSuspensions with the eventExecutionInfoId and the task number. |
JsonValidationOption
Options for how to validate json schemas.
Enums | |
---|---|
JSON_VALIDATION_OPTION_UNSPECIFIED |
As per the default behavior, no validation will be run. Will not override any option set in a Task. |
SKIP |
Do not run any validation against JSON schemas. |
PRE_EXECUTION |
Validate all potential input JSON parameters against schemas specified in IntegrationParameter. |
POST_EXECUTION |
Validate all potential output JSON parameters against schemas specified in IntegrationParameter. |
PRE_POST_EXECUTION |
Perform both PRE_EXECUTION and POST_EXECUTION validations. |
TriggerConfig
Configuration detail of a trigger.
JSON representation |
---|
{ "label": string, "startTasks": [ { object ( |
Fields | |
---|---|
label |
Optional. The user created label for a particular trigger. |
startTasks[] |
Optional. Set of tasks numbers from where the integration execution is started by this trigger. If this is empty, then integration is executed with default start tasks. In the list of start tasks, none of two tasks can have direct ancestor-descendant relationships (i.e. in a same integration execution graph). |
properties |
Optional. Configurable properties of the trigger, not to be confused with integration parameters. E.g. "name" is a property for API triggers and "subscription" is a property for Pub/sub triggers. An object containing a list of |
triggerType |
Optional. Type of trigger |
triggerNumber |
Required. A number to uniquely identify each trigger config within the integration on UI. |
alertConfig[] |
Optional. An alert threshold configuration for the [trigger + client + integration] tuple. If these values are not specified in the trigger config, default values will be populated by the system. Note that there must be exactly one alert threshold configured per [client + trigger + integration] when published. |
nextTasksExecutionPolicy |
Optional. Dictates how next tasks will be executed. |
triggerId |
Optional. The backend trigger ID. |
description |
Optional. User-provided description intended to give additional business context about the task. |
cloudSchedulerConfig |
Optional. Cloud Scheduler Trigger related metadata |
TriggerType
Types of triggers.
Enums | |
---|---|
TRIGGER_TYPE_UNSPECIFIED |
Unknown. |
CRON |
Trigger by scheduled time. |
API |
Trigger by API call. |
SFDC_CHANNEL |
Trigger by Salesforce Channel. |
CLOUD_PUBSUB_EXTERNAL |
Trigger by Pub/Sub external. |
SFDC_CDC_CHANNEL |
SFDC Channel Trigger for CDC. |
CLOUD_SCHEDULER |
Trigger by Cloud Scheduler job. |
IntegrationAlertConfig
Message to be used to configure custom alerting in the {@code EventConfig} protos for an event.
JSON representation |
---|
{ "displayName": string, "metricType": enum ( |
Fields | |
---|---|
displayName |
Name of the alert. This will be displayed in the alert subject. If set, this name should be unique within the scope of the integration. |
metricType |
The type of metric. |
thresholdType |
The threshold type, whether lower(expected_min) or upper(expected_max), for which this alert is being configured. If value falls below expected_min or exceeds expected_max, an alert will be fired. |
thresholdValue |
The metric value, above or below which the alert should be triggered. |
durationThreshold |
Should be specified only for *AVERAGE_DURATION and *PERCENTILE_DURATION metrics. This member should be used to specify what duration value the metrics should exceed for the alert to trigger. A duration in seconds with up to nine fractional digits, ending with ' |
aggregationPeriod |
The period over which the metric value should be aggregated and evaluated. Format is |
alertThreshold |
For how many contiguous aggregation periods should the expected min or max be violated for the alert to be fired. |
disableAlert |
Set to false by default. When set to true, the metrics are not aggregated or pushed to Monarch for this integration alert. |
onlyFinalAttempt |
For either events or tasks, depending on the type of alert, count only final attempts, not retries. |
MetricType
The metric for which this alert is being configured. The *RATE metrics refer to the number of instances of a task/event/error/warning per aggregation period. The *AVERAGE_DURATION metrics refer to the average duration of all the concerned tasks/events over the aggregationPeriod. The *PERCENTILE_DURATION metrics refer to the duration value of the specified percentile of the tasks or events.
Enums | |
---|---|
METRIC_TYPE_UNSPECIFIED |
The default value. Metric type should always be set to one of the other non-default values, otherwise it will result in an INVALID_ARGUMENT error. |
EVENT_ERROR_RATE |
Specifies alerting on the rate of errors for the enclosing integration. |
EVENT_WARNING_RATE |
Specifies alerting on the rate of warnings for the enclosing integration. Warnings use the same enum values as errors. |
TASK_ERROR_RATE |
Specifies alerting on the rate of errors for any task in the enclosing integration. |
TASK_WARNING_RATE |
Specifies alerting on the rate of warnings for any task in the enclosing integration. |
TASK_RATE |
Specifies alerting on the rate of executions over all tasks in the enclosing integration. |
EVENT_RATE |
Specifies alerting on the number of events executed in the given aggregationPeriod. |
EVENT_AVERAGE_DURATION |
Specifies alerting on the average duration of executions for this integration. |
EVENT_PERCENTILE_DURATION |
Specifies alerting on the duration value of a particular percentile of integration executions. E.g. If 10% or more of the integration executions have durations above 5 seconds, alert. |
TASK_AVERAGE_DURATION |
Specifies alerting on the average duration of any task in the enclosing integration, |
TASK_PERCENTILE_DURATION |
Specifies alerting on the duration value of a particular percentile of any task executions within the enclosing integration. E.g. If 10% or more of the task executions in the integration have durations above 5 seconds, alert. |
ThresholdType
The threshold type specifies whether the thresholdValue is to be interpreted as a lower bound or an upper bound, i.e. whether the alert fires when the value falls below expected_min or goes above expected_max.
Enums | |
---|---|
THRESHOLD_TYPE_UNSPECIFIED |
Default. |
EXPECTED_MIN |
Note that this field will only trigger alerts if the integration specifying it runs at least once in 24 hours (which is our in-memory retention period for monarch streams). Also note that aggregationPeriod for this alert configuration must be less than 24 hours. Min value threshold. |
EXPECTED_MAX |
Max value threshold. |
ThresholdValue
The threshold value of the metric, above or below which the alert should be triggered. See EventAlertConfig or TaskAlertConfig for the different alert metric types in each case.
For the *RATE metrics, one or both of these fields may be set. Zero is the default value and can be left at that.
For *PERCENTILE_DURATION metrics, one or both of these fields may be set, and also, the duration threshold value should be specified in the threshold_duration_ms member below.
For *AVERAGE_DURATION metrics, these fields should not be set at all. A different member, threshold_duration_ms, must be set in the EventAlertConfig or the TaskAlertConfig.
JSON representation |
---|
{ "absolute": string, "percentage": integer } |
Fields | |
---|---|
absolute |
Absolute value threshold. |
percentage |
Percentage threshold. |
CloudSchedulerConfig
Cloud Scheduler Trigger configuration
JSON representation |
---|
{ "serviceAccountEmail": string, "cronTab": string, "location": string, "errorMessage": string } |
Fields | |
---|---|
serviceAccountEmail |
Required. Service account used by Cloud Scheduler to trigger the integration at scheduled time |
cronTab |
Required. The cron tab of cloud scheduler trigger. |
location |
Required. The location where associated cloud scheduler job will be created |
errorMessage |
Optional. When the job was deleted from Pantheon UI, errorMessage will be populated when Get/List integrations |
IntegrationParameter
Integration Parameter is defined in the integration config and are used to provide information about data types of the expected parameters and provide any default values if needed. They can also be used to add custom attributes. These are static in nature and should not be used for dynamic event definition.
JSON representation |
---|
{ "key": string, "dataType": enum ( |
Fields | |
---|---|
key |
Key is used to retrieve the corresponding parameter value. This should be unique for a given fired event. These parameters must be predefined in the integration definition. |
dataType |
Type of the parameter. |
defaultValue |
Default values for the defined keys. Each value can either be string, int, double or any proto message or a serialized object. |
searchable |
Searchable in the execution log or not. |
displayName |
The name (without prefix) to be displayed in the UI for this parameter. E.g. if the key is "foo.bar.myName", then the name would be "myName". |
inputOutputType |
Specifies the input/output type for the parameter. |
isTransient |
Whether this parameter is a transient parameter. |
producer |
The identifier of the node (TaskConfig/TriggerConfig) this parameter was produced by, if it is a transient param or a copy of an input param. |
jsonSchema |
This schema will be used to validate runtime JSON-typed values of this parameter. |
InOutType
Is the parameter input, output of a integration.
Enums | |
---|---|
IN_OUT_TYPE_UNSPECIFIED |
Default. |
IN |
Input parameters for the integration. EventBus validates that these parameters exist in the integrations before execution. |
OUT |
Output Parameters for the integration. EventBus will only return the integration parameters tagged with OUT in the response back. |
IN_OUT |
Input and Output Parameters. These can be used as both input and output. EventBus will validate for the existence of these parameters before execution and will also return this parameter back in the response. |
IntegrationState
Indicates the status of the integration.
Enums | |
---|---|
INTEGRATION_STATE_UNSPECIFIED |
Default. |
DRAFT |
Draft. |
ACTIVE |
Active. |
ARCHIVED |
Archived. |
SNAPSHOT |
Snapshot. |
DatabasePersistencePolicy
Flag to disable database persistence for execution data, including event execution info, execution export info, execution metadata index and execution param index.
Enums | |
---|---|
DATABASE_PERSISTENCE_POLICY_UNSPECIFIED |
Enables persistence for all execution data. |
DATABASE_PERSISTENCE_DISABLED |
Disables persistence for all execution data. |
Methods |
|
---|---|
|
Create a integration with a draft version in the specified project. |
|
Soft-deletes the integration. |
|
Downloads an integration. |
|
Get a integration in the specified project. |
|
Returns the list of all integration versions in the specified project. |
|
Update a integration with a draft version in the specified project. |
|
This RPC throws an exception if the integration is in ARCHIVED or ACTIVE state. |
|
Sets the status of the ACTIVE integration to SNAPSHOT with a new tag "PREVIOUSLY_PUBLISHED" after validating it. |
|
Uploads an integration. |