REST Resource: projects.locations.agents.environments.experiments

Resource: Experiment

Represents an experiment in an environment.

JSON representation 
{
  "name": string,
  "displayName": string,
  "description": string,
  "state": enum (State),
  "definition": {
    object (Definition)
  },
  "rolloutConfig": {
    object (RolloutConfig)
  },
  "rolloutState": {
    object (RolloutState)
  },
  "rolloutFailureReason": string,
  "result": {
    object (Result)
  },
  "createTime": string,
  "startTime": string,
  "endTime": string,
  "lastUpdateTime": string,
  "experimentLength": string,
  "variantsHistory": [
    {
      object (VariantsHistory)
    }
  ]
}
Fields
name

string

The name of the experiment. Format: projects//locations//agents//environments//experiments/..
displayName

string

Required. The human-readable name of the experiment (unique in an environment). Limit of 64 characters.
description

string

The human-readable description of the experiment.
state

enum (State)

The current state of the experiment. Transition triggered by Experiments.StartExperiment: DRAFT->RUNNING. Transition triggered by Experiments.CancelExperiment: DRAFT->DONE or RUNNING->DONE.
definition

object (Definition)

The definition of the experiment.
rolloutConfig

object (RolloutConfig)

The configuration for auto rollout. If set, there should be exactly two variants in the experiment (control variant being the default version of the flow), the traffic allocation for the non-control variant will gradually increase to 100% when conditions are met, and eventually replace the control variant to become the default version of the flow.
rolloutState

object (RolloutState)

State of the auto rollout process.
rolloutFailureReason

string

The reason why rollout has failed. Should only be set when state is ROLLOUT_FAILED.
result

object (Result)

Inference result of the experiment.
createTime

string (Timestamp format)

Creation time of this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
startTime

string (Timestamp format)

Start time of this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
endTime

string (Timestamp format)

End time of this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
lastUpdateTime

string (Timestamp format)

Last update time of this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
experimentLength

string (Duration format)

Maximum number of days to run the experiment. If auto-rollout is not enabled, default value and maximum will be 30 days. If auto-rollout is enabled, default value and maximum will be 6 days.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".
variantsHistory[]

object (VariantsHistory)

The history of updates to the experiment variants.

State

The state of the experiment.

Enums
STATE_UNSPECIFIED State unspecified.
DRAFT The experiment is created but not started yet.
RUNNING The experiment is running.
DONE The experiment is done.
ROLLOUT_FAILED The experiment with auto-rollout enabled has failed.

Definition

Definition of the experiment.

JSON representation 
{
  "condition": string,

  // Union field variants can be only one of the following:
  "versionVariants": {
    object (VersionVariants)
  }
  // End of list of possible types for union field variants.
}
Fields
condition

string

The condition defines which subset of sessions are selected for this experiment. If not specified, all sessions are eligible. E.g. "queryInput.language_code=en" See the conditions reference.
Union field variants. The variants of the experiment. We currently only support single variant experiment. variants can be only one of the following:
versionVariants

object (VersionVariants)

The flow versions as the variants of this experiment.

VersionVariants

A list of flow version variants.

JSON representation 
{
  "variants": [
    {
      object (Variant)
    }
  ]
}
Fields
variants[]

object (Variant)

A list of flow version variants.

Variant

A single flow version with specified traffic allocation.

JSON representation 
{
  "version": string,
  "trafficAllocation": number,
  "isControlGroup": boolean
}
Fields
version

string

The name of the flow version. Format: projects/<Project ID>/locations/<Location ID>/agents/<Agent ID>/flows/<Flow ID>/versions/<Version ID>.
trafficAllocation

number

Percentage of the traffic which should be routed to this version of flow. Traffic allocation for a single flow must sum up to 1.0.
isControlGroup

boolean

Whether the variant is for the control group.

RolloutConfig

The configuration for auto rollout.

JSON representation 
{
  "rolloutSteps": [
    {
      object (RolloutStep)
    }
  ],
  "rolloutCondition": string,
  "failureCondition": string
}
Fields
rolloutSteps[]

object (RolloutStep)

Steps to roll out a flow version. Steps should be sorted by percentage in ascending order.
rolloutCondition

string

The conditions that are used to evaluate the success of a rollout step. If not specified, all rollout steps will proceed to the next one unless failure conditions are met. E.g. "containment_rate > 60% AND callback_rate < 20%". See the conditions reference.
failureCondition

string

The conditions that are used to evaluate the failure of a rollout step. If not specified, no rollout steps will fail. E.g. "containment_rate < 10% OR average_turn_count < 3". See the conditions reference.

RolloutStep

A single rollout step with specified traffic allocation.

JSON representation 
{
  "displayName": string,
  "trafficPercent": integer,
  "minDuration": string
}
Fields
displayName

string

The name of the rollout step;
trafficPercent

integer

The percentage of traffic allocated to the flow version of this rollout step. (0%, 100%].
minDuration

string (Duration format)

The minimum time that this step should last. Should be longer than 1 hour. If not set, the default minimum duration for each step will be 1 hour.

A duration in seconds with up to nine fractional digits, ending with 's'. Example: "3.5s".

RolloutState

State of the auto-rollout process.

JSON representation 
{
  "step": string,
  "stepIndex": integer,
  "startTime": string
}
Fields
step

string

Display name of the current auto rollout step.
stepIndex

integer

Index of the current step in the auto rollout steps list.
startTime

string (Timestamp format)

Start time of the current step.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

Result

The inference result which includes an objective metric to optimize and the confidence interval.

JSON representation 
{
  "versionMetrics": [
    {
      object (VersionMetrics)
    }
  ],
  "lastUpdateTime": string
}
Fields
versionMetrics[]

object (VersionMetrics)

Version variants and metrics.
lastUpdateTime

string (Timestamp format)

The last time the experiment's stats data was updated. Will have default value if stats have never been computed for this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

VersionMetrics

Version variant and associated metrics.

JSON representation 
{
  "version": string,
  "metrics": [
    {
      object (Metric)
    }
  ],
  "sessionCount": integer
}
Fields
version

string

The name of the flow Version. Format: projects/<Project ID>/locations/<Location ID>/agents/<Agent ID>/flows/<Flow ID>/versions/<Version ID>.
metrics[]

object (Metric)

The metrics and corresponding confidence intervals in the inference result.
sessionCount

integer

Number of sessions that were allocated to this version.

Metric

Metric and corresponding confidence intervals.

JSON representation 
{
  "type": enum (MetricType),
  "countType": enum (CountType),
  "confidenceInterval": {
    object (ConfidenceInterval)
  },

  // Union field value can be only one of the following:
  "ratio": number,
  "count": number
  // End of list of possible types for union field value.
}
Fields
type

enum (MetricType)

Ratio-based metric type. Only one of type or countType is specified in each Metric.
countType

enum (CountType)

Count-based metric type. Only one of type or countType is specified in each Metric.
confidenceInterval

object (ConfidenceInterval)

The probability that the treatment is better than all other treatments in the experiment
Union field value. The actual value of the metric. value can be only one of the following:
ratio

number

Ratio value of a metric.
count

number

Count value of a metric.

MetricType

Types of ratio-based metric for Dialogflow experiment.

Enums
METRIC_UNSPECIFIED Metric unspecified.
CONTAINED_SESSION_NO_CALLBACK_RATE Percentage of contained sessions without user calling back in 24 hours.
LIVE_AGENT_HANDOFF_RATE Percentage of sessions that were handed to a human agent.
CALLBACK_SESSION_RATE Percentage of sessions with the same user calling back.
ABANDONED_SESSION_RATE Percentage of sessions where user hung up.
SESSION_END_RATE Percentage of sessions reached Dialogflow 'END_PAGE' or 'END_SESSION'.

CountType

types of count-based metric for Dialogflow experiment.

Enums
COUNT_TYPE_UNSPECIFIED Count type unspecified.
TOTAL_NO_MATCH_COUNT Total number of occurrences of a 'NO_MATCH'.
TOTAL_TURN_COUNT Total number of turn counts.
AVERAGE_TURN_COUNT Average turn count in a session.

ConfidenceInterval

A confidence interval is a range of possible values for the experiment objective you are trying to measure.

JSON representation 
{
  "confidenceLevel": number,
  "ratio": number,
  "lowerBound": number,
  "upperBound": number
}
Fields
confidenceLevel

number

The confidence level used to construct the interval, i.e. there is X% chance that the true value is within this interval.
ratio

number

The percent change between an experiment metric's value and the value for its control.
lowerBound

number

Lower bound of the interval.
upperBound

number

Upper bound of the interval.

VariantsHistory

The history of variants update.

JSON representation 
{
  "updateTime": string,

  // Union field variants can be only one of the following:
  "versionVariants": {
    object (VersionVariants)
  }
  // End of list of possible types for union field variants.
}
Fields
updateTime

string (Timestamp format)

Update time of the variants.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
Union field variants. The variants updated. We currently only support single variant experiment. variants can be only one of the following:
versionVariants

object (VersionVariants)

The flow versions as the variants.

Methods

create

 Creates an Experiment in the specified Environment.

delete

 Deletes the specified Experiment.

get

 Retrieves the specified Experiment.

list

 Returns the list of all experiments in the specified Environment.

patch

 Updates the specified Experiment.

start

 Starts the specified Experiment.

stop

 Stops the specified Experiment.