REST Resource: projects.locations.agents.environments.experiments

Resource: Experiment

Represents an experiment in an environment.

JSON representation 
{
  "name": string,
  "displayName": string,
  "description": string,
  "state": enum (State),
  "definition": {
    object (Definition)
  },
  "result": {
    object (Result)
  },
  "createTime": string,
  "startTime": string,
  "endTime": string,
  "lastUpdateTime": string,
  "experimentLength": string,
  "variantsHistory": [
    {
      object (VariantsHistory)
    }
  ]
}
Fields
name

string

The name of the experiment. Format: projects//locations//agents//environments//experiments/..
displayName

string

Required. The human-readable name of the experiment (unique in an environment). Limit of 64 characters.
description

string

The human-readable description of the experiment.
state

enum (State)

The current state of the experiment. Transition triggered by Expriments.StartExperiment: PENDING->RUNNING. Transition triggered by Expriments.CancelExperiment: PENDING->CANCELLED or RUNNING->CANCELLED.
definition

object (Definition)

The definition of the experiment.
result

object (Result)

Inference result of the experiment.
createTime

string (Timestamp format)

Creation time of this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
startTime

string (Timestamp format)

Start time of this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
endTime

string (Timestamp format)

End time of this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
lastUpdateTime

string (Timestamp format)

Last update time of this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
experimentLength

string (Duration format)

Maximum number of days to run the experiment/rollout. If auto-rollout is not enabled, default value and maximum will be 30 days. If auto-rollout is enabled, default value and maximum will be 6 days.

A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".
variantsHistory[]

object (VariantsHistory)

The history of updates to the experiment variants.

State

The state of the experiment.

Enums
STATE_UNSPECIFIED State unspecified.
DRAFT The experiment is created but not started yet.
RUNNING The experiment is running.
DONE The experiment is done.

Definition

Definition of the experiment.

JSON representation 
{
  "condition": string,
  "versionVariants": {
    object (VersionVariants)
  }
}
Fields
condition

string

The condition defines which subset of sessions are selected for this experiment. If not specified, all sessions are eligible. E.g. "queryInput.language_code=en" See the conditions reference.
versionVariants

object (VersionVariants)

The flow versions as the variants of this experiment.

VersionVariants

A list of flow version variants.

JSON representation 
{
  "variants": [
    {
      object (Variant)
    }
  ]
}
Fields
variants[]

object (Variant)

A list of flow version variants.

Variant

A single flow version with specified traffic allocation.

JSON representation 
{
  "version": string,
  "trafficAllocation": number,
  "isControlGroup": boolean
}
Fields
version

string

The name of the flow version. Format: projects/<Project ID>/locations/<Location ID>/agents/<Agent ID>/flows/<Flow ID>/versions/<Version ID>.
trafficAllocation

number

Percentage of the traffic which should be routed to this version of flow. Traffic allocation for a single flow must sum up to 1.0.
isControlGroup

boolean

Whether the variant is for the control group.

Result

The inference result which includes an objective metric to optimize and the confidence interval.

JSON representation 
{
  "versionMetrics": [
    {
      object (VersionMetrics)
    }
  ],
  "lastUpdateTime": string
}
Fields
versionMetrics[]

object (VersionMetrics)

Version variants and metrics.
lastUpdateTime

string (Timestamp format)

The last time the experiment's stats data was updated. Will have default value if stats have never been computed for this experiment.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".

VersionMetrics

Version variant and associated metrics.

JSON representation 
{
  "version": string,
  "metrics": [
    {
      object (Metric)
    }
  ],
  "sessionCount": integer
}
Fields
version

string

The name of the flow Version. Format: projects/<Project ID>/locations/<Location ID>/agents/<Agent ID>/flows/<Flow ID>/versions/<Version ID>.
metrics[]

object (Metric)

The metrics and corresponding confidence intervals in the inference result.
sessionCount

integer

Number of sessions that were allocated to this version.

Metric

Metric and corresponding confidence intervals.

JSON representation 
{
  "type": enum (MetricType),
  "countType": enum (CountType),
  "confidenceInterval": {
    object (ConfidenceInterval)
  },

  // Union field value can be only one of the following:
  "ratio": number,
  "count": number
  // End of list of possible types for union field value.
}
Fields
type

enum (MetricType)

Ratio-based metric type. Only one of type or countType is specified in each Metric.
countType

enum (CountType)

Count-based metric type. Only one of type or countType is specified in each Metric.
confidenceInterval

object (ConfidenceInterval)

The probability that the treatment is better than all other treatments in the experiment
Union field value. The actual value of the metric. value can be only one of the following:
ratio

number

Ratio value of a metric.
count

number

Count value of a metric.

MetricType

Types of ratio-based metric for Dialogflow experiment.

Enums
METRIC_UNSPECIFIED Metric unspecified.
CONTAINED_SESSION_NO_CALLBACK_RATE Percentage of contained sessions without user calling back in 24 hours.
LIVE_AGENT_HANDOFF_RATE Percentage of sessions that were handed to a human agent.
CALLBACK_SESSION_RATE Percentage of sessions with the same user calling back.
ABANDONED_SESSION_RATE Percentage of sessions where user hung up.
SESSION_END_RATE Percentage of sessions reached Dialogflow 'END_PAGE' or 'END_SESSION'.

CountType

Types of count-based metric for Dialogflow experiment.

Enums
COUNT_TYPE_UNSPECIFIED Count type unspecified.
TOTAL_NO_MATCH_COUNT Total number of occurrences of a 'NO_MATCH'.
TOTAL_TURN_COUNT Total number of turn counts.
AVERAGE_TURN_COUNT Average turn count in a session.

ConfidenceInterval

A confidence interval is a range of possible values for the experiment objective you are trying to measure.

JSON representation 
{
  "confidenceLevel": number,
  "ratio": number,
  "lowerBound": number,
  "upperBound": number
}
Fields
confidenceLevel

number

The confidence level used to construct the interval, i.e. there is X% chance that the true value is within this interval.
ratio

number

The percent change between an experiment metric's value and the value for its control.
lowerBound

number

Lower bound of the interval.
upperBound

number

Upper bound of the interval.

VariantsHistory

The history of variants update.

JSON representation 
{
  "updateTime": string,
  "versionVariants": {
    object (VersionVariants)
  }
}
Fields
updateTime

string (Timestamp format)

Update time of the variants.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
versionVariants

object (VersionVariants)

The flow versions as the variants.

Methods

create

 Creates an Experiment in the specified Environment.

delete

 Deletes the specified Experiment.

get

 Retrieves the specified Experiment.

list

 Returns the list of all experiments in the specified Environment.

patch

 Updates the specified Experiment.

start

 Starts the specified Experiment.

stop

 Stops the specified Experiment.