- Resource: Experiment
- State
- Definition
- VersionVariants
- Variant
- Result
- VersionMetrics
- Metric
- MetricType
- CountType
- ConfidenceInterval
- VariantsHistory
- Methods
Resource: Experiment
Represents an experiment in an environment.
JSON representation | |
---|---|
{ "name": string, "displayName": string, "description": string, "state": enum ( |
Fields | |
---|---|
name |
The name of the experiment. Format: projects/ |
displayName |
Required. The human-readable name of the experiment (unique in an environment). Limit of 64 characters. |
description |
The human-readable description of the experiment. |
state |
The current state of the experiment. Transition triggered by Expriments.StartExperiment: PENDING->RUNNING. Transition triggered by Expriments.CancelExperiment: PENDING->CANCELLED or RUNNING->CANCELLED. |
definition |
The definition of the experiment. |
result |
Inference result of the experiment. |
createTime |
Creation time of this experiment. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
startTime |
Start time of this experiment. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
endTime |
End time of this experiment. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
lastUpdateTime |
Last update time of this experiment. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
experimentLength |
Maximum number of days to run the experiment. A duration in seconds with up to nine fractional digits, terminated by ' |
variantsHistory[] |
The history of updates to the experiment variants. |
State
The state of the experiment.
Enums | |
---|---|
STATE_UNSPECIFIED |
State unspecified. |
DRAFT |
The experiment is created but not started yet. |
RUNNING |
The experiment is running. |
DONE |
The experiment is done. |
Definition
Definition of the experiment.
JSON representation | |
---|---|
{
"condition": string,
"versionVariants": {
object ( |
Fields | |
---|---|
condition |
The condition defines which subset of sessions are selected for this experiment. If not specified, all sessions are eligible. E.g. "queryInput.language_code=en" See the conditions reference. |
versionVariants |
The flow versions as the variants of this experiment. |
VersionVariants
A list of flow version variants.
JSON representation | |
---|---|
{
"variants": [
{
object ( |
Fields | |
---|---|
variants[] |
A list of flow version variants. |
Variant
A single flow version with specified traffic allocation.
JSON representation | |
---|---|
{ "version": string, "trafficAllocation": number, "isControlGroup": boolean } |
Fields | |
---|---|
version |
The name of the flow version. Format: |
trafficAllocation |
Percentage of the traffic which should be routed to this version of flow. Traffic allocation for a single flow must sum up to 1.0. |
isControlGroup |
Whether the variant is for the control group. |
Result
The inference result which includes an objective metric to optimize and the confidence interval.
JSON representation | |
---|---|
{
"versionMetrics": [
{
object ( |
Fields | |
---|---|
versionMetrics[] |
Version variants and metrics. |
lastUpdateTime |
The last time the experiment's stats data was updated. Will have default value if stats have never been computed for this experiment. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
VersionMetrics
Version variant and associated metrics.
JSON representation | |
---|---|
{
"version": string,
"metrics": [
{
object ( |
Fields | |
---|---|
version |
The name of the flow |
metrics[] |
The metrics and corresponding confidence intervals in the inference result. |
sessionCount |
Number of sessions that were allocated to this version. |
Metric
Metric and corresponding confidence intervals.
JSON representation | |
---|---|
{ "type": enum ( |
Fields | ||
---|---|---|
type |
Ratio-based metric type. Only one of type or countType is specified in each Metric. |
|
countType |
Count-based metric type. Only one of type or countType is specified in each Metric. |
|
confidenceInterval |
The probability that the treatment is better than all other treatments in the experiment |
|
Union field value . The actual value of the metric. value can be only one of the following: |
||
ratio |
Ratio value of a metric. |
|
count |
Count value of a metric. |
MetricType
Types of ratio-based metric for Dialogflow experiment.
Enums | |
---|---|
METRIC_UNSPECIFIED |
Metric unspecified. |
CONTAINED_SESSION_NO_CALLBACK_RATE |
Percentage of contained sessions without user calling back in 24 hours. |
LIVE_AGENT_HANDOFF_RATE |
Percentage of sessions that were handed to a human agent. |
CALLBACK_SESSION_RATE |
Percentage of sessions with the same user calling back. |
ABANDONED_SESSION_RATE |
Percentage of sessions where user hung up. |
SESSION_END_RATE |
Percentage of sessions reached Dialogflow 'END_PAGE' or 'END_SESSION'. |
CountType
Types of count-based metric for Dialogflow experiment.
Enums | |
---|---|
COUNT_TYPE_UNSPECIFIED |
Count type unspecified. |
TOTAL_NO_MATCH_COUNT |
Total number of occurrences of a 'NO_MATCH'. |
TOTAL_TURN_COUNT |
Total number of turn counts. |
AVERAGE_TURN_COUNT |
Average turn count in a session. |
ConfidenceInterval
A confidence interval is a range of possible values for the experiment objective you are trying to measure.
JSON representation | |
---|---|
{ "confidenceLevel": number, "ratio": number, "lowerBound": number, "upperBound": number } |
Fields | |
---|---|
confidenceLevel |
The confidence level used to construct the interval, i.e. there is X% chance that the true value is within this interval. |
ratio |
The percent change between an experiment metric's value and the value for its control. |
lowerBound |
Lower bound of the interval. |
upperBound |
Upper bound of the interval. |
VariantsHistory
The history of variants update.
JSON representation | |
---|---|
{
"updateTime": string,
"versionVariants": {
object ( |
Fields | |
---|---|
updateTime |
Update time of the variants. A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: |
versionVariants |
The flow versions as the variants. |
Methods |
|
---|---|
|
Creates an Experiment in the specified Environment . |
|
Deletes the specified Experiment . |
|
Retrieves the specified Experiment . |
|
Returns the list of all experiments in the specified Environment . |
|
Updates the specified Experiment . |
|
Starts the specified Experiment . |
|
Stops the specified Experiment . |