REST Resource: projects.locations.qaScorecards.revisions.qaQuestions

Resource: QaQuestion

A single question to be scored by the Insights QA feature.

JSON representation 
{
  "name": string,
  "abbreviation": string,
  "createTime": string,
  "updateTime": string,
  "questionBody": string,
  "answerInstructions": string,
  "answerChoices": [
    {
      object (AnswerChoice)
    }
  ],
  "tags": [
    string
  ],
  "order": integer,
  "metrics": {
    object (Metrics)
  },
  "tuningMetadata": {
    object (TuningMetadata)
  }
}
Fields
name

string

Identifier. The resource name of the question. Format: projects/{project}/locations/{location}/qaScorecards/{qaScorecard}/revisions/{revision}/qaQuestions/{qaQuestion}
abbreviation

string

Short, descriptive string, used in the UI where it's not practical to display the full question body. E.g., "Greeting".
createTime

string (Timestamp format)

Output only. The time at which this question was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. The most recent time at which the question was updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
questionBody

string

Question text. E.g., "Did the agent greet the customer?"
answerInstructions

string

Instructions describing how to determine the answer.
answerChoices[]

object (AnswerChoice)

A list of valid answers to the question, which the LLM must choose from.
tags[]

string

User-defined list of arbitrary tags for the question. Used for grouping/organization and for weighting the score of each question.
order

integer

Defines the order of the question within its parent scorecard revision.
metrics

object (Metrics)

Metrics of the underlying tuned LLM over a holdout/test set while fine tuning the underlying LLM for the given question. This field will only be populated if and only if the question is part of a scorecard revision that has been tuned.
tuningMetadata

object (TuningMetadata)

Metadata about the tuning operation for the question.This field will only be populated if and only if the question is part of a scorecard revision that has been tuned.

AnswerChoice

Message representing a possible answer to the question.

JSON representation 
{
  "key": string,

  // Union field value can be only one of the following:
  "strValue": string,
  "numValue": number,
  "boolValue": boolean,
  "naValue": boolean
  // End of list of possible types for union field value.
  "score": number
}
Fields
key

string

A short string used as an identifier.
Union field value. The answer value may be one of a few different types. value can be only one of the following:
strValue

string

String value.
numValue

number

Numerical value.
boolValue

boolean

Boolean value.
naValue

boolean

A value of "Not Applicable (N/A)". If provided, this field may only be set to true. If a question receives this answer, it will be excluded from any score calculations.
score

number

Numerical score of the answer, used for generating the overall score of a QaScorecardResult. If the answer uses naValue, this field is unused.

Metrics

A wrapper representing metrics calculated against a test-set on a LLM that was fine tuned for this question.

JSON representation 
{
  "accuracy": number
}
Fields
accuracy

number

Output only. Accuracy of the model. Measures the percentage of correct answers the model gave on the test set.

TuningMetadata

Metadata about the tuning operation for the question. Will only be set if a scorecard containing this question has been tuned.

JSON representation 
{
  "totalValidLabelCount": string,
  "datasetValidationWarnings": [
    enum (DatasetValidationWarning)
  ],
  "tuningError": string
}
Fields
totalValidLabelCount

string (int64 format)

Total number of valid labels provided for the question at the time of tuining.
datasetValidationWarnings[]

enum (DatasetValidationWarning)

A list of any applicable data validation warnings about the question's feedback labels.
tuningError

string

Error status of the tuning operation for the question. Will only be set if the tuning operation failed.

DatasetValidationWarning

Enum for the different types of issues a tuning dataset can have. These warnings are currentlyraised when trying to validate a dataset for tuning a scorecard.

Enums
DATASET_VALIDATION_WARNING_UNSPECIFIED Unspecified data validation warning.
TOO_MANY_INVALID_FEEDBACK_LABELS A non-trivial percentage of the feedback labels are invalid.
INSUFFICIENT_FEEDBACK_LABELS The quantity of valid feedback labels provided is less than the recommended minimum.
INSUFFICIENT_FEEDBACK_LABELS_PER_ANSWER One or more of the answers have less than the recommended minimum of feedback labels.
ALL_FEEDBACK_LABELS_HAVE_THE_SAME_ANSWER All the labels in the dataset come from a single answer choice.

Methods

create

 Create a QaQuestion.

delete

 Deletes a QaQuestion.

get

 Gets a QaQuestion.

list

 Lists QaQuestions.

patch

 Updates a QaQuestion.