REST Resource: projects.locations.qaScorecards.revisions.qaQuestions

Resource: QaQuestion

A single question to be scored by the Insights QA feature.

JSON representation 
{
  "name": string,
  "abbreviation": string,
  "createTime": string,
  "updateTime": string,
  "questionBody": string,
  "answerInstructions": string,
  "answerChoices": [
    {
      object (AnswerChoice)
    }
  ],
  "tags": [
    string
  ],
  "order": integer,
  "metrics": {
    object (Metrics)
  },
  "tuningMetadata": {
    object (TuningMetadata)
  }
}
Fields
name

string

Identifier. The resource name of the question. Format: projects/{project}/locations/{location}/qaScorecards/{qaScorecard}/revisions/{revision}/qaQuestions/{qaQuestion}
abbreviation

string

Short, descriptive string, used in the UI where it's not practical to display the full question body. E.g., "Greeting".
createTime

string (Timestamp format)

Output only. The time at which this question was created.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
updateTime

string (Timestamp format)

Output only. The most recent time at which the question was updated.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
questionBody

string

Question text. E.g., "Did the agent greet the customer?"
answerInstructions

string

Instructions describing how to determine the answer.
answerChoices[]

object (AnswerChoice)

A list of valid answers to the question, which the LLM must choose from.
tags[]

string

User-defined list of arbitrary tags for the question. Used for grouping/organization and for weighting the score of each question.
order

integer

Defines the order of the question within its parent scorecard revision.
metrics

object (Metrics)

Metrics of the underlying tuned LLM over a holdout/test set while fine tuning the underlying LLM for the given question. This field will only be populated if and only if the question is part of a scorecard revision that has been tuned.
tuningMetadata

object (TuningMetadata)

Metadata about the tuning operation for the question.This field will only be populated if and only if the question is part of a scorecard revision that has been tuned.

AnswerChoice

Message representing a possible answer to the question.

JSON representation 
{
  "key": string,

  // Union field value can be only one of the following:
  "strValue": string,
  "numValue": number,
  "boolValue": boolean,
  "naValue": boolean
  // End of list of possible types for union field value.
  "score": number
}
Fields
key

string

A short string used as an identifier.
Union field value. The answer value may be one of a few different types. value can be only one of the following:
strValue

string

String value.
numValue

number

Numerical value.
boolValue

boolean

Boolean value.
naValue

boolean

A value of "Not Applicable (N/A)". If provided, this field may only be set to true. If a question receives this answer, it will be excluded from any score calculations.
score

number

Numerical score of the answer, used for generating the overall score of a QaScorecardResult. If the answer uses naValue, this field is unused.

Metrics

A wrapper representing metrics calculated against a test-set on a LLM that was fine tuned for this question.

JSON representation 
{
  "accuracy": number
}
Fields
accuracy

number

Output only. Accuracy of the model. Measures the percentage of correct answers the model gave on the test set.

TuningMetadata

Metadata about the tuning operation for the question. Will only be set if a scorecard containing this question has been tuned.

JSON representation 
{
  "totalValidLabelCount": string,
  "datasetValidationWarnings": [
    enum (DatasetValidationWarning)
  ],
  "tuningError": string
}
Fields
totalValidLabelCount

string (int64 format)

Total number of valid labels provided for the question at the time of tuining.
datasetValidationWarnings[]

enum (DatasetValidationWarning)

A list of any applicable data validation warnings about the question's feedback labels.
tuningError

string

Error status of the tuning operation for the question. Will only be set if the tuning operation failed.

Methods

create

 Create a QaQuestion.

delete

 Deletes a QaQuestion.

get

 Gets a QaQuestion.

list

 Lists QaQuestions.

patch

 Updates a QaQuestion.