Evaluates a dataset based on a set of given metrics.
Endpoint
posthttps://aiplatform.googleapis.com/v1/{location}:evaluateDataset
Path parameters
location
string
Required. The resource name of the Location to evaluate the dataset. Format: projects/{project}/locations/{location}
Request body
The request body contains data with the following structure:
Required. The dataset used for evaluation.
Required. The metrics used for evaluation.
Required. Config for evaluation output.
Optional. Autorater config used for evaluation.
Response body
If successful, the response body contains an instance of Operation
.
EvaluationDataset
The dataset used for evaluation.
source
Union type
source
can be only one of the following:Cloud storage source holds the dataset.
BigQuery source holds the dataset.
JSON representation |
---|
{ // source "gcsSource": { object ( |
BigQuerySource
The BigQuery location for the input content.
inputUri
string
Required. BigQuery URI to a table, up to 2000 characters long. Accepted forms:
- BigQuery path. For example:
bq://projectId.bqDatasetId.bqTableId
.
JSON representation |
---|
{ "inputUri": string } |
Metric
The metric used for dataset level evaluation.
Optional. The aggregation metrics to use.
metric_spec
Union type
metric_spec
can be only one of the following:Spec for pointwise metric.
Spec for pairwise metric.
Spec for exact match metric.
Spec for bleu metric.
Spec for rouge metric.
JSON representation |
---|
{ "aggregationMetrics": [ enum ( |
AggregationMetric
The aggregation metrics supported by EvaluationService.EvaluateDataset.
Enums | |
---|---|
AGGREGATION_METRIC_UNSPECIFIED |
Unspecified aggregation metric. |
AVERAGE |
Average aggregation metric. |
MODE |
Mode aggregation metric. |
STANDARD_DEVIATION |
Standard deviation aggregation metric. |
VARIANCE |
Variance aggregation metric. |
MINIMUM |
Minimum aggregation metric. |
MAXIMUM |
Maximum aggregation metric. |
MEDIAN |
Median aggregation metric. |
PERCENTILE_P90 |
90th percentile aggregation metric. |
PERCENTILE_P95 |
95th percentile aggregation metric. |
PERCENTILE_P99 |
99th percentile aggregation metric. |
OutputConfig
Config for evaluation output.
destination
Union type
destination
can be only one of the following:Cloud storage destination for evaluation output.
JSON representation |
---|
{
// destination
"gcsDestination": {
object ( |