Evaluates a dataset based on a set of given metrics.
Endpoint
posthttps://aiplatform.googleapis.com/v1/{location}:evaluateDataset
Path parameters
location
string
Required. The resource name of the Location to evaluate the dataset. Format: projects/{project}/locations/{location}
Request body
The request body contains data with the following structure:
dataset
object (EvaluationDataset
)
Required. The dataset used for evaluation.
metrics[]
object (Metric
)
Required. The metrics used for evaluation.
outputConfig
object (OutputConfig
)
Required. Config for evaluation output.
autoraterConfig
object (AutoraterConfig
)
Optional. Autorater config used for evaluation.
Response body
If successful, the response body contains an instance of Operation
.
EvaluationDataset
The dataset used for evaluation.
source
Union type
source
can be only one of the following:gcsSource
object (GcsSource
)
Cloud storage source holds the dataset.
bigquerySource
object (BigQuerySource
)
BigQuery source holds the dataset.
JSON representation |
---|
{ // source "gcsSource": { object ( |
BigQuerySource
The BigQuery location for the input content.
inputUri
string
Required. BigQuery URI to a table, up to 2000 characters long. Accepted forms:
- BigQuery path. For example:
bq://projectId.bqDatasetId.bqTableId
.
JSON representation |
---|
{ "inputUri": string } |
Metric
The metric used for dataset level evaluation.
aggregationMetrics[]
enum (AggregationMetric
)
Optional. The aggregation metrics to use.
metric_spec
Union type
metric_spec
can be only one of the following:pointwiseMetricSpec
object (PointwiseMetricSpec
)
Spec for pointwise metric.
pairwiseMetricSpec
object (PairwiseMetricSpec
)
Spec for pairwise metric.
exactMatchSpec
object (ExactMatchSpec
)
Spec for exact match metric.
bleuSpec
object (BleuSpec
)
Spec for bleu metric.
rougeSpec
object (RougeSpec
)
Spec for rouge metric.
JSON representation |
---|
{ "aggregationMetrics": [ enum ( |
AggregationMetric
The aggregation metrics supported by EvaluationService.EvaluateDataset.
Enums | |
---|---|
AGGREGATION_METRIC_UNSPECIFIED |
Unspecified aggregation metric. |
AVERAGE |
Average aggregation metric. |
MODE |
Mode aggregation metric. |
STANDARD_DEVIATION |
Standard deviation aggregation metric. |
VARIANCE |
Variance aggregation metric. |
MINIMUM |
Minimum aggregation metric. |
MAXIMUM |
Maximum aggregation metric. |
MEDIAN |
Median aggregation metric. |
PERCENTILE_P90 |
90th percentile aggregation metric. |
PERCENTILE_P95 |
95th percentile aggregation metric. |
PERCENTILE_P99 |
99th percentile aggregation metric. |
OutputConfig
Config for evaluation output.
destination
Union type
destination
can be only one of the following:gcsDestination
object (GcsDestination
)
Cloud storage destination for evaluation output.
JSON representation |
---|
{
// destination
"gcsDestination": {
object ( |