Method: projects.datasets.query

Execute a Timeseries Insights query over a loaded DataSet.

HTTP request

POST https://{endpoint}/v1/{name=projects/*/datasets/*}:query

Where {endpoint} is one of the supported service endpoints.

The URLs use gRPC Transcoding syntax.

Path parameters

Parameters
name

string

Required. Loaded DataSet to be queried in the format of "projects/{project}/datasets/{dataset}"

Request body

The request body contains data with the following structure:

JSON representation 
{
  "dimensionNames": [
    string
  ],
  "pinnedDimensions": [
    {
      object (EventDimension)
    }
  ],
  "testedInterval": {
    object (TimeInterval)
  },
  "forecastParams": {
    object (ForecastParams)
  },
  "returnNonAnomalies": boolean,
  "returnTimeseries": boolean
}
Fields
dimensionNames[]

string

Dimensions over which we want to aggregate the events. The names specified here come from the EventDimension.name field.
pinnedDimensions[]

object (EventDimension)

We will only consider slices for which ForecastSlice.dimensions contain all of the following pinned dimensions. A query with a pinned dimension { name: "d3" stringVal: "v3" } will only analyze events which contain the dimension { name: "d3" stringVal: "v3" }. We currently only support string value for pinned dimensions. The pinnedDimensions and dimensionNames fields can not share the same dimension names. For example:

{
  dimensionNames: ["d1", "d2"],
  pinnedDimensions: [
    { name: "d3" stringVal: "v3" },
    { name: "d4" stringVal: "v4" }
  ]
}
testedInterval

object (TimeInterval)

The time interval for which we want to detect if there are any slices in our dataset with abnormal behavior (e.g. the last hour or the last 10 minutes).
forecastParams

object (ForecastParams)

Params for time series insights forecast.
returnNonAnomalies

boolean

If we should return slices that we evaluated, but were not marked as anomalies. This is useful for debugging purposes or to simply see what was the evaluation status for slices that were not anomalies.
returnTimeseries

boolean

If given, we will populate the ForecastResult.history and ForecastResult.forecast fields.

Response body

If successful, the response body contains data with the following structure:

Response for a query executed by the system.

JSON representation 
{
  "name": string,
  "anomalyDetectionResult": {
    object (AnomalyDetectionResult)
  }
}
Fields
name

string

Loaded DataSet that was queried.
anomalyDetectionResult

object (AnomalyDetectionResult)

Filled if we ran anomaly detection as part of the query.

Authorization Scopes

Requires the following OAuth scope:

  • https://www.googleapis.com/auth/cloud-platform

For more information, see the Authentication Overview.

TimeInterval

A time interval in the [start, start+length) range.

JSON representation 
{
  "startTime": string,
  "length": string
}
Fields
startTime

string (Timestamp format)

Time interval start.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
length

string (Duration format)

Time interval length.

A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".

ForecastParams

Forecast setup.

JSON representation 
{
  "horizonTime": string,
  "holdout": number,
  "minDensity": number,
  "forecastHistory": string,
  "maxPositiveRelativeChange": number,
  "maxNegativeRelativeChange": number,
  "forecastExtraWeight": number,
  "forecastTimeseriesOffset": number,
  "aggregatedDimension": string,
  "seasonalityHint": enum (Period)
}
Fields
horizonTime

string (Timestamp format)

Forecast maximum time horizon.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
holdout

number

Percentage of data to use for accuracy evaluation (latest TimeseriesPoints from input). If unsure, set this in the [3, 10] range.

NOTE: Must be in [0, 100).
minDensity

number

Minimum density percentage of the input time series for it to be evaluated. If unsure, set this to 0.

NOTE: Must be in [0, 100].
forecastHistory

string (Duration format)

How long should we go in the past when fetching the timeline used for forecasting each slice. This must be a multiple of testedInterval.length.

A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".
maxPositiveRelativeChange

number

A slice will be marked as an anomaly if the following 2 conditions are true:

  1. testedIntervalActual > testedIntervalForecastUpperBound
  2. | testedIntervalActual - testedIntervalForecastUpperBound | > maxPositiveRelativeChange * ( testedIntervalForecastUpperBound + forecastExtraWeight )

NOTE: Must be non-negative.
maxNegativeRelativeChange

number

A slice will be marked as an anomaly if the following 2 conditions are true:

  1. testedIntervalActual < testedIntervalForecastLowerBound
  2. | testedIntervalActual - testedIntervalForecastLowerBound | > maxNegativeRelativeChange * ( testedIntervalForecastUpperBound + forecastExtraWeight )

Note testedIntervalForecastUpperBound is used on the right hand side in order to take into account the worst case scenario.

NOTE: Must be non-negative.
forecastExtraWeight

number

Extra weight for the actual and forecasted value when comparing them. Increase this to reduce the false-positive rate from slices with low counts.

Check maxPositiveRelativeChange and maxNegativeRelativeChange for how this value is used.

NOTE: Must be non-negative.
forecastTimeseriesOffset

number

We will offset all the values in the returned time series by this amount when doing the forecast. This should reduce the noise from low count time series (similar to forecastExtraWeight but not affecting the relative bounds) since they won't be affected as much by low count time series. This gets added to the time series values. It should also be used to shift all values positive. This is needed if a numerical dimension can have negative values and specified as aggregatedDimension.

NOTE: Must be non-negative.
aggregatedDimension

string

If non-empty, it denotes the name of a numerical dimension present in the events of a slice that specifies that will have its values summed up to compute the time series.

If empty, we will simply count the events to produce the time series.

NOTE: Conceptually, a time series is computed by going through the following steps:

  1. Accumulating the events that belong to a given slice.
  2. Grouping the events into time buckets with the duration testedInterval.length, based on their eventTime.
  3. If aggregatedDimension is empty, couting the events in each bucket.
  4. If aggregatedDimension specifies a dimension, summing up that value for all events in a time bucket.
seasonalityHint

enum (Period)

Specifying any known seasonality/periodicity in the time series for the slices we will analyze can improve the quality of the results. If unsure, simply leave it as empty or PERIOD_UNSPECIFIED, as it should still yield results.

If your time series has multiple seasonal patterns, then set it to the most granular one (e.g. if it has daily and weekly patterns, set this to daily).

Period

A time period of a fixed interval.

Enums
PERIOD_UNSPECIFIED Unknown or simply not given.
DAILY 24 hours
WEEKLY 7 days
MONTHLY 30 days
YEARLY 365 days

AnomalyDetectionResult

Result of running anomaly detection over the input time series.

JSON representation 
{
  "anomalies": [
    {
      object (ForecastSlice)
    }
  ],
  "nonAnomalies": [
    {
      object (ForecastSlice)
    }
  ]
}
Fields
anomalies[]

object (ForecastSlice)

Anomalies detected.
nonAnomalies[]

object (ForecastSlice)

Slices we evaluated, but which were not marked as anomalies. Only returned if QueryDataSetRequest.returnNonAnomalies is set to true.

ForecastSlice

Event forecast.

JSON representation 
{
  "dimensions": [
    {
      object (EventDimension)
    }
  ],
  "result": {
    object (ForecastResult)
  },
  "status": {
    object (Status)
  }
}
Fields
dimensions[]

object (EventDimension)

Event dimensions. The number of dimensions here is equal to the size of the dimensionNames field specified in the request.
result

object (ForecastResult)

Filled if we ran forecasting for this slice.
status

object (Status)

Slice processing status.

ForecastResult

Forecast result for a given slice.

JSON representation 
{
  "testedIntervalActual": number,
  "testedIntervalForecast": number,
  "testedIntervalForecastLowerBound": number,
  "testedIntervalForecastUpperBound": number,
  "holdoutErrors": {
    object (ForecastErrors)
  },
  "trainingErrors": {
    object (ForecastErrors)
  },
  "forecastStats": {
    object (ForecastStats)
  },
  "history": {
    object (Timeseries)
  },
  "forecast": {
    object (Timeseries)
  }
}
Fields
testedIntervalActual

number

The actual value of the tested interval (see testedInterval). This value is an estimate, so it should not be used as a source of truth.
testedIntervalForecast

number

The expected value of the tested interval, which is obtained by forecasting on the historical time series (present in the history field if requested by setting returnTimeseries to true).
testedIntervalForecastLowerBound

number

The lower bound of the expected value of the tested interval based on testedIntervalForecast and adjusting for any forecast errors (as specified in holdoutErrors).
testedIntervalForecastUpperBound

number

The upper bound of the expected value of the tested interval based on testedIntervalForecast and adjusting for any forecast errors (as specified in holdoutErrors).
holdoutErrors

object (ForecastErrors)

Forecast errors over the holdout interval of the time series.
trainingErrors

object (ForecastErrors)

Forecast errors over the training interval of the time series.
forecastStats

object (ForecastStats)

Forecast analysis stats.
history

object (Timeseries)

The actual values in the [ testedInterval.startTime - forecastHistory, testedInterval.startTime ] time range.
forecast

object (Timeseries)

The forecasted values in the [ testedInterval.startTime + testedInterval.length, forecastParams.horizonTime ] time range.

ForecastErrors

Forecast performance.

JSON representation 
{
  "mdape": number,
  "rmd": number
}
Fields
mdape

number

MDAPE observed over the forecast interval, the median version of MAPE
rmd

number

RMD observed over the forecast interval.

ForecastStats

Forecast analysis stats.

JSON representation 
{
  "density": string,
  "numAnomaliesInHoldout": integer,
  "numAnomalies": integer
}
Fields
density

string (int64 format)

Percentage in the [0, 100] interval with how many data points we have in the actual time chunk.
numAnomaliesInHoldout

integer

Number of points that would be marked as an anomaly over holdout.
numAnomalies

integer

How many points were anomalies.

Timeseries

A time series.

JSON representation 
{
  "point": [
    {
      object (TimeseriesPoint)
    }
  ]
}
Fields
point[]

object (TimeseriesPoint)

The points in this time series, ordered by their timestamp.

TimeseriesPoint

A point in a time series.

JSON representation 
{
  "time": string,
  "value": number
}
Fields
time

string (Timestamp format)

The timestamp of this point.

A timestamp in RFC3339 UTC "Zulu" format, with nanosecond resolution and up to nine fractional digits. Examples: "2014-10-02T15:01:23Z" and "2014-10-02T15:01:23.045123456Z".
value

number

The value for this point.