REST Resource: projects.datasets

Resource: DataSet

A collection of data sources sent for processing.

JSON representation
{
  "name": string,
  "dataNames": [
    string
  ],
  "dataSources": [
    {
      object (DataSource)
    }
  ],
  "state": enum (State),
  "status": {
    object (Status)
  },
  "ttl": string,
  "streaming": boolean
}
Fields
name

string

The dataset name, which will be used for querying, status and unload requests. This must be unique for a client.

dataNames[]

string

Data names allowed in this DataSet (e.g. 'QueryMetadataProto').

dataSources[]

object (DataSource)

Input data.

state

enum (State)

Dataset state in the system.

status

object (Status)

Dataset processing status.

ttl

string (Duration format)

Oldest entry TTL for streaming updates. Streaming data with timestamps older than ttl from the current time will be ignored by the system.

A duration in seconds with up to nine fractional digits, terminated by 's'. Example: "3.5s".

streaming

boolean

Streaming datasets also accept updates from clients. They are combined with the static data sources.

DataSource

A data source has multiple entries (e.g. a RecordIO file with an entry per record, a Spanner table with an entry being a subset of a row etc). A data source file consists of multiple DataItems stored on Google Cloud Storage.

JSON representation
{
  "uri": string
}
Fields
uri

string

Data source URI (e.g. the Google Cloud Storage path for a time series shard). If it is a Google Cloud Storage URI it must be in the following form: gs://bucket_name/object_name. For more details on Google Cloud Storage URIs, please see: https://cloud.google.com/storage/docs/reference-uris.

State

DataSet state.

Enums
STATE_UNSPECIFIED Unspecified / undefined state.
UNKNOWN Dataset is unknown to the system; we have never seen this dataset before or we have seen this dataset but have fully GC-ed it.
PENDING Dataset processing is pending.
LOADING Dataset is loading.
LOADED Dataset is loaded and can be queried.
UNLOADING Dataset is unloading.
UNLOADED Dataset is unloaded and is removed from the system.
FAILED Dataset processing failed.

Status

The Status type defines a logical error model that is suitable for different programming environments, including REST APIs and RPC APIs. It is used by gRPC. Each Status message contains three pieces of data: error code, error message, and error details.

You can find out more about this error model and how to work with it in the API Design Guide.

JSON representation
{
  "code": integer,
  "message": string,
  "details": [
    {
      "@type": string,
      field1: ...,
      ...
    }
  ]
}
Fields
code

integer

The status code, which should be an enum value of google.rpc.Code.

message

string

A developer-facing error message, which should be in English. Any user-facing error message should be localized and sent in the google.rpc.Status.details field, or localized by the client.

details[]

object

A list of messages that carry the error details. There is a common set of message types for APIs to use.

An object containing fields of an arbitrary type. An additional field "@type" contains a URI identifying the type. Example: { "id": 1234, "@type": "types.example.com/standard/id" }.

Methods

appendEvents

Append events to a DataSet if it is created with {"streaming": true}.

create

Create a DataSet from data stored on Google Cloud Storage.

delete

Delete a DataSet from the system.

list

Lists DataSets under the project.

query

Execute a Timeseries Insights query over a loaded DataSet.