Metadata bucket creation

This guide describes how to create a metadata bucket. You can create a metadata bucket using the Configuration API or Manufacturing Data Engine (MDE) UI.

Before you begin

Before you begin creating a bucket, read how to model metadata section in the MDE documentation.

Buckets naming

When you create a bucket, the name must be unique in a MDE deployment. Bucket names must not contain spaces or special characters such as &, @, or %.

Create a metadata bucket

You can create a metadata bucket using either the terminal or the console interface.

REST

Execute the following REST API request:

POST /metadata/v1/buckets

  {
    "name": "BUCKET_NAME",
    "type": "BUCKET_TYPE",
    "schema": "BUCKET_SCHEMA",
    "provider": "local",
    "attributes": {
      "instanceOverwriteMode": "INSTANCE_OVERWRITE_MODE"
    }
  }

Replace the following:

• BUCKET_NAME: the name of the bucket to be created.
• BUCKET_TYPE: the type of the bucket to be created. Must be one of TAG, RECORD, or LOOKUP. The natural_key of a metadata instance in a TAG bucket must be a tag name. The natural_key of a metadata instance in a RECORD bucket can be any key. The natural_key of a metadata instance in a LOOKUP bucket can be any key, but records can't reference instances in a lookup bucket, and the instances are not written to any data sink. Lookup buckets only serve as source of reference data that can be retrieved by the parser using the Whistle function lookupByKey and used to enrich records.
• BUCKET_SCHEMA: the JSON schema of the bucket. The schema constrains the instances that may be created in the bucket.
• INSTANCE_OVERWRITE_MODE: Either TRUE or FALSE. Determines whether new instances are created by merging the provided instance with the latest instance for a given natural key or if new instances are created as provided.

Console

To add new Metadata Buckets using the Console click the 'METADATA' section of the top menu of 'CLoud Management' section. The Metadata configuration page lists the available Buckets in the MDE instance:

For each Bucket the following information is provided:

• Name: the name of the bucket. Each name needs to be unique.
• Version: the Version of the Bucket. Only the latest available version is displayed by default. To display earlier versions un-check the 'Only Latest' check-box.
• Bucket Type: visualizes the type of Bucket. Three different Bucket Types are supported:
  1. Tag: buckets that qualify a given Tag. They are associated to a Type and inherited by all Tags of that given Type. They can be instantiated at Tag level as they intend to qualify the variable itself and not any of its specific records. An example of a Tag Bucket could be the 'Asset hierarchy' or the 'Units of Measure' of the Tag.
  2. Record: record Buckets qualify each of the Records of a given Tag. They are also associated to a Type and inherited by the Tag but are instantiated at the Record level. Each Record is expected to have a different value among a limited set of Bucket Instances. An example of Record Bucket could be the 'Shift' or the 'Operator Name'.
  3. Reference: they are Buckets that define a set of Key Value Pairs between a user-defined Key and an instance.

To create a new Bucket click 'Add New Metadata Bucket':

Fill out the form displayed from the side panel with the necessary Bucket parameters:

• Bucket Name: a unique, user defined name for the Bucket.
• Bucket Type: to define the Bucket class.
• Instance update mode: specifies the type of instance updates specified for the Metadata Bucket:
  1. Instance Override: the existing instance is replaced by the latest instance received from the edge, even if it is incomplete.
  2. Instance Merge: the existing instance is merged with the latest instance received from the edge. Only the attributed contained in the new edge instance are replaced.
• JSON Schema: the JSON Schema defining the structure of the Bucket. Need to be a valid Schema. All metadata instances are validated against the Bucket Schema.

The following is a sample JSON Schema of a default Metadata Bucket provided by the Default Configuration Package:

{
  "$id": "https://gmde.cloud/tag.schema.json",
  "$schema": "https://json-schema.org/draft/2019-09/schema",
  "title": "Tag metadata bucket",
  "type": "object",
  "properties": {
    "datatype": {
      "type": "string"
    },
    "description": {
      "type": "string"
    },
    "registerId": {
      "type": "string"
    },
    "edgeTagName": {
      "type": "string"
    },
    "tagName": {
      "description": "Tag name",
      "type": "string"
    },
    "deviceID": {
      "type": "string"
    },
    "deviceName": {
      "type": "string"
    },
    "deviceMetadata": {
      "type": "object"
    }
  },
  "required": ["tagName"],
  "additionalProperties": false
}