Overview
Unified types are the data types that appear in the Avro or JSON events. They are a Datastream-specific, unified representation of a data type across multiple data sources, such as an Oracle database or MySQL database.
The unified types are the superset of all type representations across all supported source types, which represent the original source type in a generic but lossless way.
The following tables list:
- The unified types associated with Datastream
- The mappings between the data types for an Oracle database or a MySQL database, and the Datastream unified types
Datastream unified types
Type name | Info | Avro definition | JSON definition |
---|---|---|---|
BOOLEAN |
boolean | boolean | boolean |
BYTES |
A sequence of unsigned bytes | bytes | string |
DECIMAL (p,s) |
An arbitrary-precision signed decimal number | A decimal logical type | number |
DOUBLE |
64-bit floating point numbers | double | number |
FLOAT |
32-bit floating point numbers | float | number |
INTEGER |
A 32-bit integer | int | number |
LONG |
A 64-bit integer | long | number |
NUMBER |
A numeric data type | A custom logical type { "type": "string", "logicalType": "number" } |
string |
RECORD |
A fully structured data type (for example, a user-defined data type) | Record | object |
STRING |
An unlimited string length | string | string |
TIME_INTERVAL |
How many microseconds elapsed between two events | A custom logical type { "type": "long", "logicalType": "time-interval-micros" } |
long |
TIMESTAMP |
How many milliseconds or microseconds elapsed since the epoch, regardless of timezone | A timestamp logical type | string [ISO-8601] |
TIMESTAMP WITH TIME ZONE |
How many milliseconds or microseconds elapsed since the epoch with a specific timezone offset | A custom type { "type": "record", "name": "timestampTz", "fields": [ {"name": "timestamp", "type": "long" "logicalType": "timestamp-micros"}, {"name": "offset", "type": "long" "logicalType": "time-millis"} ] } |
string [ISO-8601] |
UNION |
A varying data type | Union | array |
UNSUPPORTED |
An unsupported data type | A custom logical type { "type": "null", "logicalType": "unsupported" } |
null |
VARCHAR |
A string with a maximum length of n characters | A custom logical type { "type": "string", "logicalType": "varchar" "length": N } |
string |
Map Oracle data types to Datastream unified types
Oracle data type | Datastream unified type |
---|---|
ANYDATA |
Unsupported |
BFILE |
STRING |
BINARY DOUBLE |
DOUBLE |
BINARY FLOAT |
FLOAT |
BLOB |
Unsupported |
CHAR |
VARCHAR |
CLOB |
Unsupported |
DATE |
TIMESTAMP |
DOUBLE PRECISION |
DOUBLE |
FLOAT(p) |
DOUBLE |
INTERVAL DAY TO SECOND |
Unsupported |
INTERVAL YEAR TO MONTH |
Unsupported |
LONG/LONG RAW |
Unsupported |
NCHAR |
STRING |
NCLOB |
Unsupported |
NUMBER |
NUMBER |
NUMBER(p,s<=0) |
If p<=18, then |
NUMBER(p,s>0) |
If p= |
NVARCHAR2 |
STRING |
RAW |
BYTES |
ROWID |
STRING |
SMALLINT |
INTEGER |
TIMESTAMP |
TIMESTAMP |
TIMESTAMP WITH TIME ZONE |
TIMESTAMP WITH TIME ZONE |
UDT (user-defined type) |
Unsupported |
UROWID |
Unsupported |
VARCHAR |
VARCHAR |
VARCHAR2 |
VARCHAR |
XMLTYPE |
Unsupported |
Map MySQL data types to Datastream unified types
MySQL data type | Datastream unified type |
---|---|
BIGINT(size) |
LONG |
BINARY(size) |
STRING (hex encoded) |
BIT(size) |
LONG |
BLOB(size) |
STRING (hex encoded) |
BOOL |
INTEGER |
CHAR(size) |
STRING |
DATE |
TIMESTAMP |
DATETIME(fsp) |
TIMESTAMP |
DECIMAL(size, d) |
DECIMAL(size, d) |
DOUBLE(size, d) |
DOUBLE |
ENUM(val1, val2, val3, ...) |
STRING |
FLOAT(p) |
FLOAT |
FLOAT(size, d) |
FLOAT |
INTEGER(size) |
INTEGER |
LONGBLOB |
STRING (hex encoded) |
LONGTEXT |
STRING (hex encoded) |
MEDIUMBLOB |
STRING (hex encoded) |
MEDIUMINT(size) |
INTEGER |
MEDIUMTEXT |
STRING |
SET(val1, val2, val3, ...) |
STRING |
SMALLINT(size) |
INTEGER |
TEXT(size) |
STRING |
TIME(fsp) |
TIME_INTERVAL |
TIMESTAMP(fsp) |
TIMESTAMP |
TINYBLOB |
STRING (hex encoded) |
TINYINT(size) |
INTEGER |
TINYTEXT |
STRING |
VARBINARY(size) |
STRING (hex encoded) |
VARCHAR |
STRING |
YEAR |
INTEGER |