Unified types

Overview

Unified types are the data types that appear in the Avro or JSON events. They are a Datastream-specific, unified representation of a data type across multiple data sources, such as an Oracle database or MySQL database.

The unified types are the superset of all type representations across all supported source types, which represent the original source type in a generic but lossless way.

The following tables list:

• The unified types associated with Datastream
• The mappings between the data types for an Oracle database or a MySQL database, and the Datastream unified types

Datastream unified types

Type name	Info	Avro definition	JSON definition
BOOLEAN	boolean	boolean	boolean
BYTES	A sequence of unsigned bytes	bytes	string
DECIMAL (p,s)	An arbitrary-precision signed decimal number	A decimal logical type	number
DOUBLE	64-bit floating point numbers	double	number
FLOAT	32-bit floating point numbers	float	number
INTEGER	A 32-bit integer	int	number
LONG	A 64-bit integer	long	number
NUMBER	A numeric data type	A custom logical type { "type": "string", "logicalType": "number" }	string
RECORD	A fully structured data type (for example, a user-defined data type)	Record	object
STRING	An unlimited string length	string	string
TIME_INTERVAL	How many microseconds elapsed between two events	A custom logical type { "type": "long", "logicalType": "time-interval-micros" }	long
TIMESTAMP	How many milliseconds or microseconds elapsed since the epoch, regardless of timezone	A timestamp logical type	string [ISO-8601]
TIMESTAMP WITH TIME ZONE	How many milliseconds or microseconds elapsed since the epoch with a specific timezone offset	A custom type { "type": "record", "name": "timestampTz", "fields": [ {"name": "timestamp", "type": "long" "logicalType": "timestamp-micros"}, {"name": "offset", "type": "long" "logicalType": "time-millis"} ] }	string [ISO-8601]
UNION	A varying data type	Union	array
UNSUPPORTED	An unsupported data type	A custom logical type { "type": "null", "logicalType": "unsupported" }	null
VARCHAR	A string with a maximum length of n characters	A custom logical type { "type": "string", "logicalType": "varchar" "length": N }	string

Map Oracle data types to Datastream unified types

Oracle data type	Datastream unified type
ANYDATA	Unsupported
BFILE	STRING
BINARY DOUBLE	DOUBLE
BINARY FLOAT	FLOAT
BLOB	Unsupported
CHAR	VARCHAR
CLOB	Unsupported
DATE	TIMESTAMP
DOUBLE PRECISION	DOUBLE
FLOAT(p)	DOUBLE
INTERVAL DAY TO SECOND	Unsupported
INTERVAL YEAR TO MONTH	Unsupported
LONG/LONG RAW	Unsupported
NCHAR	STRING
NCLOB	Unsupported
NUMBER	NUMBER
NUMBER(p,s<=0)	If p<=18, then LONG. If p>18 or p=*, then NUMBER. If the scale value (s) is less than 0, then Datastream assigns s to be equal to 0.
NUMBER(p,s>0)	If p=*, then DECIMAL(38,s), else DECIMAL(p,s). If the precision value (p) is less than the scale value (s), then Datastream assigns p to be equal to s.
NVARCHAR2	STRING
RAW	BYTES
ROWID	STRING
SMALLINT	INTEGER
TIMESTAMP	TIMESTAMP
TIMESTAMP WITH TIME ZONE	TIMESTAMP WITH TIME ZONE
UDT (user-defined type)	Unsupported
UROWID	Unsupported
VARCHAR	VARCHAR
VARCHAR2	VARCHAR
XMLTYPE	Unsupported

Map MySQL data types to Datastream unified types

MySQL data type	Datastream unified type
BIGINT(size)	LONG
BINARY(size)	STRING (hex encoded)
BIT(size)	LONG
BLOB(size)	STRING (hex encoded)
BOOL	INTEGER
CHAR(size)	STRING
DATE	TIMESTAMP
DATETIME(fsp)	TIMESTAMP
DECIMAL(size, d)	DECIMAL(size, d)
DOUBLE(size, d)	DOUBLE
ENUM(val1, val2, val3, ...)	STRING
FLOAT(p)	FLOAT
FLOAT(size, d)	FLOAT
INTEGER(size)	INTEGER
LONGBLOB	STRING (hex encoded)
LONGTEXT	STRING (hex encoded)
MEDIUMBLOB	STRING (hex encoded)
MEDIUMINT(size)	INTEGER
MEDIUMTEXT	STRING
SET(val1, val2, val3, ...)	STRING
SMALLINT(size)	INTEGER
TEXT(size)	STRING
TIME(fsp)	TIME_INTERVAL
TIMESTAMP(fsp)	TIMESTAMP
TINYBLOB	STRING (hex encoded)
TINYINT(size)	INTEGER
TINYTEXT	STRING
VARBINARY(size)	STRING (hex encoded)
VARCHAR	STRING
YEAR	INTEGER