Protocol Documentation

Signed seconds of the span of time. Must be from -315,576,000,000 to +315,576,000,000 inclusive. Note: these bounds are computed from: 60 sec/min * 60 min/hr * 24 hr/day * 365.25 days/year * 10000 years

Signed fractions of a second at nanosecond resolution of the span of time. Durations less than one second are represented with a 0 `seconds` field and a positive or negative `nanos` field. For durations of one second or more, a non-zero value for the `nanos` field must be of the same sign as the `seconds` field. Must be from -999,999,999 to +999,999,999 inclusive.

Event sent when Speech pipeline has ASR; on Speech recognitions. Has transcribed text and metadata.

Event sent on detection of voice activity. Such as start of speech and end of speech.

Event with information about the audio stream.

Event sent when Speech pipeline has language identification capabilites. The event is sent for each audio frame.

Event sent with various debugging capabilities.

Event sent when Speech pipeline has hotwording and a hotword is detected.

A type of audio event. For example, it can be used to indicate that a finite stream of audio has ben processed through `END_OF_AUDIO` event.

Comtains information about the detected hotword.

Hotword that was triggered. This can be used to differentiate between the trigger source when there are multiple hotwords supported.

Results are ordered from the highest likelihood to the lowest. Currently, only the highest likelihood contains a confidence value.

Language recognized by the LangId model.

Confidence that the recognized language is correct.

Recognition results from the Recognition engine. The results are ordered by likelihood where the 0'th result has the highest likelihood.

Indicates that this result is finalized. - A full transcript is the concatenation of all finalized results. - If this field is false, modified versions of the result can be emitted in the future.

Indicates why a result was finalized.

Transcript result from the recognition engine.

Confidence the recognizer has in the transcript. Currently, this is only provided for the transcript with the highest likelihood.

Words detected in this result.

The textual representation of the word.

Time offset relative to the beginning of the audio, and corresponding to the start of the spoken word.

Time offset relative to the beginning of the audio, and corresponding to the end of the spoken word.

Confidence the recognizer has in the word being correct.

The type of Voice Activity Event. For example if it was `START_OF_SPEECH` or `END_OF_SPEECH`.

Configure the Speech pipeline with Speech recognition capabilites.

Configure the audio input of the Speech pipeline.

Configure the Speech pipeline with Language Identification capabilites.

Configure the threads used by Speech. The availability of the functionality will vary between platforms.

`language_pack` is a path to a directory holding an inference graph and resources for ASR.

Configure the formatting of ASR outputs.

The original results from the recognizer will be dropped and replaced (not 1:1) with new results containing the formatted text.

For example - If 'full stop' is spoken it would be transcribed as the full stop symbol '.', and 'question mark' as the question mark symbol '?'. If the model determines it to be in a context where the user is intending for it to be the symbolic representation.

For example - If 'smiley face' is spoken it would be transcribed as the smiley face symbol ':)'.

If `mask_offensive_words` is set to true and the ASR language_pack contains the necessary resources then certain offensive words will be masked. The `mask_offensive_words` is enabled by default if the field is unset.

`sample_rate_hz` is the sampling frequency of the audio in hertz.

`channel_count` is the number of channels in the audio.

`speaker_channel_count` is the number of channels of loud speaker audio fed into libgspeech through loopback.

The sample type the Speech pipeline is to expect. Prefer INT16

Configure fixed audio gain at the start of Speech pipeline.

Enable gain configuration.

Apply a fixed gain on ASR input.

`language_pack` is a path to a directory holding an inference graph and resources for langId processing.

If the Speech pipeline has Speech recognition capabilites this configuration can be used to adapt the capabilites for certain contexts.

A collection of custom classes. Refer to the defined class in phrase hints by its unique `custom_class_id`.

A collection of phrase sets, provides "hints" to the speech recognizer to favor specific words and phrases in the results. Any phrase set can use any custom class.

A unique id to be referenced in phrases. Must match regex "[A-Za-z0-9_]+" (quotes are not part of the regex).

A collection of class items.

A list of words and phrases.

Configure the CPU affinity of the primary worker threads. For example affinity { enable: true core: 1 core: 2 } Will attempt to set the CPU affinity of the working threads to core 1 and 2. This is an experimental feature. For Linux-variants it relies on the `sched_setaffinity` syscall and that child threads inherits the affinity from their parent. note: this will also set the affinity of the thread that is used to configure libgspeech. If set to true will attempt to apply the CPU affinity.

The cores to schedule the workers on. Has to be non-empty.