Google Cloud Ai Platform V1 Client - Class NgramSpeculation (1.15.0)

Version latest

• 1.15.0 (latest)
• 1.14.0
• 1.13.1
• 1.12.0
• 1.11.0
• 1.10.0
• 1.9.0
• 1.8.0
• 1.7.0
• 1.6.0
• 1.5.0
• 1.4.0
• 1.3.0
• 1.2.0
• 1.1.0
• 1.0.0
• 0.39.0
• 0.38.0
• 0.37.1
• 0.32.0
• 0.31.0
• 0.30.0
• 0.29.0
• 0.28.0
• 0.27.0
• 0.26.2
• 0.25.0
• 0.24.0
• 0.23.0
• 0.22.0
• 0.21.0
• 0.20.0
• 0.19.0
• 0.18.0
• 0.17.0
• 0.16.0
• 0.15.0
• 0.13.0
• 0.12.0
• 0.11.1
• 0.10.0

Reference documentation and code samples for the Google Cloud Ai Platform V1 Client class NgramSpeculation.

N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.

Generated from protobuf message google.cloud.aiplatform.v1.SpeculativeDecodingSpec.NgramSpeculation

Namespace

Google \ Cloud \ AIPlatform \ V1 \ SpeculativeDecodingSpec

Methods

__construct

Constructor.

Parameters
Name	Description
data	array Optional. Data for populating the Message object.
↳ ngram_size	int The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.

getNgramSize

The number of last N input tokens used as ngram to search/match against the previous prompt sequence.

This is equal to the N in N-Gram. The default value is 3 if not specified.

Returns
Type	Description
int

setNgramSize

The number of last N input tokens used as ngram to search/match against the previous prompt sequence.

This is equal to the N in N-Gram. The default value is 3 if not specified.

Parameter
Name	Description
var	int

Returns
Type	Description
$this