Vertex AI V1 API - Class Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation (v0.63.0)

Version latest

• 0.63.0 (latest)
• 0.62.0
• 0.61.0
• 0.60.0
• 0.59.0
• 0.58.0
• 0.57.0
• 0.56.0
• 0.55.0
• 0.54.0
• 0.53.0
• 0.52.0
• 0.51.0
• 0.50.0
• 0.49.0
• 0.48.0
• 0.47.0
• 0.46.0
• 0.45.0
• 0.44.0
• 0.43.0
• 0.42.0
• 0.41.0
• 0.40.0
• 0.39.0
• 0.38.0
• 0.37.0
• 0.36.0
• 0.35.0
• 0.34.0
• 0.33.0
• 0.32.0
• 0.31.0
• 0.30.0
• 0.29.0
• 0.28.0
• 0.27.0
• 0.26.0
• 0.25.0
• 0.24.0
• 0.23.0
• 0.22.0
• 0.21.0
• 0.20.0
• 0.19.0
• 0.18.0
• 0.17.0
• 0.16.0
• 0.15.0
• 0.14.0
• 0.13.0
• 0.12.0
• 0.11.0
• 0.10.0
• 0.9.1
• 0.8.0
• 0.7.0
• 0.6.0
• 0.5.0
• 0.4.0
• 0.3.0
• 0.2.0
• 0.1.0

Reference documentation and code samples for the Vertex AI V1 API class Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation.

N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.

Inherits

• Object

Extended By

• Google::Protobuf::MessageExts::ClassMethods

Includes

• Google::Protobuf::MessageExts

Methods

#ngram_size

def ngram_size() -> ::Integer

Returns

• (::Integer) — The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.

#ngram_size=

def ngram_size=(value) -> ::Integer

Parameter

• value (::Integer) — The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.

Returns

• (::Integer) — The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.