Vertex AI V1 API - Class Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec (v0.63.0)

Version latest

• 0.63.0 (latest)
• 0.62.0
• 0.61.0
• 0.60.0
• 0.59.0
• 0.58.0
• 0.57.0
• 0.56.0
• 0.55.0
• 0.54.0
• 0.53.0
• 0.52.0
• 0.51.0
• 0.50.0
• 0.49.0
• 0.48.0
• 0.47.0
• 0.46.0
• 0.45.0
• 0.44.0
• 0.43.0
• 0.42.0
• 0.41.0
• 0.40.0
• 0.39.0
• 0.38.0
• 0.37.0
• 0.36.0
• 0.35.0
• 0.34.0
• 0.33.0
• 0.32.0
• 0.31.0
• 0.30.0
• 0.29.0
• 0.28.0
• 0.27.0
• 0.26.0
• 0.25.0
• 0.24.0
• 0.23.0
• 0.22.0
• 0.21.0
• 0.20.0
• 0.19.0
• 0.18.0
• 0.17.0
• 0.16.0
• 0.15.0
• 0.14.0
• 0.13.0
• 0.12.0
• 0.11.0
• 0.10.0
• 0.9.1
• 0.8.0
• 0.7.0
• 0.6.0
• 0.5.0
• 0.4.0
• 0.3.0
• 0.2.0
• 0.1.0

Reference documentation and code samples for the Vertex AI V1 API class Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec.

Configuration for Speculative Decoding.

Inherits

• Object

Extended By

• Google::Protobuf::MessageExts::ClassMethods

Includes

• Google::Protobuf::MessageExts

Methods

#draft_model_speculation

def draft_model_speculation() -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation

Returns

• (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation) — draft model speculation.

  Note: The following fields are mutually exclusive: draft_model_speculation, ngram_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

#draft_model_speculation=

def draft_model_speculation=(value) -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation

Parameter

• value (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation) — draft model speculation.

  Note: The following fields are mutually exclusive: draft_model_speculation, ngram_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns

• (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::DraftModelSpeculation) — draft model speculation.

  Note: The following fields are mutually exclusive: draft_model_speculation, ngram_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

#ngram_speculation

def ngram_speculation() -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation

Returns

• (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation) — N-Gram speculation.

  Note: The following fields are mutually exclusive: ngram_speculation, draft_model_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

#ngram_speculation=

def ngram_speculation=(value) -> ::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation

Parameter

• value (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation) — N-Gram speculation.

  Note: The following fields are mutually exclusive: ngram_speculation, draft_model_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

Returns

• (::Google::Cloud::AIPlatform::V1::SpeculativeDecodingSpec::NgramSpeculation) — N-Gram speculation.

  Note: The following fields are mutually exclusive: ngram_speculation, draft_model_speculation. If a field in that set is populated, all other fields in the set will automatically be cleared.

#speculative_token_count

def speculative_token_count() -> ::Integer

Returns

• (::Integer) — The number of speculative tokens to generate at each step.

#speculative_token_count=

def speculative_token_count=(value) -> ::Integer

Parameter

• value (::Integer) — The number of speculative tokens to generate at each step.

Returns

• (::Integer) — The number of speculative tokens to generate at each step.