Reference documentation and code samples for the Google Cloud Ai Platform V1 Client class SpeculativeDecodingSpec.
Configuration for Speculative Decoding.
Generated from protobuf message google.cloud.aiplatform.v1.SpeculativeDecodingSpec
Namespace
Google \ Cloud \ AIPlatform \ V1Methods
__construct
Constructor.
| Parameters | |
|---|---|
| Name | Description |
data |
array
Optional. Data for populating the Message object. |
↳ draft_model_speculation |
SpeculativeDecodingSpec\DraftModelSpeculation
draft model speculation. |
↳ ngram_speculation |
SpeculativeDecodingSpec\NgramSpeculation
N-Gram speculation. |
↳ speculative_token_count |
int
The number of speculative tokens to generate at each step. |
getDraftModelSpeculation
draft model speculation.
| Returns | |
|---|---|
| Type | Description |
SpeculativeDecodingSpec\DraftModelSpeculation|null |
|
hasDraftModelSpeculation
setDraftModelSpeculation
draft model speculation.
| Parameter | |
|---|---|
| Name | Description |
var |
SpeculativeDecodingSpec\DraftModelSpeculation
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getNgramSpeculation
N-Gram speculation.
| Returns | |
|---|---|
| Type | Description |
SpeculativeDecodingSpec\NgramSpeculation|null |
|
hasNgramSpeculation
setNgramSpeculation
N-Gram speculation.
| Parameter | |
|---|---|
| Name | Description |
var |
SpeculativeDecodingSpec\NgramSpeculation
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getSpeculativeTokenCount
The number of speculative tokens to generate at each step.
| Returns | |
|---|---|
| Type | Description |
int |
|
setSpeculativeTokenCount
The number of speculative tokens to generate at each step.
| Parameter | |
|---|---|
| Name | Description |
var |
int
|
| Returns | |
|---|---|
| Type | Description |
$this |
|
getSpeculation
| Returns | |
|---|---|
| Type | Description |
string |
|