Class NgramSpeculation (1.90.0)

Version latest

• 1.90.0 (latest)
• 1.89.0
• 1.88.0
• 1.87.0
• 1.86.0
• 1.85.0
• 1.84.0
• 1.83.0
• 1.82.0
• 1.81.0
• 1.80.0
• 1.79.0
• 1.78.0
• 1.77.0
• 1.76.0
• 1.75.0
• 1.74.0
• 1.73.0
• 1.72.0
• 1.71.1
• 1.70.0
• 1.69.0
• 1.68.0
• 1.67.1
• 1.66.0
• 1.65.0
• 1.63.0
• 1.62.0
• 1.60.0
• 1.59.0
• 1.58.0
• 1.57.0
• 1.56.0
• 1.55.0
• 1.54.1
• 1.53.0
• 1.52.0
• 1.51.0
• 1.50.0
• 1.49.0
• 1.48.0
• 1.47.0
• 1.46.0
• 1.45.0
• 1.44.0
• 1.43.0
• 1.39.0
• 1.38.1
• 1.37.0
• 1.36.4
• 1.35.0
• 1.34.0
• 1.33.1
• 1.32.0
• 1.31.1
• 1.30.1
• 1.29.0
• 1.28.1
• 1.27.1
• 1.26.1
• 1.25.0
• 1.24.1
• 1.23.0
• 1.22.1
• 1.21.0
• 1.20.0
• 1.19.1
• 1.18.3
• 1.17.1
• 1.16.1
• 1.15.1
• 1.14.0
• 1.13.1
• 1.12.1
• 1.11.0
• 1.10.0
• 1.9.0
• 1.8.1
• 1.7.1
• 1.6.2
• 1.5.0
• 1.4.3
• 1.3.0
• 1.2.0
• 1.1.1
• 1.0.1
• 0.9.0
• 0.8.0
• 0.7.1
• 0.6.0
• 0.5.1
• 0.4.0
• 0.3.1

NgramSpeculation(mapping=None, *, ignore_unknown_fields=False, **kwargs)

N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.

Attribute
Name	Description
ngram_size	int The number of last N input tokens used as ngram to search/match against the previous prompt sequence. This is equal to the N in N-Gram. The default value is 3 if not specified.

Methods

NgramSpeculation

NgramSpeculation(mapping=None, *, ignore_unknown_fields=False, **kwargs)

N-Gram speculation works by trying to find matching tokens in the previous prompt sequence and use those as speculation for generating new tokens.