class
Llama::Sampler::Typical
- Llama::Sampler::Typical
- Llama::Sampler::Base
- Reference
- Object
Overview
Typical sampler
The Typical sampler selects tokens based on their "typicality" (entropy). It filters out tokens that are either too predictable or too surprising, leading to more natural and diverse text generation.
Based on the paper: https://arxiv.org/abs/2202.00666
Example:
sampler = Llama::Sampler::Typical.new(0.95, 1) # Keep tokens with typicality >= 0.95Defined in:
llama/sampler/typical.crConstructors
-
.new(p : Float32, min_keep : Int32 = 1)
Creates a new Typical sampler
Instance methods inherited from class Llama::Sampler::Base
to_unsafe : Pointer(Llama::LibLlama::LlamaSampler)
to_unsafe
Constructor methods inherited from class Llama::Sampler::Base
new(handle : Pointer(LibLlama::LlamaSampler))
new
Constructor Detail
def self.new(p : Float32, min_keep : Int32 = 1)
#
Creates a new Typical sampler
Parameters:
- p: The typicality threshold (0.0 to 1.0)
- min_keep: Minimum number of tokens to keep
Raises:
- Llama::Error if the sampler cannot be created