class
Llama::Sampler::TopNSigma
- Llama::Sampler::TopNSigma
- Llama::Sampler::Base
- Reference
- Object
Overview
Top-N Sigma sampler
The Top-N Sigma sampler selects tokens based on their distance from the mean in terms of standard deviations. This is based on the paper "Top-nσ: Not All Logits Are You Need" (https://arxiv.org/pdf/2411.07641).
Example:
sampler = Llama::Sampler::TopNSigma.new(2.0) # Keep tokens within 2 standard deviationsDefined in:
llama/sampler/top_n_sigma.crConstructors
-
.new(n : Float32)
Creates a new Top-N Sigma sampler
Instance methods inherited from class Llama::Sampler::Base
to_unsafe : Pointer(Llama::LibLlama::LlamaSampler)
to_unsafe
Constructor methods inherited from class Llama::Sampler::Base
new(handle : Pointer(LibLlama::LlamaSampler))
new
Constructor Detail
def self.new(n : Float32)
#
Creates a new Top-N Sigma sampler
Parameters:
- n: Number of standard deviations to keep
Raises:
- Llama::Error if the sampler cannot be created