class Llama::SamplerChain

Overview

SamplerChain: manages a chain of samplers, but is not itself a Sampler::Base. Ownership and lifecycle are managed internally.

Defined in:

llama/sampler_chain.cr

Constructors

Instance Method Summary

Constructor Detail

def self.new(no_perf : Bool = false) #

Creates a new SamplerChain with optional parameters

Parameters:

  • no_perf: Whether to disable performance counters (default: false)

Raises:

  • Llama::Error if the sampler chain cannot be created

[View source]

Instance Method Detail

def accept(token : Int32) #

Accepts a token, updating the internal state of the samplers

Parameters:

  • token: The token to accept

[View source]
def add(sampler : Sampler::Base) #

Adds a sampler to the chain

Parameters:

  • sampler: The sampler to add to the chain

[View source]
def finalize #

Frees the resources associated with this sampler chain


[View source]
def print_perf #

Print performance information for this sampler chain


[View source]
def remove(index : Int) #

Removes a sampler from the chain at the given index. Returns the original Sampler::Base instance with ownership restored.


[View source]
def reset_perf #

Reset performance counters for this sampler chain


[View source]
def sample(ctx : Context, idx : Int32 = -1) : Int32 #

Samples a token using the sampler chain

Parameters:

  • ctx: The context to sample from
  • idx: The index of the logits to sample from (-1 for the last token)

Returns:

  • The sampled token

[View source]
def to_unsafe : Pointer(Llama::LibLlama::LlamaSampler) #

For C API compatibility


[View source]