class Llama::Sampler::GrammarLazyPatterns

Overview

Grammar Lazy Patterns sampler

The Grammar Lazy Patterns sampler is an extension of the Grammar sampler that only applies grammar constraints when triggered by specific patterns or tokens. This is useful for mixed-format generation where grammar constraints should only apply to certain parts of the output.

Example:

# Define a JSON grammar that only activates when the text contains "JSON:"
grammar = %q{
  root ::= object
  object ::= "{" ws (string ":" ws value ("," ws string ":" ws value)*)? "}" ws
  array ::= "[" ws (value ("," ws value)*)? "]" ws
  value ::= object | array | string | number | "true" | "false" | "null"
  string ::= "\"" ([^"\\] | "\\" .)* "\""
  number ::= "-"? ("0" | [1-9] [0-9]*) ("." [0-9]+)? ([eE] [-+]? [0-9]+)?
  ws ::= [ \t\n]*
}

trigger_patterns = ["JSON:"]
sampler = Llama::Sampler::GrammarLazyPatterns.new(
  model.vocab, grammar, "root", trigger_patterns
)

Defined in:

llama/sampler/grammar_lazy_patterns.cr

Constructors

Instance Method Summary

Instance methods inherited from class Llama::Sampler::Base

to_unsafe : Pointer(Llama::LibLlama::LlamaSampler) to_unsafe

Constructor methods inherited from class Llama::Sampler::Base

new(handle : Pointer(LibLlama::LlamaSampler)) new

Constructor Detail

def self.new(vocab : Vocab, grammar_str : String, grammar_root : String, trigger_patterns : Array(String) = [] of String, trigger_tokens : Array(Int32) = [] of Int32) #

Creates a new Grammar Lazy Patterns sampler

Parameters:

  • vocab: The vocabulary to use
  • grammar_str: The grammar definition string in GBNF format
  • grammar_root: The root symbol of the grammar
  • trigger_patterns: Array of string patterns that will trigger the grammar
  • trigger_tokens: Array of token IDs that will trigger the grammar

Raises:

  • Llama::Error if the sampler cannot be created

[View source]

Instance Method Detail

def finalize #

Overrides the parent class's finalize method to ensure proper cleanup


[View source]