class Tokenizer::Abstract
- Tokenizer::Abstract
- Tokenizer
- Reference
- Object
Defined in:
tokenizer/abstract.crConstructors
Instance Method Summary
Instance methods inherited from class Tokenizer
buffer : IO::Memory
buffer,
clear
clear
Constructor methods inherited from class Tokenizer
new(token_size : Int32, indicator = nil)new(*delimiter)
new(&callback : IO::Memory -> Int32) new