class Cadmium::RegexTokenizer

Direct Known Subclasses

Defined in:

cadmium/tokenizer/regex_tokenizer.cr

Constructors

.new(pattern : Regex, *, gaps = nil, discard_empty = nil)

Instance Method Summary

#tokenize(string : String) : Array(String)

Instance methods inherited from class `Cadmium::Tokenizer`

,

Constructor Detail

def self.new(pattern : Regex, *, gaps = nil, discard_empty = nil) #

[View source]

Instance Method Detail

def tokenize(string : String) : Array(String) #

[View source]