class Cadmium::Tokenizer::Word
Defined in:
cadmium/tokenizer/word.crConstant Summary
-
REGEX_PATTERN =
/[^A-Za-zА-Яа-я0-9_]+/
/[^A-Za-zА-Яа-я0-9_]+/
Cadmium::Tokenizer::Regex
Cadmium::Tokenizer::Regex
Cadmium::Tokenizer::Base
Cadmium::Tokenizer::Diacritics
Cadmium::Tokenizer::StopWords