class Cadmium::SentenceTokenizer

• Cadmium::SentenceTokenizer
• Cadmium::Tokenizer
• Reference
• Object

Defined in:

cadmium/tokenizer/sentence_tokenizer.cr

Constant Summary

REGEX_PATTERN = /([\"\'\‘\“\'\"\[\(\{\⟨][^\.\?\!]+[\.\?\!][\"\'\’\”\'\"\]\)\}\⟩]|[^\.\?\!]+[\.\?\!])\s?/

Instance Method Summary

• #tokenize(string : String) : Array(String)

Instance Method Detail