class Cadmium::AggressiveTokenizer

Included Modules

Defined in:

cadmium/tokenizer/aggressive_tokenizer.cr

Constructors

Instance Method Summary

Instance methods inherited from module Cadmium::Normalizers::RemoveDiacritics

remove_diacritics(str : String) remove_diacritics

Instance methods inherited from class Cadmium::Tokenizer

tokenize(string : String) : Array(String) tokenize, trim(arr) trim

Constructor Detail

def self.new(*, lang = nil) #

[View source]

Instance Method Detail

def tokenize(string : String) : Array(String) #

ameba:disable Metrics/CyclomaticComplexity


[View source]