class CLTK::Lexer
- CLTK::Lexer
- Reference
- Object
Overview
The Lexer class may be sub-classed to produce new lexers. These lexers have a lot of features, and are described in the main documentation.
Direct Known Subclasses
Defined in:
cltk/lexer.crcltk/lexer/environment.cr
cltk/lexer/rule.cr
Class Method Summary
-
.lex(string, file_name = nil, env = @@env.new(@@start_state))
Lex string, using env as the environment.
-
.lex_file(file_name, env = @@env.new(@start_state))
A wrapper function that calls {Lexer.lex} on the contents of a file.
-
.match_first
Used to tell a lexer to use the first match found instead of the longest match found.
- .rule(pattern, state = :default, flags = [] of String)
-
.start(state)
Changes the starting state of the lexer.
- .start_state
Instance Method Summary
-
#env
@return [Environment] Environment used by an instantiated lexer.
-
#start_state
@return [Symbol] State in which the lexer starts.
Macro Summary
Instance methods inherited from class Object
in?(collection : Array | Set)
in?
Class Method Detail
Lex string, using env as the environment. This method will return the array of tokens generated by the lexer with a token of type EOS (End of Stream) appended to the end.
@param [String] string String to be lexed. @param [String] file_name File name used for recording token positions. @param [Environment] env Lexing environment.
@return [Array
A wrapper function that calls {Lexer.lex} on the contents of a file.
@param [String] file_name File to be lexed. @param [Environment] env Lexing environment.
@return [Array
Used to tell a lexer to use the first match found instead of the longest match found.
@return [void]
Changes the starting state of the lexer.
@param [Symbol] state Starting state for this lexer.
@return [void]