class Mint::SemanticTokenizer
- Mint::SemanticTokenizer
- Reference
- Object
Defined in:
semantic_tokenizer.crConstant Summary
-
TOKEN_MAP =
{Ast::TypeVariable => TokenType::TypeParameter, Ast::Variable => TokenType::Variable, Ast::Comment => TokenType::Comment, Ast::StringLiteral => TokenType::String, Ast::RegexpLiteral => TokenType::Regexp, Ast::NumberLiteral => TokenType::Number, Ast::TypeId => TokenType::Type}
-
This represents which token types are used for which node.
-
TOKEN_TYPES =
TokenType.names.map!(&.camelcase(lower: true))
Class Method Summary
Instance Method Summary
- #add(from : Int32, to : Int32, type : TokenType)
- #add(node : Ast::Node, type : TokenType)
-
#cache : Set(Mint::Ast::Node)
We keep a cache of all tokenized nodes to avoid duplications
- #tokenize(ast : Ast)
- #tokenize(nodes : Array(Ast::Node))
- #tokenize(node : Ast::CssDefinition)
- #tokenize(node : Ast::ArrayAccess)
- #tokenize(node : Ast::HtmlElement)
- #tokenize(node : Ast::HtmlComponent)
- #tokenize(node : Ast::Node | Nil)
-
#tokens : Array(Mint::SemanticTokenizer::Token)
This is where the resulting tokens are stored.
Class Method Detail
Instance Method Detail
This is where the resulting tokens are stored.