class Mint::SemanticTokenizer

• Mint::SemanticTokenizer
• Reference
• Object

Defined in:

semantic_tokenizer.cr

Constant Summary

TOKEN_MAP = {Ast::TypeVariable => TokenType::TypeParameter, Ast::Comment => TokenType::Comment, Ast::RegexpLiteral => TokenType::Regexp, Ast::NumberLiteral => TokenType::Number, Ast::Id => TokenType::Type}

This represents which token types are used for which node.

TOKEN_TYPES = TokenType.names.map!(&.camelcase(lower: true))

This is used by the language server.

Class Method Summary

• .highlight(path : String, *, html : Bool = false) : String

  Highlights the file.

• .tokenize(ast : Ast) : Array(Item)

  Returns the tokenized version of the AST.

• .tokenize_with_lines(ast : Ast) : Array(Array(Item))

  Returns the tokenized version of the AST where the lines contain their respective tokens.

Instance Method Summary

• #add(from : Parser::Location, to : Parser::Location, type : TokenType)
• #add(node : Ast::Node, type : TokenType)
• #cache : Set(Mint::Ast::Node)

  We keep a cache of all tokenized nodes to avoid duplications

• #tokenize(nodes : Array(Ast::Node))
• #tokenize(node : Ast::CssDefinition)
• #tokenize(node : Ast::HtmlComponent)
• #tokenize(node : Ast::HtmlElement)
• #tokenize(node : Ast::Variable)
• #tokenize(node : Ast::StringLiteral | Ast::HereDocument)
• #tokenize(node : Ast::Node | Nil)
• #tokenize(ast : Ast) : Nil
• #tokens : Array(Mint::SemanticTokenizer::Token)

  This is where the resulting tokens are stored.

Class Method Detail

Instance Method Detail