class Pegmatite::TokenIterator
- Pegmatite::TokenIterator
- Reference
- Object
Overview
TokenIterator is used to traverse a flat array of Tokens as if it were a tree.
Child tokens in the tokens array are represented as tokens that follow after their parent in the array and have an offset range whose finish offset is less than or equal to the the finish offset of the parent.
TokenIterator is aware of this relationship and can be used to easily traverse them, given a little discipline on the part of the caller. In particular, the caller must commit to consuming tokens in a depth-first traversal pattern in order for child relationships to be properly observed.
See spec/fixtures/json.cr for a real-world example of using TokenIterator.
Defined in:
pegmatite/token_iterator.crConstructors
Instance Method Summary
-
#assert_next_not_child_of(parent : Token)
Raise IndexError if the next token is a child of the given parent token.
-
#next : Token
Consume the next token and return it.
-
#next_as_child_of(parent : Token) : Token
Consume the next token if it is a child of parent and return it.
-
#peek : Token | Nil
Return the next token without consuming it.
-
#peek_as_child_of(parent : Token) : Token | Nil
Return the next token without consuming it, if it is a child of parent.
-
#while_next_is_child_of(parent : Token, &)
For each next token that is a child of the given parent, yield it.
Constructor Detail
Instance Method Detail
Raise IndexError if the next token is a child of the given parent token.
Consume the next token and return it. Raises IndexError if the end of the token stream has been reached.
Consume the next token if it is a child of parent and return it. Raises IndexError if isn't a child, or if at the end of the token stream.
Return the next token without consuming it. Returns nil if the end of the token stream has been reached.
Return the next token without consuming it, if it is a child of parent. Returns nil if isn't a child, or if at the end of the token stream.
For each next token that is a child of the given parent, yield it.
This method assumes that the code in the block calls this recursively or otherwise deals with any nested children of the yielded child token, in a pattern of depth-first traversal, because that is how the original flat token array is ordered. If this pattern is not followed then there is no guarantee that the following yields from this method will be correct.