traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: remove controversial Earliest Starting Position item added previously by 87a0517

Browse source
This commit is contained in:
Andrew Hlynskyi 2023-04-14 03:43:03 +03:00
parent f403edc9eb
commit bfd56a1e59
1 changed files with 0 additions and 2 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/section-3-creating-parsers.md
View file

@ -530,8 +530,6 @@ Grammars often contain multiple tokens that can match the same characters. For e
1. **Context-Aware Lexing** - Tree-sitter performs lexing on-demand, during the parsing process. At any given position in a source document, the lexer only tries to recognize tokens that are *valid* at that position in the document.
1. **Earliest Starting Position** - Tree-sitter will prefer tokens with an earlier starting position. This is most often seen with very permissive regular expressions similar to `/.*/`, which are greedy and will consume as much text as possible. In this example the regex would consume all text until hitting a newline - even if text on that line could be interpreted as a different token.
1. **Explicit Lexical Precedence** - When the precedence functions described [above](#the-grammar-dsl) are used within the `token` function, the given precedence values serve as instructions to the lexer. If there are two valid tokens that match the characters at a given position in the document, Tree-sitter will select the one with the higher precedence.
1. **Match Length** - If multiple valid tokens with the same precedence match the characters at a given position in a document, Tree-sitter will select the token that matches the [longest sequence of characters][longest-match].