docs: improve docs around lexical precedence

This commit is contained in:
Amaan Qureshi 2025-01-10 23:26:37 -05:00
parent 5a2c5ed865
commit 1695e454a7
2 changed files with 9 additions and 2 deletions

View file

@ -55,6 +55,12 @@ a true ambiguity or a *local* ambiguity given one token of lookahead, Tree-sitte
the rule with the higher precedence. The default precedence of all rules is zero. This works similarly to the
[precedence directives][yacc-prec] in Yacc grammars.
This function can also be used to assign lexical precedence to a given
token, but it must be wrapped in a `token` call, such as `token(prec(1, 'foo'))`. This reads as "the token `foo` has a
lexical precedence of 1". The purpose of lexical precedence is to solve the issue where multiple tokens can match the same
set of characters, but one token should be preferred over the other. See [Lexical Precedence vs Parse Precedence][lexical vs parse]
for a more detailed explanation.
- **Left Associativity : `prec.left([number], rule)`** — This function marks the given rule as left-associative (and optionally
applies a numerical precedence). When an LR(1) conflict arises in which all the rules have the same numerical precedence,
Tree-sitter will consult the rules' associativity. If there is a left-associative rule, Tree-sitter will prefer matching
@ -137,6 +143,7 @@ object that coreesponds an empty array, signifying *no* keywords are reserved.
[ebnf]: https://en.wikipedia.org/wiki/Extended_Backus%E2%80%93Naur_form
[external-scanners]: ./4-external-scanners.md
[keyword-extraction]: ./3-writing-the-grammar.md#keyword-extraction
[lexical vs parse]: ./3-writing-the-grammar.md#lexical-precedence-vs-parse-precedence
[lr-conflict]: https://en.wikipedia.org/wiki/LR_parser#Conflicts_in_the_constructed_tables
[named-vs-anonymous-nodes]: ../using-parsers/2-basic-parsing.md#named-vs-anonymous-nodes
[rust regex]: https://docs.rs/regex/1.1.8/regex/#grouping-and-flags

View file

@ -418,8 +418,8 @@ which rule is chosen to interpret a given sequence of tokens. _Lexical precedenc
at a given position of text, and it is a lower-level operation that is done first. The above list fully captures Tree-sitter's
lexical precedence rules, and you will probably refer back to this section of the documentation more often than any other.
Most of the time when you really get stuck, you're dealing with a lexical precedence problem. Pay particular attention to
the difference in meaning between using `prec` inside the `token` function versus outside it. The _lexical precedence_ syntax
is `token(prec(N, ...))`.
the difference in meaning between using `prec` inside the `token` function versus outside it. The _lexical precedence_ syntax,
as mentioned in the previous page, is `token(prec(N, ...))`.
## Keywords