traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

docs: add note on not duplicating the word token rule

Browse source
This commit is contained in:
Amaan Qureshi 2025-01-10 02:29:46 -05:00
parent d38aa596e1
commit e58b1faac1
1 changed files with 6 additions and 0 deletions
Download patch file Download diff file Expand all files Collapse all files

6
docs/src/creating-parsers/3-writing-the-grammar.md
View file

@ -427,6 +427,12 @@ It would then correctly recognize the code as invalid.
Aside from improving error detection, keyword extraction also has performance benefits. It allows Tree-sitter to generate
a smaller, simpler lexing function, which means that **the parser will compile much more quickly**.
```admonish note
The word token must be a unique token that is not reused by another rule. If you want to have a word token used in a
rule that's called something else, you should just alias the word token instead, like how the Rust grammar does it
<a href="https://github.com/tree-sitter/tree-sitter-rust/blob/1f63b33efee17e833e0ea29266dd3d713e27e321/grammar.js#L1605">here</a>
```
[ambiguous-grammar]: https://en.wikipedia.org/wiki/Ambiguous_grammar
[antlr]: https://www.antlr.org
[bison]: https://en.wikipedia.org/wiki/GNU_bison