From e58b1faac1e4ad35e60eaa940268f4c47dcc09db Mon Sep 17 00:00:00 2001 From: Amaan Qureshi Date: Fri, 10 Jan 2025 02:29:46 -0500 Subject: [PATCH] docs: add note on not duplicating the word token rule --- docs/src/creating-parsers/3-writing-the-grammar.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/docs/src/creating-parsers/3-writing-the-grammar.md b/docs/src/creating-parsers/3-writing-the-grammar.md index 837a0691..a42c99a4 100644 --- a/docs/src/creating-parsers/3-writing-the-grammar.md +++ b/docs/src/creating-parsers/3-writing-the-grammar.md @@ -427,6 +427,12 @@ It would then correctly recognize the code as invalid. Aside from improving error detection, keyword extraction also has performance benefits. It allows Tree-sitter to generate a smaller, simpler lexing function, which means that **the parser will compile much more quickly**. +```admonish note +The word token must be a unique token that is not reused by another rule. If you want to have a word token used in a +rule that's called something else, you should just alias the word token instead, like how the Rust grammar does it +here +``` + [ambiguous-grammar]: https://en.wikipedia.org/wiki/Ambiguous_grammar [antlr]: https://www.antlr.org [bison]: https://en.wikipedia.org/wiki/GNU_bison