traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

style: correct typos

Browse source
This commit is contained in:
Will Lillis 2024-12-23 02:11:09 -05:00 • committed by GitHub
parent faf97b896a
commit 2a63077cac
No known key found for this signature in database
GPG key ID: B5690EEEBB952194
7 changed files with 14 additions and 14 deletions
Download patch file Download diff file Expand all files Collapse all files

2
docs/section-3-creating-parsers.md
View file

@ -1078,7 +1078,7 @@ If you put terminal keywords in the `externals` array, for example `externals: $
If in the `externals` array use literal keywords then lexing works in two steps, the external scanner will be called first and if it sets a resulting token and returns `true` then the token considered as recognized and Tree-sitter moves to a next token. But the external scanner may return `false` and in this case Tree-sitter fallbacks to the internal lexing mechanism.
In case of some keywords defined in the `externals` array in a rule referencing form like `$.if_keyword` and there is no additional definition of that rule in the grammar rules, e.g., `if_keyword: $ => 'if'` then fallback to the internal lexer isn't possible because Tree-sitter doesn't know the actual keyword and it's fully the external scanner resposibilty to recognize such tokens.
In case of some keywords defined in the `externals` array in a rule referencing form like `$.if_keyword` and there is no additional definition of that rule in the grammar rules, e.g., `if_keyword: $ => 'if'` then fallback to the internal lexer isn't possible because Tree-sitter doesn't know the actual keyword and it's fully the external scanner responsibilty to recognize such tokens.
External scanners are a common cause of infinite loops.
Be very careful when emitting zero-width tokens from your external scanner, and if you consume characters in a loop be sure use the `eof` function to check whether you are at the end of the file.