traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
5957 commits 16 branches 98 tags 37 MiB
Commit graph

41 commits

Author SHA1 Message Date
ObserverOfTime
d13657c40c refactor(generate): use the logger 
Co-authored-by: Amaan Qureshi <git@amaanq.com>
 2025-09-21 01:53:22 -04:00
Amaan Qureshi
311585d304 refactor!: rename stage flag to emit 2025-09-20 22:35:23 -04:00
Will Lillis
46ea65c89b refactor: remove url dependency 2025-09-17 04:31:53 -04:00
Will Lillis
6a28a62369 test: add safety checks to ensure langauge version constants are kept in 
sync

The generate crate defines the `LANGUAGE_VERSION` constant separately
from the TREE_SITTER_LANGUAGE_VERSION definition in `api.h`.
 2025-09-17 02:58:31 -04:00
Amaan Qureshi
317e2e74c2 Revert "feat(generate): allow more characters for keywords" 
This reverts commit 0269357c5a.
 2025-09-17 02:19:29 -04:00
Amaan Qureshi
57c6105897 fix(generate): remove warning message for CJS grammars 2025-09-16 16:42:17 -04:00
Christian Clason
339bad2de4 feat(generate): don't embed tree-sitter CLI version in parser 
Problem: embedding the CLI version used to generate a parser triggers CI
failures on all grammars for every (patch) release of tree-sitter, even
if there are no actual parser changes.

Solution: do not embed the version; instead rely on whether the update
introduces actual (presumably desirable) changes in the parser to
indicate regeneration is necessary.
 2025-09-16 19:21:34 +02:00
Will Lillis
31ff62445b fix(generate): assert there is a Nfa last state before retrieving it 
Prevents unsigned subtraction wrapping antics in release builds
 2025-09-16 03:51:13 -04:00
bbb651
9593737871 build(generate): remove tree-sitter dependency 
It was only used to share two constants, and balloons its dependencies.

This also makes `generate_parser_for_grammar` work in wasm.
(Tested in `wasm32-wasip2` in wasmtime with the json grammar,
`wasm32-unknown-unknown` running in the same setup exited successfully
so I'm pretty confident it works as well)

Co-authored-by: Amaan Qureshi <contact@amaanq.com>
 2025-09-16 03:48:30 -04:00
Amaan Qureshi
0269357c5a feat(generate): allow more characters for keywords 2025-09-16 03:01:56 -04:00
Amaan Qureshi
39a67eec61 feat: migrate to ESM 2025-09-16 02:24:11 -04:00
Amaan Qureshi
eedbec8f24 feat: remove the need of an external JS runtime for processing grammars 2025-09-16 02:24:11 -04:00
Shadaj Laddad
27e5147a5f feat(generate): place file APIs behind a feature flag 2025-09-07 11:44:09 +03:00
Will Lillis
5fd818babe fix(generate): use correct state id when adding terminal states to 
non terminal extras
 2025-09-04 04:15:51 -04:00
Will Lillis
310c0b86a7 fix(generate): return error when single state transitions have 
indirectly recursive cycles.

This can cause infinite loops in the parser near EOF.

Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
 2025-09-04 00:59:44 -04:00
ObserverOfTime
b57b7213a9 fix(rust): make some methods const 
Addresses clippy::missing-const-for-fn
 2025-08-30 16:47:47 +03:00
Will Lillis
ac171eb280 fix(generate): warn users when extra rule can lead to parser hang 
When a *named* rule in the extras is able to match the empty string,
parsing can hang in certain situations (i.e. near EOF).
 2025-08-29 21:38:41 -04:00
Amaan Qureshi
1152bf4c9d fix(cli): show rule names for rules with metadata 2025-08-28 13:28:37 -04:00
Will Lillis
e749a97f9f fix(rust): make String clones more explicit 2025-08-09 16:02:28 -04:00
Will Lillis
3d9f4d1bd3 fix(rust): remove redundant String clones 2025-08-09 16:02:28 -04:00
Pieter Goetschalckx
07b4c8d05d
fix(cli): add reserved type declarations and schema 
- Use `globalThis` for `reserved` function export
- Add `reserved` field and function to DSL declarations
- Add `reserved` rule to grammar schema
 2025-08-02 12:01:41 +03:00
Antonin Delpeuch
8e90799e27 feat(generate): Support for --stage=json/parser/lib 2025-07-26 22:48:00 -04:00
Antonin Delpeuch
d810217e63 refactor: Expose node_types_json without generating C code 2025-07-26 22:48:00 -04:00
Antonin Delpeuch
8676eda663 feat: Only evaluate grammar.js to grammar.json 
This adds an `--evaluate-only` option to `tree-sitter generate`
so that it only does the evaluation of `grammar.js` to
`src/grammar.json`, without continuing on with the generation of
`src/parser.c` and related files.

It's a follow-up to #4580.
 2025-07-26 22:48:00 -04:00
Christian Clason
c18d019db0 Revert "feat(generate): warn about unreachable variables (#4567)" 
This reverts commit 25c601bd2f.

Reason: Too many false positives.
 2025-07-22 09:49:26 +02:00
Antonin Delpeuch
25c601bd2f
feat(generate): warn about unreachable variables (#4567) 2025-07-21 11:39:18 +02:00
Will Lillis
86b2c939c7 fix(rust): address nightly clippy lint useless_if_let_seq 2025-07-20 21:14:47 -04:00
Will Lillis
0f79c61188 perf(generate): reserve more Vec capacities 2025-07-17 01:51:56 -04:00
Will Lillis
854f527f6e perf: reorder conditional operands 2025-07-13 19:42:30 +02:00
Will Lillis
c740f244ba fix(generate): propagate node types error 2025-07-13 19:42:30 +02:00
Will Lillis
5ed2c77b59 perf: reorder expensive conditional operand 2025-07-12 13:51:54 -04:00
Will Lillis
ad0f953c21 chore(rust): remove empty grammar_files file 2025-07-10 23:44:10 -04:00
Will Lillis
3c56f53105 fix(test): unwrap result of get_node_types in 
`test_node_types_deeper_extras` test
 2025-07-10 17:42:52 -04:00
WillLillis
d2e06bf130 fix(generate): use topological sort for subtype map 2025-07-10 08:57:55 -04:00
Antonin Delpeuch
36d93aeff3 perf: More efficient computation of used symbols 
As the call to `symbol_is_used` does not depend
on the production, it is more efficient to call it
only once outside the loop over productions.

I'm not sure if `rustc` is able to do this optimization
on its own (it would need to know that the function
is pure, which sounds difficult in general).
 2025-07-09 22:34:53 -04:00
Will Lillis
1e7d77c517 perf: reserve Vec capacities where appropriate 2025-07-09 21:46:25 -04:00
liangmulu
81849352fc chore: remove redundant words in comment 
Signed-off-by: liangmulu <liangmulu@outlook.com>
 2025-07-09 18:38:37 +02:00
Antonin Delpeuch
1a3b0375fa Fix 'extra' field generation for node-types.json 2025-07-07 21:15:09 -04:00
Antonin Delpeuch
59bcffe83b Add test demonstrating failure to populate 'extra' 
The test is currently failing, will be fixed by the next commit.
 2025-07-07 21:15:09 -04:00
Will Lillis
a9818e4b17 fix(generate): only display conflicting symbol name in non-terminal 
word token error message if available
 2025-06-26 00:55:03 -04:00
Max Brunsfeld
0fdf569571
Reorganize rust crates into a flat crates directory, simplify some CI steps (#4496) 
* Move all rust crates (except lib) into crates dir, w/o nesting

* Remove stale path from .gitattributes

* Rename lib.rs files for easier navigation

* Rename mod.rs file for easier navigation

* Fix emscripten-version path

* Fix fixtures dir paths

* Use the default rustfmt settings

* Don't use nightly on CI
 2025-06-06 14:25:37 -07:00