Commit graph

66 commits

Author SHA1 Message Date
Will Lillis
14c4d2f8ca fix(generate): return error when single state transitions have
indirectly recursive cycles.

This can cause infinite loops in the parser near EOF.

Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
(cherry picked from commit 310c0b86a7)
2025-09-04 01:47:36 -04:00
Will Lillis
253003ccf8 fix(generate): warn users when extra rule can lead to parser hang
When a *named* rule in the extras is able to match the empty string,
parsing can hang in certain situations (i.e. near EOF).

(cherry picked from commit ac171eb280)
2025-08-29 23:32:57 -04:00
Pieter Goetschalckx
05bfeb5b69 fix(cli): add reserved type declarations and schema
- Use `globalThis` for `reserved` function export
- Add `reserved` field and function to DSL declarations
- Add `reserved` rule to grammar schema

(cherry picked from commit 07b4c8d05d)
2025-08-02 11:51:09 +02:00
Will Lillis
58edb3a11c perf(generate): reserve more Vec capacities
(cherry picked from commit 0f79c61188)
2025-07-19 12:32:09 +02:00
Will Lillis
8bb33f7d8c perf: reorder conditional operands
(cherry picked from commit 854f527f6e)
2025-07-13 20:05:01 +02:00
Will Lillis
6f944de32f fix(generate): propagate node types error
(cherry picked from commit c740f244ba)
2025-07-13 20:05:01 +02:00
Will Lillis
94b55bfcdc perf: reorder expensive conditional operand
(cherry picked from commit 5ed2c77b59)
2025-07-12 20:17:47 +02:00
WillLillis
bcb30f7951 fix(generate): use topological sort for subtype map 2025-07-10 17:43:08 -04:00
Antonin Delpeuch
3bd8f7df8e perf: More efficient computation of used symbols
As the call to `symbol_is_used` does not depend
on the production, it is more efficient to call it
only once outside the loop over productions.

I'm not sure if `rustc` is able to do this optimization
on its own (it would need to know that the function
is pure, which sounds difficult in general).

(cherry picked from commit 36d93aeff3)
2025-07-10 09:25:22 +02:00
Will Lillis
d7529c3265 perf: reserve Vec capacities where appropriate
(cherry picked from commit 1e7d77c517)
2025-07-09 22:33:57 -04:00
Antonin Delpeuch
bb7b339ae2 Fix 'extra' field generation for node-types.json
(cherry picked from commit 1a3b0375fa)
2025-07-07 21:58:47 -04:00
Antonin Delpeuch
9184a32b4b Add test demonstrating failure to populate 'extra'
The test is currently failing, will be fixed by the next commit.

(cherry picked from commit 59bcffe83b)
2025-07-07 21:58:47 -04:00
Will Lillis
6b84118e33 fix(generate): only display conflicting symbol name in non-terminal
word token error message if available

(cherry picked from commit a9818e4b17)
2025-06-26 15:40:48 +02:00
James McCoy
415a657d08 fix(test): remove period in test_flatten_grammar_with_recursive_inline_variable
The period was dropped in the `thiserror` refactor
(79444e07f9), which caused the
`test_flatten_grammar_with_recursive_inline_variable` test to fail.

Signed-off-by: James McCoy <jamessan@jamessan.com>
(cherry picked from commit a6e530b33d)
2025-06-06 16:39:45 +02:00
Olive Easton
8ef6f0685b fix(generate): re-enable default url features
(cherry picked from commit 50622f71f8)
2025-06-04 10:56:00 +02:00
Christian Clason
baf222f772 Revert "feat: add build sha to parser.c header comment" (#4475)
This reverts commit dc4e232e6e.

Reason: The sha in the generated output (which most distro builds of
tree-sitter, including `cargo install`, strip) produces too many
conflicts when verifying via CI that parsers are regenerated on every
grammar change.

(cherry picked from commit e7f9160867)
2025-05-29 23:14:25 +02:00
Amaan Qureshi
41413e7a71 fix(generate): mark url as a Windows-only dependency
(cherry picked from commit 3056dc5be4)
2025-04-29 09:20:34 +02:00
Jon Shea
e295c99eca fix(rust): clarify error message for non-token reserved words
Improve the `NonTokenReservedWord` error message by including the
specific reserved word that was not used as a token.

(cherry picked from commit 92c5d3b8e2)
2025-04-10 01:10:25 -04:00
Edgar Onghena
d2914ca243 chore(generate): add @generated to parser.c header (#4338)
This makes `parser.c` follow the https://generated.at/ convention for generated files. This potentially allows any compatible IDE to discourage editing it directly.

(cherry picked from commit 52d2865365)
2025-04-08 11:20:25 +02:00
Amaan Qureshi
ff8bf05def fix(rust): adapt to new clippy lints
(cherry picked from commit 74d7ca8582)
2025-04-06 16:12:21 +02:00
WillLillis
c313be63b2 fix(rust): adapt to new nightly lint
(cherry picked from commit 11071ed682)
2025-03-06 18:25:24 -05:00
polazarus
b1a7074010 fix(generate): mark TSCharacterRange as static (#4255)
Problem: Linking different parsers into one executable fails due to duplicate symbols.

Solution: Mark `TSCharacterRange` as `static` when generating parsers.

fixes #4209

(cherry picked from commit 8138dba800)
2025-03-04 16:52:58 +01:00
WillLillis
4eb46b493f fix(rust): adapt to some new nightly lints
(cherry picked from commit cb30ec5b17)
2025-03-02 23:20:08 +01:00
Amaan Qureshi
959f096a89 feat(generate): defer to ABI 14 if tree-sitter.json doesn't exist, rather than hard failing 2025-01-26 13:12:58 -05:00
Amaan Qureshi
8bb1448a6f feat: add the semantic version to TSLanguage, and expose an API for retrieving it 2025-01-25 01:14:30 -05:00
Yuri Astrakhan
3e7721e554 chore: a few more minor lints 2025-01-23 00:10:16 -05:00
Yuri Astrakhan
9dbe165296 chore: a few minor lints
* do not use `&` for the format args as it cannot (yet) be optimized by the compiler
* a few format inlining
2025-01-21 00:57:58 -05:00
Yuri Astrakhan
48059b72a8 feat: remove lazy_static in favor of LazyLock
This switches to the built-in `std::sync::LazyLock`
2025-01-21 00:21:35 -05:00
WillLillis
23e0891cd5 fix(cli): improve error message for nonterminals used in token rule 2025-01-13 01:12:20 -05:00
Amaan Qureshi
b26adf4265 feat(generate): add an extra field for extra nodes in node-types.json 2025-01-12 18:41:25 -05:00
Amaan Qureshi
ac8bb1b777 build: bump other crates' MSRV to 1.82 2025-01-12 04:47:59 -05:00
Amaan Qureshi
52ef2992aa feat(generate): properly filter out unused rules 2025-01-12 04:47:59 -05:00
Amaan Qureshi
9d9c76e693 feat(generate): explicitly disallow non-terminals in non-terminals 2025-01-12 04:47:59 -05:00
WillLillis
329dcc92a1 fix(rust): adapt to a few new nightly lints 2025-01-10 22:00:23 -05:00
Amaan Qureshi
d38aa596e1 fix(generate): improve error message when a duplicate token is used as the word token 2025-01-10 02:58:38 -05:00
Amaan Qureshi
4d6740980c fix(generate): improve display of token sets with escape characters 2025-01-10 02:58:38 -05:00
Amaan Qureshi
68e707eb4f feat: support passing in a Rust regex in the grammar dsl 2025-01-06 20:52:19 -05:00
WillLillis
867433afd7 feat(rust): use thiserror for generate crate
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2025-01-05 00:27:12 -05:00
Riley Bruins
19482834bd feat: add Supertype API
Introduces a new function that takes in a supertype symbol and returns
all associated subtypes. Can be used by query.c to give better errors
for invalid subtypes, as well as downstream applications like the query
LSP to give better diagnostics.
2025-01-05 00:14:09 -05:00
Owen Shepherd
77f74a05c7
chore(generate): remove unused fields 2025-01-02 22:11:04 -05:00
Max Brunsfeld
201b41cf11
feat: add 'reserved word' construct
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-12-23 03:06:32 -05:00
Amaan Qureshi
8744a4e3f2 feat(lib): use const for TSCharacterRanges 2024-12-23 01:19:10 -05:00
Amaan Qureshi
274e60a523
fix: clippy lints 2024-11-16 03:20:59 -05:00
Amaan Qureshi
05b2f443ba
fix: clippy lint 2024-11-15 23:54:18 -05:00
Riley Bruins
bcf82da55c refactor: reuse symbol variable in subtype map generator
Tiny change to prevent redundant creation of a symbol value in
`node_types.rs`.
2024-11-13 20:54:57 -05:00
Amaan Qureshi
998d2c9d8c fix(generate): do not set the unit reduction symbol if it's in the extras array 2024-11-02 04:20:40 -04:00
Amaan Qureshi
310a9f0704 fix: disallow tokens that match the empty string 2024-11-02 03:36:11 -04:00
Amaan Qureshi
7baefa8609 fix(cli): use contains over is in warning 2024-11-02 01:31:46 -04:00
dundargoc
a3de650024 fix: make sha generation work with submodules
More specifically, change `is_dir` to `exists` as `.git` is a file when
in a submodule.
2024-10-28 12:16:48 +01:00
dundargoc
dc4e232e6e feat: add build sha to parser.c header comment 2024-10-28 12:16:48 +01:00