Commit graph

1623 commits

Author SHA1 Message Date
Riley Bruins
6cabd9e67f fix(query)!: assert that predicates end in ! or ?
Predicates/directives are documented to end in either `!` or `?`.
However, `query.c` allows them to be any valid identifier, and also
allows `?` or `!` characters anywhere inside an identifier.

This commit removes `?` and `!` from the list of valid identifier
characters, and asserts that predicates/directives only *end* in `?` or
`!`, respectively.

This commit is breaking because you can no longer do something like
`(#eq? @capture foo!bar)` (`foo!bar` must now be quoted).
2025-06-06 10:34:00 +02:00
Will Lillis
ac13c86675 fix(cli): use a generic Into<Path> over single type for path arguments
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2025-06-05 02:41:18 -04:00
Will Lillis
50eaf5befd fix(query): correct expected and actual capture names in query assertions error message 2025-06-05 02:41:18 -04:00
Will Lillis
0220d9d93d feat(cli): add grammar-path to init, test, version, fuzz,
`query`, `highlight` and `tags` subcommands

Allows users to run various commands on a grammar without
being inside the directory for said grammar.
2025-06-05 02:41:18 -04:00
Thalia Archibald
6ba73fd888
fix(highlight): account for carriage return at EOF and chunk ends 2025-06-05 01:48:01 -04:00
Olive Easton
50622f71f8 fix(generate): re-enable default url features 2025-06-04 10:29:59 +02:00
Max Brunsfeld
2ab9c9b590
Fully fix field underflow in go_to_previous_sibling (#4483)
Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
2025-06-02 15:34:25 -07:00
Kai Pastor
0bdf698673
fix(build): pkgconfig and use of GNUInstallDirs (#4319)
* Fix pkgconfig

Init CMAKE_INSTALL_INCLUDEDIR before pc file generation.
Install pc file to CMAKE_INSTALL_LIBDIR/pkgconfig -
it accompanies the architecture-dependent library.

* Include GNUInstallDirs early

The CMake module initializes variables which are used for
exported information (CMake and pkgconfig).

* Change pc file install destination
2025-05-31 11:45:35 +02:00
Christian Clason
e7f9160867
Revert "feat: add build sha to parser.c header comment" (#4475)
This reverts commit dc4e232e6e.

Reason: The sha in the generated output (which most distro builds of
tree-sitter, including `cargo install`, strip) produces too many
conflicts when verifying via CI that parsers are regenerated on every
grammar change.
2025-05-29 11:47:06 -07:00
Max Brunsfeld
ca64399f9f Ignore lock files in grammar repos
It is very common practice to ignore
these lock files for libraries, since they do not apply to applications
that use the libraries. The lock files are especially not useful in
tree-sitter grammar repos, since tree-sitter grammars should not have
dependencies. The lock files are just a source of merge conflicts and
spurious CI failures.
2025-05-28 10:58:45 -07:00
Max Brunsfeld
b8ba76b1ae Fix hang in npm install script 2025-05-27 17:35:23 -07:00
Max Brunsfeld
f91255a201
Fix crash w/ goto_previous_sibling when parent node has leading extra child (#4472)
* Fix crash w/ goto_previous_sibling when parent node has leading extra
child Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>

Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>

* Fix lint

Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>

---------

Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
2025-05-27 16:56:33 -07:00
Will Lillis
b1d2b7cfb8 fix(query): correct last_child_step_index in cases where a new step
wasn't created.

This fixes an OOB access to `self.steps` when a last child anchor
immediately follows a predicate.
2025-05-03 17:27:37 -04:00
Will Lillis
cc634236b1 fix(rust): address new clippy lint 2025-05-03 19:57:11 +02:00
Amaan Qureshi
3056dc5be4 fix(generate): mark url as a Windows-only dependency 2025-04-28 18:31:09 -04:00
Will Lillis
dcdd5bc372
fix(parse): explicitly move temporaries in the logger callback
This fixes problems where these stack-local temporaries are used after their scope ends.
2025-04-28 03:30:05 -04:00
Will Lillis
d0e0e0322c
fix(xtask): properly trim emscripten version at the source (#4402)
* fix(xtask): properly trim emscripten version at the source

* fix(loader): trim emscripten version in `cargo:rustc-env=` print
2025-04-27 20:59:04 -07:00
Max Brunsfeld
ca7ff033db
Fix wasm build issues introduced by switching to wasi-sdk (#4407)
* Don't shell out for extracting tar.gz files

* Avoid wasi-sdk adding dependency on libc.so

* Clippy

* Fix -nostdlib flag
2025-04-27 19:42:34 -07:00
Max Brunsfeld
d4d8ed32b3
cli: Compile parsers to wasm using wasi-sdk, not emscripten (#4393)
* Build wasm modules using wasi-sdk, not emscripten

* Log a warning if --docker is passed
2025-04-24 16:00:09 -07:00
Will Lillis
c7475e4bf3 perf(highlight): use BTreeMap over IndexMap for highlight configs 2025-04-20 05:55:37 -04:00
Amaan Qureshi
69d553c82a style(cli): formatting 2025-04-19 01:15:53 -04:00
Daniel Jalkut
4514751803
docs(cli): improve documentation for the edits argument when parsing code 2025-04-18 23:45:52 -04:00
MichiRecRoom
853ca46899
docs(rust): improve bindings' crate doc 2025-04-18 22:55:59 -04:00
Paul Gey
b341073192 Make highlighting more deterministic when themes are ambiguous 2025-04-10 23:04:00 -04:00
Jon Shea
92c5d3b8e2 fix(rust): clarify error message for non-token reserved words
Improve the `NonTokenReservedWord` error message by including the
specific reserved word that was not used as a token.
2025-04-09 18:15:48 -04:00
Edgar Onghena
52d2865365
chore(generate): add @generated to parser.c header (#4338)
This makes `parser.c` follow the https://generated.at/ convention for generated files. This potentially allows any compatible IDE to discourage editing it directly.
2025-04-08 10:52:37 +02:00
Will Lillis
0f949168ef fix(cli): display "N/A" in parse stats where appropriate when no parsing
took place
2025-04-06 09:45:50 -04:00
Will Lillis
71941d8bda fix(highlight): account for multiple rows in highlight testing assertions 2025-04-06 09:45:50 -04:00
WillLillis
521da2b0a7 fix(rust): address new nightly lint for pointer comparisons 2025-03-26 08:58:50 +01:00
Christian Clason
37a9ecd5b7 build(deps): update emscripten to v4.0.5 2025-03-25 15:18:44 +01:00
dependabot[bot]
d05e4ae7ff build(deps): bump the cargo group across 1 directory with 8 updates
Bumps the cargo group with 8 updates in the / directory:

| Package | From | To |
| --- | --- | --- |
| [cc](https://github.com/rust-lang/cc-rs) | `1.2.16` | `1.2.17` |
| [clap_complete](https://github.com/clap-rs/clap) | `4.5.46` | `4.5.47` |
| [git2](https://github.com/rust-lang/git2-rs) | `0.20.0` | `0.20.1` |
| [log](https://github.com/rust-lang/log) | `0.4.26` | `0.4.27` |
| [once_cell](https://github.com/matklad/once_cell) | `1.21.0` | `1.21.1` |
| [tempfile](https://github.com/Stebalien/tempfile) | `3.18.0` | `3.19.1` |
| [widestring](https://github.com/starkat99/widestring-rs) | `1.1.0` | `1.2.0` |
| [ureq](https://github.com/algesten/ureq) | `3.0.8` | `3.0.10` |



Updates `cc` from 1.2.16 to 1.2.17
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/cc-rs/compare/cc-v1.2.16...cc-v1.2.17)

Updates `clap_complete` from 4.5.46 to 4.5.47
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.46...clap_complete-v4.5.47)

Updates `git2` from 0.20.0 to 0.20.1
- [Changelog](https://github.com/rust-lang/git2-rs/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/git2-rs/compare/git2-0.20.0...git2-0.20.1)

Updates `log` from 0.4.26 to 0.4.27
- [Release notes](https://github.com/rust-lang/log/releases)
- [Changelog](https://github.com/rust-lang/log/blob/master/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/log/compare/0.4.26...0.4.27)

Updates `once_cell` from 1.21.0 to 1.21.1
- [Changelog](https://github.com/matklad/once_cell/blob/master/CHANGELOG.md)
- [Commits](https://github.com/matklad/once_cell/compare/v1.21.0...v1.21.1)

Updates `tempfile` from 3.18.0 to 3.19.1
- [Changelog](https://github.com/Stebalien/tempfile/blob/master/CHANGELOG.md)
- [Commits](https://github.com/Stebalien/tempfile/compare/v3.18.0...v3.19.1)

Updates `widestring` from 1.1.0 to 1.2.0
- [Release notes](https://github.com/starkat99/widestring-rs/releases)
- [Changelog](https://github.com/VoidStarKat/widestring-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/starkat99/widestring-rs/compare/v1.1.0...v1.2.0)

Updates `ureq` from 3.0.8 to 3.0.10
- [Changelog](https://github.com/algesten/ureq/blob/main/CHANGELOG.md)
- [Commits](https://github.com/algesten/ureq/compare/3.0.8...3.0.10)

---
updated-dependencies:
- dependency-name: cc
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: clap_complete
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: git2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: log
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: once_cell
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: tempfile
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: cargo
- dependency-name: widestring
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: cargo
- dependency-name: ureq
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-03-25 12:24:08 +01:00
WillLillis
f0e7ac2f18 chore(cli): remove deprecated --no-bindings flag from generate command 2025-03-16 11:04:29 +01:00
Peter Oliver
17471bdfcc
fix(build): make install shouldn’t fail when a parser bundles no queries (#4284) 2025-03-14 09:37:51 +01:00
Christian Clason
07a1b7fc00 fix(build): bump version to 0.26.0
After a release, `master` branch should be bumped to the next _minor_ version.
2025-03-12 14:52:09 +01:00
dependabot[bot]
12aff698b9 build(deps): bump emscripten to 4.0.4 2025-03-12 09:30:11 +00:00
Riley Bruins
cc3994928c feat(lib): quote invalid nodes, fields, and captures 2025-03-12 10:28:57 +01:00
WillLillis
efd212ee46 fix(cli): properly escape invisible characters in parse error output 2025-03-11 17:27:12 -04:00
WillLillis
11071ed682 fix(rust): adapt to new nightly lint 2025-03-06 09:32:21 +01:00
Max Brunsfeld
066fd77d39
Fix cases where error recovery could infinite loop (#4257)
* Rename corpus test functions to allow easy filtering by language

* Use usize for seed argument

* Avoid retaining useless stack versions when reductions merge

We found this problem when debugging an infinite loop that happened
during error recovery when using the Zig grammar. The large number of
unnecessary paused stack versions were preventing the correct recovery
strategy from being tried.

* Fix leaked lookahead token when reduction results in a merged stack

* Enable running PHP tests in CI

* Fix possible infinite loop during error recovery at EOF

* Account for external scanner state changes when detecting changed ranges in subtrees
2025-03-04 13:50:56 -08:00
polazarus
8138dba800
fix(generate): mark TSCharacterRange as static (#4255)
Problem: Linking different parsers into one executable fails due to duplicate symbols.

Solution: Mark `TSCharacterRange` as `static` when generating parsers.

fixes #4209
2025-03-04 15:26:13 +01:00
Yuri Astrakhan
b749c02eb5 Mark all format-like macros for Clippy
See https://doc.rust-lang.org/nightly/clippy/attribs.html#clippyformat_args

Ensure that usage of the macros with `format!`-like parameters will be validated by Clippy
2025-03-03 02:16:23 -05:00
WillLillis
998fb34d15 fix(test): update expected tree-sitter-rust supertypes 2025-03-02 16:37:03 -05:00
WillLillis
cb30ec5b17 fix(rust): adapt to some new nightly lints 2025-03-02 16:37:03 -05:00
Antonin Décimo
f0571b1e33
feat: Windows support using MinGW-w64 (#4201)
Building tree-sitter on Windows within a Cygwin or MSYS2 environment
for MinGW-w64 targets should work. As there's no configure-like step,
the tools have to be specified with:

    make CC=x86_64-w64-mingw32-gcc \
         AR=x86_64-w64-mingw32-ar \
         STRIP=x86_64-w64-mingw32-strip

Useful reference:
[How to build and use DLLs on Windows](https://nullprogram.com/blog/2021/05/31/)

This commit doesn't tag public functions with dllexport or
dllimport. This results in exporting non-static functions visible
between translation units, and generating slightly less efficient code
for calling exported functions from the DLL.

This commit doesn't include support to build libtree-sitter with MSVC or
clang-cl, but generates a libtree-sitter.lib file for MSVC/clang-cl
consumers.
2025-02-12 16:31:06 -08:00
Amaan Qureshi
a40265cbeb refactor(web): rename tree-sitter.js to web-tree-sitter.js
This is not breaking for consumers of the web bindings, nor the
playground as both filenames will be kept in the .github.io repo
2025-02-11 22:56:50 -05:00
Amaan Qureshi
74d7ca8582 fix(rust): adapt to new clippy lints 2025-02-08 13:15:56 -05:00
Amaan Qureshi
f5afe475de
build: bump version to 0.25.1 2025-02-02 02:07:36 -05:00
Amaan Qureshi
eed662df98
fix(bindings): correct Zig bindings to expose a language function
Instead of having users declare the extern function themselves, they can
pass in the language to `Language.create` in the zig bindings. If they
really want, they can always opt into the `extern fn tree_sitter_LANG()
*const ts.Language` approach.
2025-02-02 02:07:36 -05:00
Riley Bruins
9ad096ef22 fix(lib): prevent finished_tree assertion failure
**Problem:** When resetting the parser during subtree balancing, an
error is thrown:

```
parser.c:2198: ts_parser_parse: Assertion `self->finished_tree.ptr' failed.
```

**Solution:** Reset `canceled_balancing` to false in
`ts_parser_reset()`.
2025-02-01 16:19:14 -05:00
ObserverOfTime
c2221f2732 feat(cli): specify abi version via env var 2025-02-01 13:57:53 -05:00