Commit graph

1153 commits

Author SHA1 Message Date
Riley Bruins
6850df969d fix(query): prevent cycles when analyzing hidden children
**Problem:** `query.c` compares the current analysis state with the
previous analysis state to see if they are equal, so that it can return
early if so. This prevents redundant work. However, the comparison
function here differs from the one used for sorted insertion/lookup in
that it does not check any state data other than the child index. This
is problematic because it leads to infinite analysis when hidden nodes
have cycles.

**Solution:** Remove the custom comparison function, and apply the
insertion/lookup comparison function in place of it.

**NOTE:** This commit also changes the comparison function slightly, so
that some comparisons are reordered. Namely, for performance, it returns
early if the lhs depth is less than the rhs depth. Is this acceptable?
Tests still pass and nothing hangs in my testing, but it still seems
sketchy. Returning early if the lhs depth is greater than the rhs depth
does seem to make query analysis hang, weirdly enough... Keeping the
depth checks at the end of the loop also works, but it introduces a
noticeable performance regression (for queries that otherwise wouldn't
have had analysis cycles, of course).
2025-07-30 00:41:01 -04:00
Daniel Müller
f0165dee92 feat(rust): Add direct links to documentation
Make the URL to the documentation of the various crates known so that it
will be easier to click through to it from a crates.io search [0].

[0] https://doc.rust-lang.org/cargo/reference/manifest.html#the-documentation-field
2025-07-22 18:36:08 -04:00
Riley Bruins
dff828cdbe fix(rust): prevent overflow in error message calculation
**Problem:** When encountering an invalid symbol at the beginning of the
file, the rust bindings attempt to index the character at position -1 of
the query source, which leads to an overflow and thus invalid character
index which causes a panic.

**Solution:** Bounds check the offset before performing the subtraction.
2025-07-22 09:51:28 +02:00
Will Lillis
86b2c939c7 fix(rust): address nightly clippy lint useless_if_let_seq 2025-07-20 21:14:47 -04:00
Alex Aron
aeab755033
fix(lib): add wasm32 support to portable/endian.h (#4607) 2025-07-14 17:47:40 +02:00
RedCMD
912167a9cd
chore(web): fix indentation in type files 2025-07-14 12:50:04 +02:00
Will Lillis
be8b3e282a fix(rust): remove unused Command import 2025-06-26 00:54:44 -04:00
dependabot[bot]
8a15b881fd build(deps): bump esbuild and tsx in /lib/binding_web
Bumps [esbuild](https://github.com/evanw/esbuild) to 0.25.0 and updates ancestor dependency [tsx](https://github.com/privatenumber/tsx). These dependencies need to be updated together.


Updates `esbuild` from 0.23.1 to 0.25.0
- [Release notes](https://github.com/evanw/esbuild/releases)
- [Changelog](https://github.com/evanw/esbuild/blob/main/CHANGELOG-2024.md)
- [Commits](https://github.com/evanw/esbuild/compare/v0.23.1...v0.25.0)

Updates `tsx` from 4.19.2 to 4.19.3
- [Release notes](https://github.com/privatenumber/tsx/releases)
- [Changelog](https://github.com/privatenumber/tsx/blob/master/release.config.cjs)
- [Commits](https://github.com/privatenumber/tsx/compare/v4.19.2...v4.19.3)

---
updated-dependencies:
- dependency-name: esbuild
  dependency-version: 0.25.0
  dependency-type: indirect
- dependency-name: tsx
  dependency-version: 4.19.3
  dependency-type: direct:development
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-15 12:48:55 +03:00
Tamir Bahar
64760ffa76
fix(web): fix typing for Parser.init
Since we're usually only providing `locateFile`, we need the type to be
`Partial<>` to allow it.

This also matches the typing in `@types/emscripten`'s
`EmscriptenModuleFactory` type signature.
2025-06-15 12:41:21 +03:00
Andrew Dupont
02fff92b91 Fix type definitions in exports
…and generate `.d.cts` files for CommonJS exports.
2025-06-15 09:46:05 +03:00
James McCoy
889015f03b
build(rust): use $CARGO_PKG_RUST_VERSION when generating bindings (#4512)
Since cargo 1.63, $CARGO_PKG_RUST_VERSION is set in the build
environment to the value of the rust-version Cargo.toml field.

This removes the need to manually invoke cargo from build.rs during a
build of the tree-sitter crate with the bindgen feature enabled.

Removing the cargo invocation also ensures the build doesn't write to
the current directory when the target directory has been redirected
elsewhere. "cargo metadata" will attempt to update Cargo.lock, which
will fail if the source tree is read-only.
2025-06-13 11:11:22 -07:00
dependabot[bot]
46309a1f95 build(deps): bump the cargo group with 8 updates
Bumps the cargo group with 8 updates:

| Package | From | To |
| --- | --- | --- |
| [anstyle](https://github.com/rust-cli/anstyle) | `1.0.10` | `1.0.11` |
| [cc](https://github.com/rust-lang/cc-rs) | `1.2.25` | `1.2.26` |
| [clap](https://github.com/clap-rs/clap) | `4.5.39` | `4.5.40` |
| [clap_complete](https://github.com/clap-rs/clap) | `4.5.52` | `4.5.54` |
| [clap_complete_nushell](https://github.com/clap-rs/clap) | `4.5.6` | `4.5.7` |
| [flate2](https://github.com/rust-lang/flate2-rs) | `1.1.1` | `1.1.2` |
| [toml](https://github.com/toml-rs/toml) | `0.8.22` | `0.8.23` |
| [bindgen](https://github.com/rust-lang/rust-bindgen) | `0.71.1` | `0.72.0` |


Updates `anstyle` from 1.0.10 to 1.0.11
- [Commits](https://github.com/rust-cli/anstyle/compare/v1.0.10...v1.0.11)

Updates `cc` from 1.2.25 to 1.2.26
- [Release notes](https://github.com/rust-lang/cc-rs/releases)
- [Changelog](https://github.com/rust-lang/cc-rs/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/cc-rs/compare/cc-v1.2.25...cc-v1.2.26)

Updates `clap` from 4.5.39 to 4.5.40
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.39...clap_complete-v4.5.40)

Updates `clap_complete` from 4.5.52 to 4.5.54
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete-v4.5.52...clap_complete-v4.5.54)

Updates `clap_complete_nushell` from 4.5.6 to 4.5.7
- [Release notes](https://github.com/clap-rs/clap/releases)
- [Changelog](https://github.com/clap-rs/clap/blob/master/CHANGELOG.md)
- [Commits](https://github.com/clap-rs/clap/compare/clap_complete_nushell-v4.5.6...clap_complete_nushell-v4.5.7)

Updates `flate2` from 1.1.1 to 1.1.2
- [Release notes](https://github.com/rust-lang/flate2-rs/releases)
- [Commits](https://github.com/rust-lang/flate2-rs/compare/1.1.1...1.1.2)

Updates `toml` from 0.8.22 to 0.8.23
- [Commits](https://github.com/toml-rs/toml/compare/toml-v0.8.22...toml-v0.8.23)

Updates `bindgen` from 0.71.1 to 0.72.0
- [Release notes](https://github.com/rust-lang/rust-bindgen/releases)
- [Changelog](https://github.com/rust-lang/rust-bindgen/blob/main/CHANGELOG.md)
- [Commits](https://github.com/rust-lang/rust-bindgen/compare/v0.71.1...v0.72.0)

---
updated-dependencies:
- dependency-name: anstyle
  dependency-version: 1.0.11
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: cc
  dependency-version: 1.2.26
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: clap
  dependency-version: 4.5.40
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: clap_complete
  dependency-version: 4.5.54
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: clap_complete_nushell
  dependency-version: 4.5.7
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: flate2
  dependency-version: 1.1.2
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: toml
  dependency-version: 0.8.23
  dependency-type: direct:production
  update-type: version-update:semver-patch
  dependency-group: cargo
- dependency-name: bindgen
  dependency-version: 0.72.0
  dependency-type: direct:production
  update-type: version-update:semver-minor
  dependency-group: cargo
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-06-10 09:10:17 +02:00
Max Brunsfeld
0fdf569571
Reorganize rust crates into a flat crates directory, simplify some CI steps (#4496)
* Move all rust crates (except lib) into crates dir, w/o nesting

* Remove stale path from .gitattributes

* Rename lib.rs files for easier navigation

* Rename mod.rs file for easier navigation

* Fix emscripten-version path

* Fix fixtures dir paths

* Use the default rustfmt settings

* Don't use nightly on CI
2025-06-06 14:25:37 -07:00
Riley Bruins
6cabd9e67f fix(query)!: assert that predicates end in ! or ?
Predicates/directives are documented to end in either `!` or `?`.
However, `query.c` allows them to be any valid identifier, and also
allows `?` or `!` characters anywhere inside an identifier.

This commit removes `?` and `!` from the list of valid identifier
characters, and asserts that predicates/directives only *end* in `?` or
`!`, respectively.

This commit is breaking because you can no longer do something like
`(#eq? @capture foo!bar)` (`foo!bar` must now be quoted).
2025-06-06 10:34:00 +02:00
Will Lillis
8bd923ab9e fix(lib): replace raw array accesses with array_get 2025-06-05 00:53:11 -04:00
Max Brunsfeld
2ab9c9b590
Fully fix field underflow in go_to_previous_sibling (#4483)
Co-authored-by: Conrad Irwin <conrad.irwin@gmail.com>
2025-06-02 15:34:25 -07:00
Kai Pastor
0bdf698673
fix(build): pkgconfig and use of GNUInstallDirs (#4319)
* Fix pkgconfig

Init CMAKE_INSTALL_INCLUDEDIR before pc file generation.
Install pc file to CMAKE_INSTALL_LIBDIR/pkgconfig -
it accompanies the architecture-dependent library.

* Include GNUInstallDirs early

The CMake module initializes variables which are used for
exported information (CMake and pkgconfig).

* Change pc file install destination
2025-05-31 11:45:35 +02:00
Max Brunsfeld
f91255a201
Fix crash w/ goto_previous_sibling when parent node has leading extra child (#4472)
* Fix crash w/ goto_previous_sibling when parent node has leading extra
child Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>

Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>

* Fix lint

Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>

---------

Co-authored-by: Smit Barmase <heysmitbarmase@gmail.com>
2025-05-27 16:56:33 -07:00
Haoxiang Fei
06537fda83 fix: wasi has endian.h 2025-05-24 12:27:13 +02:00
Mike Zeller
4339b0fe05 illumos has endian.h 2025-05-15 09:53:45 +02:00
Will Lillis
31b9717ca3 fix(lib): return early for empty predicate step slice 2025-05-11 08:57:30 -04:00
Will Lillis
b1d2b7cfb8 fix(query): correct last_child_step_index in cases where a new step
wasn't created.

This fixes an OOB access to `self.steps` when a last child anchor
immediately follows a predicate.
2025-05-03 17:27:37 -04:00
dependabot[bot]
bfc5d1180c build(deps): bump vite from 6.3.2 to 6.3.4 in /lib/binding_web
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 6.3.2 to 6.3.4.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/main/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.3.4/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.3.4
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-05-01 09:18:47 +02:00
Amaan Qureshi
21c658a12c fix(lib): do not access the alias sequence for the end subtree in ts_subtree_summarize_children 2025-04-28 23:13:13 -04:00
Amaan Qureshi
0191e94226 docs(rust): remove adding cc as a build dependency for consumers 2025-04-28 18:30:45 -04:00
WillLillis
eee41925aa fix(ci): increase timeouts for flaky tests 2025-04-19 10:59:09 +02:00
dependabot[bot]
043e0c65ec build(deps): bump vite from 6.1.0 to 6.2.6 in /lib/binding_web
Bumps [vite](https://github.com/vitejs/vite/tree/HEAD/packages/vite) from 6.1.0 to 6.2.6.
- [Release notes](https://github.com/vitejs/vite/releases)
- [Changelog](https://github.com/vitejs/vite/blob/v6.2.6/packages/vite/CHANGELOG.md)
- [Commits](https://github.com/vitejs/vite/commits/v6.2.6/packages/vite)

---
updated-dependencies:
- dependency-name: vite
  dependency-version: 6.2.6
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2025-04-18 23:57:56 -04:00
vemoo
4dffb818e2
feat(web): export wasm files to better support bundling use cases 2025-04-18 23:47:27 -04:00
Tamir Bahar
27fa1088b9
fix(web): replace dynamic require with import 2025-04-18 23:40:40 -04:00
Riley Bruins
21390af2dd fix(web): correct childWithDescendant() functionality
This fix allows for more granular address control when marshalling nodes
across WASM. This is necessary for node methods which accept another
node as a parameter (i.e., `childWithDescendant()`)
2025-04-18 18:11:42 -04:00
Will Lillis
9fdf685dd5 fix(bindings): report wasm store error in rust bindings set_language 2025-04-14 22:15:40 -04:00
Riley Bruins
733d7513af fix(lib): reset parser options after use
**Problem:** After `ts_parser_parse_with_options()`, the parser options
are still stored in the parser object, meaning that a successive call to
`ts_parser_parse()` will actually behave like
`ts_parser_parse_with_options()`, which is not obvious and can have
unintended consequences.

**Solution:** Reset to empty options state after
`ts_parser_parse_with_options()`.
2025-04-14 21:35:40 -04:00
Jason Boatman
abc5c6bc50
Fix WASI build by not calling a non-existent function. (#4343) 2025-04-08 09:33:48 -07:00
Simon Willshire
ee8d529552
fix(rust): use core crates for no_std
also add `no_std` build to CI
2025-03-25 14:30:32 +01:00
WillLillis
dac6300558 fix(rust): address new clippy lint for pointer comparisons 2025-03-25 13:44:42 +01:00
Tamir Bahar
a380e1a259
refactor(web): change return types to Node[] in child-related methods 2025-03-15 09:09:31 -04:00
Christian Clason
07a1b7fc00 fix(build): bump version to 0.26.0
After a release, `master` branch should be bumped to the next _minor_ version.
2025-03-12 14:52:09 +01:00
Riley Bruins
cc3994928c feat(lib): quote invalid nodes, fields, and captures 2025-03-12 10:28:57 +01:00
June Gschwantner
0b28226615
docs(bindings): Add "Using WASM Grammar files" section to binding_rust (#4235) 2025-03-06 14:15:21 -08:00
NOT XVilka
a00fab7dc4
fix(lib): remove duplicate TSLanguageMetadata typedef (#4268) 2025-03-06 14:14:25 -08:00
WillLillis
11071ed682 fix(rust): adapt to new nightly lint 2025-03-06 09:32:21 +01:00
Max Brunsfeld
066fd77d39
Fix cases where error recovery could infinite loop (#4257)
* Rename corpus test functions to allow easy filtering by language

* Use usize for seed argument

* Avoid retaining useless stack versions when reductions merge

We found this problem when debugging an infinite loop that happened
during error recovery when using the Zig grammar. The large number of
unnecessary paused stack versions were preventing the correct recovery
strategy from being tried.

* Fix leaked lookahead token when reduction results in a merged stack

* Enable running PHP tests in CI

* Fix possible infinite loop during error recovery at EOF

* Account for external scanner state changes when detecting changed ranges in subtrees
2025-03-04 13:50:56 -08:00
WillLillis
998fb34d15 fix(test): update expected tree-sitter-rust supertypes 2025-03-02 16:37:03 -05:00
Will Lillis
3b67861def
fix(wasm): restore passing in ERROR to descendantsOfType (#4226) 2025-02-19 14:47:10 -08:00
Max Brunsfeld
b26b7f8d62
Decrease the MSRV for the tree-sitter-language crate (#4221) 2025-02-17 18:52:57 -08:00
Max Brunsfeld
2bd400dcee
Reset result_symbol field of lexer in wasm memory in between invocations (#4218) 2025-02-17 17:36:46 -08:00
Max Brunsfeld
dedcc5255a
Ignore external tokens that are zero-length and extra (#4213)
Co-authored-by: Anthony <anthony@zed.dev>
2025-02-17 15:07:44 -08:00
Max Brunsfeld
14b8ead412
Fix crash when loading languages w/ old ABI via wasm (#4210) 2025-02-17 13:56:53 -08:00
Amaan Qureshi
a40265cbeb refactor(web): rename tree-sitter.js to web-tree-sitter.js
This is not breaking for consumers of the web bindings, nor the
playground as both filenames will be kept in the .github.io repo
2025-02-11 22:56:50 -05:00
Roberto Huertas
f95e0e3a56 fix(web): provide type in the exports
When using TypeScript projects using other module settings than CommonJs, the types were not correctly exposed, and the compilation failed.

This adds the types path to the exports so compilation works for `module: NodeNext` and other variants.
2025-02-08 12:47:17 -05:00