Commit graph

494 commits

Author SHA1 Message Date
Amaan Qureshi
538a197976 fix(lib): correct unexpected side effect in get_column when the lexer is at EOF 2024-10-08 23:27:42 -04:00
Amaan Qureshi
9c08edb066 build: configure clippy lints at the workspace level 2024-10-08 21:07:41 -04:00
Will Lillis
5c6445edea
chore: misc clippy lints 2024-10-06 17:55:00 -04:00
Amaan Qureshi
8943983df6 feat!: properly handle UTF-16 endianness encoding 2024-10-05 21:12:48 -04:00
Amaan Qureshi
c611e15a31 chore(cli): minor correction in comments 2024-10-05 01:41:05 -04:00
Riley Bruins
0683136ca0 feat(api): expose function to check if symbol represents a supertype 2024-09-30 14:44:13 -04:00
Amaan Qureshi
934a2814fd fix: deprecate child_containing_descendant and add child_with_descendant instead 2024-09-30 13:57:28 -04:00
Amaan Qureshi
ea3846a2c5
feat: move tree-sitter configuration to dedicated file (#3700) 2024-09-30 11:11:23 -04:00
Ron Panduwana
a83b893016 fix: handle more cases of editing subtrees that depend on column values 2024-09-29 20:36:08 -04:00
Jeong, Hun
bbc1370dd5 feat(lib)!: treat nodes' end ranges exclusively in goto_first_child_for_{byte,point}
This goes back on #1640, because now cursors are bi-directional, and
going to the previous sibling is simple.
2024-09-29 17:45:07 -04:00
Riley Bruins
b36ef4b7f4 fix(lib)!: child_containing_descendant now returns direct children
Previously, `child_containing_descendant` would return `null` when
called on a node's direct parent. In my opinion, this doesn't make much
sense; it seems like a node would contain itself. This (breaking)
commit changes the function so that it can return direct children.
2024-09-29 17:44:37 -04:00
Lukas Seidel
6b1ebd3d29
feat!: implement StreamingIterator instead of Iterator for QueryMatches and QueryCaptures
This fixes UB when either `QueryMatches` or `QueryCaptures` had collect called on it.

Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-09-29 17:34:48 -04:00
Amaan Qureshi
871e8966c5 build: bump deps 2024-09-28 13:58:58 -04:00
Amaan Qureshi
31f24395b4 feat: move generate logic to its own crate 2024-09-27 17:04:21 -04:00
Amaan Qureshi
90efa34608 chore: clippy fixes 2024-09-27 16:32:48 -04:00
Amaan Qureshi
b2359e4020 feat!: move generation of grammar files to an init command
The generate subcommand should stick to solely generating a parser and
headers.
2024-09-27 16:06:57 -04:00
Amaan Qureshi
0c43988a5e fix(lib): correct descendant-for-range behavior with zero-width tokens 2024-09-27 00:37:13 -04:00
Samuel Tardieu
1aa28e04ee style(tests): do not use .as_bytes().len() on strings 2024-09-22 20:27:38 -04:00
Amaan Qureshi
6f050f0da5 fix: properly handle utf8 code points for highlight and tag assertions 2024-09-22 01:03:59 -04:00
Amaan Qureshi
0a85744eba fix(lib): peek at the next sibling when iterating to find the child that contains a given descendant
This issue shows up when we have a zero-width token that is the target
descendant node, previously the previous sibling would be returned as
the child that contains the descendant, which is incorrect.
2024-09-17 04:37:43 +02:00
Amaan Qureshi
7e3f572655 feat: add field_name_for_named_child 2024-09-09 11:05:29 -04:00
Amaan Qureshi
10e474f488 feat!: remove filter flag from commands in favor of include and exclude 2024-09-07 20:53:04 -04:00
Amaan Qureshi
ee06325f67 fix(lib): correct extra node creation from non-zero root-alias cursors 2024-08-31 23:58:19 -04:00
Amaan Qureshi
3f424c0121 feat: add an API to time out query executions
Currently, if a predicate is hard to match on the Rust side, a sizable
query against a very large file can take forever, and ends up hanging.
This commit adds an API function `ts_query_cursor_set_timeout_micros` to
limit how long query execution is allowed to take, thereby negating the
chance of a hang to occur.
2024-08-31 14:33:28 -04:00
Amaan Qureshi
6ef76858c0 fix!: revert interning of a sequence or choice of a single rule
This reverts commit 252e2a4bc0
2024-08-24 19:43:04 -04:00
Ryan Patterson
b5e4ef6d9a
clone wasm store engine (#3542)
This resolves https://github.com/tree-sitter/tree-sitter/issues/3454.

This brings the usage of wasmtime::Engine in line with how wasmtime
intends it to be used. All wasmtime functions that receive an Engine
always receive an `&Engine`, never an owned `Engine`.  They are always
responsible for cloning the reference if they need it.

This brings the usage of wasmtime::Engine in line with how TSParser
treats TSLanguages: when setting a language to the parser, the parser is
responsible for cloning the reference to the TSLanguage. It is
counterintuitive for TSParser to have different behavior when receiving
wasmtime_engine_t.

C API users also expect this behavior, see "Memory Management"
[here](https://docs.wasmtime.dev/c-api/wasm_8h.html). Talking about the
C API: without this change, failing to clone the `wasmtime_engine_t`
(which, again, is never something API users need to do in wasmtime) and
then reusing the engine in multiple TSLanguages results in a use after
free. With this change, failing to call `wasm_engine_delete` on your
owned Engine results in a memory leak. Memory leaks are safer than
use-after-free.
2024-08-22 08:01:37 -07:00
Amaan Qureshi
6dd459b4ab fix(lib): an empty root node should not precede an empty range
The problem is, given an empty file, the root node of this file spans 0
bytes. As such, the logic for determining whether or not the node
precedes the range fails, and is true when it should be false.
2024-08-17 17:05:26 -04:00
Ron Panduwana
2bb20fe2fe
feat: allow external scanners to use the logger
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-08-17 14:46:28 -04:00
Ryan Patterson
779566f588
Reset language when resetting wasm store (#3495)
* Reset language when resetting wasm store

* test behavior of language copying
2024-07-31 10:30:58 -07:00
ObserverOfTime
3950dddfde fix(rust): fix new clippy warnings 2024-07-28 10:12:32 +03:00
Amaan Qureshi
25c7189180 feat(lib): add ts_query_end_byte_for_pattern 2024-07-07 20:29:09 -04:00
Quentin LE DILAVREC
9610a84600
fix(lib): restrict pattern_map optimization when a wildcard step has an immediate first child
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-07-05 03:35:24 -04:00
Amaan Qureshi
252e2a4bc0 fix: intern a sequence or choice of a single element the same as the element itself 2024-06-21 20:20:02 -04:00
Amaan Qureshi
e553578696 feat: add fuzz subcommand 2024-05-30 23:00:45 -04:00
Amaan Qureshi
8e8648afa9 fix(test): multi-grammar corpus tests are now in the repo root 2024-05-06 15:25:42 -04:00
ObserverOfTime
572cdaa7c0 ci: fix address sanitizer step 2024-05-05 13:06:45 -04:00
Amaan Qureshi
67a043911c
test: temporarily disable php test 2024-05-04 03:05:58 -04:00
Amaan Qureshi
4c083252ec fix(lib): advance the lookahead end byte by 4 when there's an invalid code point
This helps in the case where an edit was made in the middle of a code
point, but bytes 1-3 are valid, thus we could advance by at most 4 bytes
2024-04-30 20:55:43 -04:00
Amaan Qureshi
61d0395543 fix(lib): do not return field names for extras 2024-04-30 19:23:11 -04:00
WillLillis
946acfd70f feat(cli): support NO_COLOR
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>

This allows users to avoid colored output when NO_COLOR is set to 1.
2024-04-29 20:25:41 -04:00
Amaan Qureshi
627617edb4 refactor(tests): migrate remaining grammar.json tests to grammar.js 2024-04-29 00:01:35 -04:00
vanaigr
90e0e28b95
feat: reverse iteration through node parents (#3214) 2024-04-23 15:19:57 +01:00
Amaan Qureshi
abc7910381 refactor(rust): misc fixes & tidying 2024-04-11 22:35:43 -04:00
Amaan Qureshi
5825e24d56 style: wrap comments 2024-04-11 22:35:43 -04:00
Amaan Qureshi
b35efa8f33 style: format imports 2024-04-11 22:35:43 -04:00
Amaan Qureshi
62cd13a3de test: add regression test for node parent + string bug 2024-03-20 19:23:08 -04:00
Max Brunsfeld
09b18fad5b
Merge pull request #3181 from tree-sitter/handle-wasm-oom
When loading languages via WASM, gracefully handle memory errors and leaks in external scanners
2024-03-18 13:15:06 -07:00
Max Brunsfeld
fed9b98d6c Clean up wasm store construction, avoid hard-coding initial memory size
Retrieve the initial memory size from the stdlib's memory import type.
2024-03-17 10:20:18 -07:00
Max Brunsfeld
7a9b3076ef Handle memory errors occurring in wasm scanners
* In WASM, use a custom, simple malloc implementation that lets us
  expicitly reset the heap with a new start location.
* When a WASM call traps or errors, propagate that as a parse failure.
* Reset the WASM heap after every parse.

Co-authored-by: Conrad <conrad@zed.dev>
2024-03-17 10:19:42 -07:00
Amaan Qureshi
e8dbe011d3 test: fix header writes 2024-03-17 05:36:30 -04:00