Commit graph

509 commits

Author SHA1 Message Date
Yuta Saito
bf094bd98a fix: exclude APIs that dup given file descriptors from WASI builds
WASI doesn't support `dup(2)` system call, so we cannot implement the
`print_dot_graph` and `print_dot_graphs` functions with exactly the same
semantics as in other platforms.

(cherry picked from commit 94a8262110)
2024-09-30 09:21:54 -04:00
Ron Panduwana
d10308528d fix: handle more cases of editing subtrees that depend on column values
(cherry picked from commit a83b893016)
2024-09-29 20:37:24 -04:00
Amaan Qureshi
865f6595e7 fix(lib): correct descendant-for-range behavior with zero-width tokens
(cherry picked from commit 0c43988a5e)
2024-09-27 09:21:17 -04:00
Amaan Qureshi
a6b248c1ad fix(lib): peek at the next sibling when iterating to find the child that contains a given descendant
This issue shows up when we have a zero-width token that is the target
descendant node, previously the previous sibling would be returned as
the child that contains the descendant, which is incorrect.

(cherry picked from commit 0a85744eba)
2024-09-17 17:22:13 +02:00
Amaan Qureshi
43e16dd75c fix(lib): backtrack to the last relevant iterator if no child was found
(cherry picked from commit 9b398c2b84)
2024-09-05 18:03:50 -04:00
Amaan Qureshi
1a3f486059 fix(lib): correct extra node creation from non-zero root-alias cursors
(cherry picked from commit ee06325f67)
2024-09-01 16:21:33 -04:00
Ryan Patterson
b5e4ef6d9a
clone wasm store engine (#3542)
This resolves https://github.com/tree-sitter/tree-sitter/issues/3454.

This brings the usage of wasmtime::Engine in line with how wasmtime
intends it to be used. All wasmtime functions that receive an Engine
always receive an `&Engine`, never an owned `Engine`.  They are always
responsible for cloning the reference if they need it.

This brings the usage of wasmtime::Engine in line with how TSParser
treats TSLanguages: when setting a language to the parser, the parser is
responsible for cloning the reference to the TSLanguage. It is
counterintuitive for TSParser to have different behavior when receiving
wasmtime_engine_t.

C API users also expect this behavior, see "Memory Management"
[here](https://docs.wasmtime.dev/c-api/wasm_8h.html). Talking about the
C API: without this change, failing to clone the `wasmtime_engine_t`
(which, again, is never something API users need to do in wasmtime) and
then reusing the engine in multiple TSLanguages results in a use after
free. With this change, failing to call `wasm_engine_delete` on your
owned Engine results in a memory leak. Memory leaks are safer than
use-after-free.
2024-08-22 08:01:37 -07:00
Amaan Qureshi
6dd459b4ab fix(lib): an empty root node should not precede an empty range
The problem is, given an empty file, the root node of this file spans 0
bytes. As such, the logic for determining whether or not the node
precedes the range fails, and is true when it should be false.
2024-08-17 17:05:26 -04:00
Ron Panduwana
2bb20fe2fe
feat: allow external scanners to use the logger
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-08-17 14:46:28 -04:00
Ryan Patterson
779566f588
Reset language when resetting wasm store (#3495)
* Reset language when resetting wasm store

* test behavior of language copying
2024-07-31 10:30:58 -07:00
Amaan Qureshi
25c7189180 feat(lib): add ts_query_end_byte_for_pattern 2024-07-07 20:29:09 -04:00
Quentin LE DILAVREC
9610a84600
fix(lib): restrict pattern_map optimization when a wildcard step has an immediate first child
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-07-05 03:35:24 -04:00
Davis Vaughan
218a071d33 Swap sprintf() for snprintf() 2024-06-24 09:56:24 -04:00
Amaan Qureshi
7f4a57817d build(deps): bump wasmtime to v21 2024-05-29 18:57:26 -04:00
Amaan Qureshi
ae35a36747 fix(lib): a null clock must have tv_nsec be 0 as well 2024-05-19 02:19:37 -04:00
Amaan Qureshi
4c083252ec fix(lib): advance the lookahead end byte by 4 when there's an invalid code point
This helps in the case where an edit was made in the middle of a code
point, but bytes 1-3 are valid, thus we could advance by at most 4 bytes
2024-04-30 20:55:43 -04:00
Amaan Qureshi
61d0395543 fix(lib): do not return field names for extras 2024-04-30 19:23:11 -04:00
Max Brunsfeld
37a919184f Position serialization buffer after language statics in wasm linear memory 2024-04-25 14:21:28 -07:00
Max Brunsfeld
78efd96bbe Crash earlier if native external scanner overflows serialization buffer 2024-04-25 13:55:47 -07:00
Max Brunsfeld
e9fd4ccf0c Guard against serialization buffer overflows from wasm languages 2024-04-25 13:51:05 -07:00
Max Brunsfeld
d7a23cf017 fix: Correctly load field data from wasm languages
Previously, if the last production id in a language did not have
a unique set of fields, the field maps weren't loaded correctly
from wasm.

Co-authored-by: Marshall <marshall@zed.dev>
2024-04-24 16:29:44 -07:00
vanaigr
90e0e28b95
feat: reverse iteration through node parents (#3214) 2024-04-23 15:19:57 +01:00
Max Brunsfeld
7ec40b0ab4 Implement single-char state transitions using a static array and for loop
This reduces compile time, compared to generating many individual if statements.
2024-04-12 14:40:11 -07:00
Max Brunsfeld
be6e6d3708 Merge branch 'master' into simpler-large-char-set-code 2024-04-11 16:03:20 -07:00
ObserverOfTime
818cd8c291
refactor(parser): make REDUCE macro non-variadic 2024-04-11 20:47:08 +03:00
Torsten Schmits
cdd00defdb
fix(lib): use correct format specifier in log message 2024-04-04 14:51:15 -04:00
ObserverOfTime
78b6067a5d fix(parser): fix variadic macro 2024-04-02 03:18:05 -04:00
Max Brunsfeld
39be6972fe Use static arrays and a fixed binary search for large char set checks 2024-03-29 23:00:48 -07:00
Max Brunsfeld
0fc92c9a7d Add strncat to wasm stdlib
Co-authored-by: Marshall <marshall@zed.dev>
2024-03-25 09:51:54 -07:00
Max Brunsfeld
8945e3a111 Fix sorting of wasm stdlib symbols 2024-03-20 18:23:44 -07:00
Amaan Qureshi
09d2b23a64 fix(lib): account for the root node of a tree cursor being an alias 2024-03-20 19:23:08 -04:00
Amaan Qureshi
24a68697ac fix(lib): properly account for aliased root nodes and root nodes with
children in `ts_subtree_string`
2024-03-20 19:23:08 -04:00
Amaan Qureshi
ab485da756 fix(lib): do not consider childless nodes for ts_node_parent 2024-03-20 19:23:08 -04:00
Max Brunsfeld
f415f0c6f2 Add strncmp to wasm stdlib 2024-03-20 15:49:47 -07:00
Max Brunsfeld
09b18fad5b
Merge pull request #3181 from tree-sitter/handle-wasm-oom
When loading languages via WASM, gracefully handle memory errors and leaks in external scanners
2024-03-18 13:15:06 -07:00
Max Brunsfeld
4676cd4200 Defer creation of external scanner until the beginning of parse 2024-03-17 10:20:25 -07:00
Max Brunsfeld
fed9b98d6c Clean up wasm store construction, avoid hard-coding initial memory size
Retrieve the initial memory size from the stdlib's memory import type.
2024-03-17 10:20:18 -07:00
Max Brunsfeld
7a9b3076ef Handle memory errors occurring in wasm scanners
* In WASM, use a custom, simple malloc implementation that lets us
  expicitly reset the heap with a new start location.
* When a WASM call traps or errors, propagate that as a parse failure.
* Reset the WASM heap after every parse.

Co-authored-by: Conrad <conrad@zed.dev>
2024-03-17 10:19:42 -07:00
ObserverOfTime
4bbaee2f56 fix(lib): allow hiding symbols 2024-03-17 07:21:06 -04:00
Amaan Qureshi
30fd71f5ac fix(lib): avoid possible UB of calling memset on a null ptr when 0 is passed into array_grow_by 2024-03-13 00:02:53 -04:00
Amaan Qureshi
304f8b7c04 fix: don't use __declspec(dllexport) on windows 2024-03-04 13:23:06 -05:00
geekvest
b43e8ce902
docs: remove duplicate the's
Signed-off-by: geekvest <cuimoman@sohu.com>
2024-03-03 10:30:21 -05:00
Daumantas Kavolis
ce5afeb4b3
Fix regression in subtree_compare 2024-02-29 10:15:58 +02:00
ObserverOfTime
22c85c6139 refactor(array): use pragma GCC in clang too 2024-02-29 01:50:04 -05:00
ObserverOfTime
799833f9cf build: use c11 everywhere
And improve the makefiles
2024-02-27 15:54:38 -05:00
Amaan Qureshi
e75a36232b fix(lib): include api.h so ts_set_allocator is visible 2024-02-27 09:21:17 -05:00
Max Brunsfeld
9b485924d6 Fix invalid parse stack recursive merging with mismatched error cost
Allowing this invalid merge caused an invariant to be violated
later on during parsing, when handling a later error.

Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
Co-authored-by: Marshall <marshall@zed.dev>
2024-02-26 20:16:21 -05:00
Max Brunsfeld
527848d7c0
Merge pull request #3084 from tree-sitter/no-vendored-wasmtime-headers
Remove vendored wasmtime headers
2024-02-26 15:36:10 -08:00
Max Brunsfeld
dfc6d93bae Rename wasm C sources to avoid conflicts with standard wasm headers
Co-authored-by: Marshall <marshall@zed.dev>
2024-02-26 13:08:30 -08:00
Amaan Qureshi
a31f084b1b fix: ignore unused variables in the array macros 2024-02-26 14:05:16 -05:00