Commit graph

512 commits

Author SHA1 Message Date
Amaan Qureshi
5032b341a7
fix(lib): set the error cost of children that are terminal error nodes, as they have not been set yet 2024-09-27 16:10:07 -04:00
Amaan Qureshi
50eaf0b6cd style(lib): add parameter names in declarations that are missing them 2024-09-27 13:38:08 -04:00
Amaan Qureshi
28972f916a fix(lib): silence warnings with -Wpedantic 2024-09-27 12:58:26 -04:00
Amaan Qureshi
0c43988a5e fix(lib): correct descendant-for-range behavior with zero-width tokens 2024-09-27 00:37:13 -04:00
Amaan Qureshi
0a85744eba fix(lib): peek at the next sibling when iterating to find the child that contains a given descendant
This issue shows up when we have a zero-width token that is the target
descendant node, previously the previous sibling would be returned as
the child that contains the descendant, which is incorrect.
2024-09-17 04:37:43 +02:00
Amaan Qureshi
7e3f572655 feat: add field_name_for_named_child 2024-09-09 11:05:29 -04:00
Amaan Qureshi
9b398c2b84 fix(lib): backtrack to the last relevant iterator if no child was found 2024-09-05 18:01:29 -04:00
Amaan Qureshi
ee06325f67 fix(lib): correct extra node creation from non-zero root-alias cursors 2024-08-31 23:58:19 -04:00
Amaan Qureshi
3f424c0121 feat: add an API to time out query executions
Currently, if a predicate is hard to match on the Rust side, a sizable
query against a very large file can take forever, and ends up hanging.
This commit adds an API function `ts_query_cursor_set_timeout_micros` to
limit how long query execution is allowed to take, thereby negating the
chance of a hang to occur.
2024-08-31 14:33:28 -04:00
Ryan Patterson
b5e4ef6d9a
clone wasm store engine (#3542)
This resolves https://github.com/tree-sitter/tree-sitter/issues/3454.

This brings the usage of wasmtime::Engine in line with how wasmtime
intends it to be used. All wasmtime functions that receive an Engine
always receive an `&Engine`, never an owned `Engine`.  They are always
responsible for cloning the reference if they need it.

This brings the usage of wasmtime::Engine in line with how TSParser
treats TSLanguages: when setting a language to the parser, the parser is
responsible for cloning the reference to the TSLanguage. It is
counterintuitive for TSParser to have different behavior when receiving
wasmtime_engine_t.

C API users also expect this behavior, see "Memory Management"
[here](https://docs.wasmtime.dev/c-api/wasm_8h.html). Talking about the
C API: without this change, failing to clone the `wasmtime_engine_t`
(which, again, is never something API users need to do in wasmtime) and
then reusing the engine in multiple TSLanguages results in a use after
free. With this change, failing to call `wasm_engine_delete` on your
owned Engine results in a memory leak. Memory leaks are safer than
use-after-free.
2024-08-22 08:01:37 -07:00
Amaan Qureshi
6dd459b4ab fix(lib): an empty root node should not precede an empty range
The problem is, given an empty file, the root node of this file spans 0
bytes. As such, the logic for determining whether or not the node
precedes the range fails, and is true when it should be false.
2024-08-17 17:05:26 -04:00
Ron Panduwana
2bb20fe2fe
feat: allow external scanners to use the logger
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-08-17 14:46:28 -04:00
Ryan Patterson
779566f588
Reset language when resetting wasm store (#3495)
* Reset language when resetting wasm store

* test behavior of language copying
2024-07-31 10:30:58 -07:00
Amaan Qureshi
25c7189180 feat(lib): add ts_query_end_byte_for_pattern 2024-07-07 20:29:09 -04:00
Quentin LE DILAVREC
9610a84600
fix(lib): restrict pattern_map optimization when a wildcard step has an immediate first child
Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
2024-07-05 03:35:24 -04:00
Davis Vaughan
218a071d33 Swap sprintf() for snprintf() 2024-06-24 09:56:24 -04:00
Amaan Qureshi
7f4a57817d build(deps): bump wasmtime to v21 2024-05-29 18:57:26 -04:00
Amaan Qureshi
ae35a36747 fix(lib): a null clock must have tv_nsec be 0 as well 2024-05-19 02:19:37 -04:00
Amaan Qureshi
4c083252ec fix(lib): advance the lookahead end byte by 4 when there's an invalid code point
This helps in the case where an edit was made in the middle of a code
point, but bytes 1-3 are valid, thus we could advance by at most 4 bytes
2024-04-30 20:55:43 -04:00
Amaan Qureshi
61d0395543 fix(lib): do not return field names for extras 2024-04-30 19:23:11 -04:00
Max Brunsfeld
37a919184f Position serialization buffer after language statics in wasm linear memory 2024-04-25 14:21:28 -07:00
Max Brunsfeld
78efd96bbe Crash earlier if native external scanner overflows serialization buffer 2024-04-25 13:55:47 -07:00
Max Brunsfeld
e9fd4ccf0c Guard against serialization buffer overflows from wasm languages 2024-04-25 13:51:05 -07:00
Max Brunsfeld
d7a23cf017 fix: Correctly load field data from wasm languages
Previously, if the last production id in a language did not have
a unique set of fields, the field maps weren't loaded correctly
from wasm.

Co-authored-by: Marshall <marshall@zed.dev>
2024-04-24 16:29:44 -07:00
vanaigr
90e0e28b95
feat: reverse iteration through node parents (#3214) 2024-04-23 15:19:57 +01:00
Max Brunsfeld
7ec40b0ab4 Implement single-char state transitions using a static array and for loop
This reduces compile time, compared to generating many individual if statements.
2024-04-12 14:40:11 -07:00
Max Brunsfeld
be6e6d3708 Merge branch 'master' into simpler-large-char-set-code 2024-04-11 16:03:20 -07:00
ObserverOfTime
818cd8c291
refactor(parser): make REDUCE macro non-variadic 2024-04-11 20:47:08 +03:00
Torsten Schmits
cdd00defdb
fix(lib): use correct format specifier in log message 2024-04-04 14:51:15 -04:00
ObserverOfTime
78b6067a5d fix(parser): fix variadic macro 2024-04-02 03:18:05 -04:00
Max Brunsfeld
39be6972fe Use static arrays and a fixed binary search for large char set checks 2024-03-29 23:00:48 -07:00
Max Brunsfeld
0fc92c9a7d Add strncat to wasm stdlib
Co-authored-by: Marshall <marshall@zed.dev>
2024-03-25 09:51:54 -07:00
Max Brunsfeld
8945e3a111 Fix sorting of wasm stdlib symbols 2024-03-20 18:23:44 -07:00
Amaan Qureshi
09d2b23a64 fix(lib): account for the root node of a tree cursor being an alias 2024-03-20 19:23:08 -04:00
Amaan Qureshi
24a68697ac fix(lib): properly account for aliased root nodes and root nodes with
children in `ts_subtree_string`
2024-03-20 19:23:08 -04:00
Amaan Qureshi
ab485da756 fix(lib): do not consider childless nodes for ts_node_parent 2024-03-20 19:23:08 -04:00
Max Brunsfeld
f415f0c6f2 Add strncmp to wasm stdlib 2024-03-20 15:49:47 -07:00
Max Brunsfeld
09b18fad5b
Merge pull request #3181 from tree-sitter/handle-wasm-oom
When loading languages via WASM, gracefully handle memory errors and leaks in external scanners
2024-03-18 13:15:06 -07:00
Max Brunsfeld
4676cd4200 Defer creation of external scanner until the beginning of parse 2024-03-17 10:20:25 -07:00
Max Brunsfeld
fed9b98d6c Clean up wasm store construction, avoid hard-coding initial memory size
Retrieve the initial memory size from the stdlib's memory import type.
2024-03-17 10:20:18 -07:00
Max Brunsfeld
7a9b3076ef Handle memory errors occurring in wasm scanners
* In WASM, use a custom, simple malloc implementation that lets us
  expicitly reset the heap with a new start location.
* When a WASM call traps or errors, propagate that as a parse failure.
* Reset the WASM heap after every parse.

Co-authored-by: Conrad <conrad@zed.dev>
2024-03-17 10:19:42 -07:00
ObserverOfTime
4bbaee2f56 fix(lib): allow hiding symbols 2024-03-17 07:21:06 -04:00
Amaan Qureshi
30fd71f5ac fix(lib): avoid possible UB of calling memset on a null ptr when 0 is passed into array_grow_by 2024-03-13 00:02:53 -04:00
Amaan Qureshi
304f8b7c04 fix: don't use __declspec(dllexport) on windows 2024-03-04 13:23:06 -05:00
geekvest
b43e8ce902
docs: remove duplicate the's
Signed-off-by: geekvest <cuimoman@sohu.com>
2024-03-03 10:30:21 -05:00
Daumantas Kavolis
ce5afeb4b3
Fix regression in subtree_compare 2024-02-29 10:15:58 +02:00
ObserverOfTime
22c85c6139 refactor(array): use pragma GCC in clang too 2024-02-29 01:50:04 -05:00
ObserverOfTime
799833f9cf build: use c11 everywhere
And improve the makefiles
2024-02-27 15:54:38 -05:00
Amaan Qureshi
e75a36232b fix(lib): include api.h so ts_set_allocator is visible 2024-02-27 09:21:17 -05:00
Max Brunsfeld
9b485924d6 Fix invalid parse stack recursive merging with mismatched error cost
Allowing this invalid merge caused an invariant to be violated
later on during parsing, when handling a later error.

Co-authored-by: Amaan Qureshi <amaanq12@gmail.com>
Co-authored-by: Marshall <marshall@zed.dev>
2024-02-26 20:16:21 -05:00