Commit graph

95 commits

Author SHA1 Message Date
Amaan Qureshi
1a99bfd9ff
wow 2024-09-26 17:51:46 -04:00
Amaan Qureshi
3f424c0121 feat: add an API to time out query executions
Currently, if a predicate is hard to match on the Rust side, a sizable
query against a very large file can take forever, and ends up hanging.
This commit adds an API function `ts_query_cursor_set_timeout_micros` to
limit how long query execution is allowed to take, thereby negating the
chance of a hang to occur.
2024-08-31 14:33:28 -04:00
Ryan Patterson
779566f588
Reset language when resetting wasm store (#3495)
* Reset language when resetting wasm store

* test behavior of language copying
2024-07-31 10:30:58 -07:00
Davis Vaughan
218a071d33 Swap sprintf() for snprintf() 2024-06-24 09:56:24 -04:00
Max Brunsfeld
78efd96bbe Crash earlier if native external scanner overflows serialization buffer 2024-04-25 13:55:47 -07:00
Torsten Schmits
cdd00defdb
fix(lib): use correct format specifier in log message 2024-04-04 14:51:15 -04:00
Max Brunsfeld
4676cd4200 Defer creation of external scanner until the beginning of parse 2024-03-17 10:20:25 -07:00
Max Brunsfeld
7a9b3076ef Handle memory errors occurring in wasm scanners
* In WASM, use a custom, simple malloc implementation that lets us
  expicitly reset the heap with a new start location.
* When a WASM call traps or errors, propagate that as a parse failure.
* Reset the WASM heap after every parse.

Co-authored-by: Conrad <conrad@zed.dev>
2024-03-17 10:19:42 -07:00
geekvest
b43e8ce902
docs: remove duplicate the's
Signed-off-by: geekvest <cuimoman@sohu.com>
2024-03-03 10:30:21 -05:00
ObserverOfTime
799833f9cf build: use c11 everywhere
And improve the makefiles
2024-02-27 15:54:38 -05:00
Max Brunsfeld
dfc6d93bae Rename wasm C sources to avoid conflicts with standard wasm headers
Co-authored-by: Marshall <marshall@zed.dev>
2024-02-26 13:08:30 -08:00
Amaan Qureshi
037c71c7bd fix: apply some scan-build suggestions (unused assignment/garbage access) 2024-02-21 15:38:13 -05:00
dundargoc
ce191e02c5 fix: use correct printf specifiers 2024-02-08 22:46:43 +01:00
dundargoc
df1fe842eb docs: various fixes
Closes https://github.com/tree-sitter/tree-sitter/issues/1317.
Closes https://github.com/tree-sitter/tree-sitter/issues/1752.
Closes https://github.com/tree-sitter/tree-sitter/issues/2439.

Co-authored-by: Simon Hengel <sol@typeful.net>
Co-authored-by: Akash Yadav <itsaky01@gmail.com>
Co-authored-by: Sebastiaan Speck <12570668+sebastiaanspeck@users.noreply.github.com>
Co-authored-by: Laytan Laats <laytanlaats@hotmail.com>
2024-02-08 00:08:59 +01:00
Max Brunsfeld
4a8e4b1963 Allow wasm languages to be deleted 2023-12-27 15:00:16 -08:00
Max Brunsfeld
034f0d0280
Merge pull request #1864 from tree-sitter/wasm-language
Add optional WASM feature to the native library, allowing it to run wasm-compiled parsers via wasmtime
2023-11-28 12:08:47 -08:00
Max Brunsfeld
5e2456c2f2 Avoid using recursion for ts_subtree_compare
This can lead to stack overflow crashes.
2023-11-27 11:35:33 -08:00
Max Brunsfeld
f4e2f68f14 Merge branch 'master' into wasm-language 2023-10-27 12:11:43 +01:00
Amaan Qureshi
05a4f9f3e3
Merge pull request #1411 from aminya/c-warnings
Fix compiling C library with Clang on Windows
2023-08-03 13:39:52 -04:00
Amaan Qureshi
13f6ec2b0c
fix: rename shadowed variables from -Wshadow warnings and apply some useful clang-tidy warnings 2023-07-19 18:12:26 -04:00
Amaan Qureshi
753fa1c3ff
fix(lib): explicitly cast numbers to the same size in potential spots for infinite loops 2023-07-19 03:49:14 -04:00
Max Brunsfeld
0b817a609f Clear the parse stack when terminating parsing early due to error cost
This fixes a bug where the parse tree would not be rebalanced if this
code path was taken.
2023-02-13 13:45:12 -08:00
Max Brunsfeld
f7c9989b48 Fix crash when parser's wasm store is taken before its language is changed 2023-01-23 15:26:54 -08:00
Max Brunsfeld
555277a102 Allow testing highlight and tag queries when testing wasm build
Replace non-mutating `ts_parser_wasm_store` function with
`ts_parser_take_wasm_store`, which removes and returns the wasm
store, in order to facilitate single ownership.
2023-01-23 11:46:59 -08:00
Max Brunsfeld
51720beeb0 Start work on a --wasm flag for the test subcommand 2022-11-15 17:14:33 -08:00
Max Brunsfeld
7dc81303f6 Get wasm store working with parser with stateless external scanners 2022-11-15 17:14:33 -08:00
Max Brunsfeld
042e6f9d57 Restucture wasm module, return a normal TSLanguage from load_language 2022-11-15 17:14:33 -08:00
Max Brunsfeld
3f1a7f9cd4 Start work on ability to load wasm languages from native lib, via wasmtime 2022-11-15 17:14:33 -08:00
Max Brunsfeld
efd22e452b Fix suppression of empty tokens during error handling at included range boundaries 2022-11-14 12:20:39 -08:00
Logan Collins
5337a678fa Resolving a potential crash freeing a null pointer when cancelling parses. 2022-10-31 11:35:52 -07:00
Matt
af6611e530
matching integer types 2022-09-21 08:50:01 -04:00
Max Brunsfeld
3ac36b0cbe Handle backslashes in token names when printing DOT debug graphs 2022-06-25 17:13:11 -07:00
Max Brunsfeld
5aa2f4dc8c Log when ignoring an empty external token after an error 2022-06-24 19:07:27 -07:00
Max Brunsfeld
d223a81b50 Allow empty external tokens during err recovery if they change the scanner's state 2022-06-24 15:58:13 -07:00
Max Brunsfeld
c0e1991f6b 🎨 ts_parser__lex 2022-06-24 14:24:21 -07:00
Max Brunsfeld
ca902065cb Fix bug when stack versions merge after reducing a non-terminal extra 2022-06-24 14:24:21 -07:00
Rich Siegel
150eb2966b Fixed warning/error when compiling with clang -Os.
DISCUSSION:

When compiling with `-Os` for "smallest, fastest", an error is reported in `parser.c`:

```
/Users/siegel/git/tree-sitter/lib/src/./parser.c:1368:10: error: unused variable 'did_merge' [-Werror,-Wunused-variable]
    bool did_merge = ts_stack_merge(self->stack, version, previous_version_count);
         ^
1 error generated.
```

This is because with `NDEBUG` set,  `assert(e)` collapses to `(void)0`,
which in turn means that `did_merge` does not actually get consumed.
This seems to get caught when compiling with `-Os`, but not otherwise.

Compiler version:
```
Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: arm64-apple-darwin21.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
```
2022-03-04 18:00:16 -05:00
Max Brunsfeld
8decec3774 Properly incorporate lookahead bytes when recovering via missing token 2022-03-02 17:12:25 -08:00
Max Brunsfeld
0fb864c1a0 Retain information about the lexer's lookahead for the token where an error was detected 2022-02-22 09:45:26 -08:00
Max Brunsfeld
0bdd9b640c Store the lookahead subtree of paused stack versions, not just the lookahead symbol 2022-02-22 09:45:26 -08:00
Max Brunsfeld
d08f1af15c 🎨 2022-02-22 09:43:57 -08:00
Amin Yahyaabadi
6e0c49305e Use ISO C and C++ conformant name: _fdopen on windows
Fixes
```
warning: In file included from src\lib.c:12:
warning: src/./parser.c:1781:28: warning: 'fdopen' is deprecated: The POSIX name for this item is deprecated. Instead, use the ISO C and C++ conformant name: _fdopen. See online help for details. [-Wdeprecated-declarations]
warning:     self->dot_graph_file = fdopen(fd, "a");
warning:                            ^
warning: C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\ucrt\stdio.h:2431:28: note: 'fdopen' has been explicitly marked deprecated here
warning:         _Check_return_     _CRT_NONSTDC_DEPRECATE(_fdopen)    _ACRTIMP FILE* __cdecl fdopen(_In_ int _FileHandle, _In_z_ char const* _Format);
warning:                            ^
warning: C:\Program Files (x86)\Windows Kits\10\Include\10.0.19041.0\ucrt\corecrt.h:414:50: note: expanded from macro '_CRT_NONSTDC_DEPRECATE'
warning:         #define _CRT_NONSTDC_DEPRECATE(_NewName) _CRT_DEPRECATE_TEXT(             \
warning:                                                  ^
warning: C:\Program Files (x86)\Microsoft Visual Studio\2019\Preview\VC\Tools\MSVC\14.29.30133\include\vcruntime.h:310:47: note: expanded from macro '_CRT_DEPRECATE_TEXT'
warning: #define _CRT_DEPRECATE_TEXT(_Text) __declspec(deprecated(_Text))
```
2022-01-21 22:39:24 -08:00
Max Brunsfeld
584b55df8d Delete unused code, tweak whitespace 2022-01-19 16:54:57 -08:00
Max Brunsfeld
4e2e059865 Ensure 'extra' bit is set correctly when reusing a node
Fixes #1444
2021-11-19 12:43:55 -08:00
Andrew Hlynskyi
7f538170bf fix(parser): count rows in the debug log from 0 2021-07-15 11:47:14 +03:00
Max Brunsfeld
a40045a419 When editing, properly invalidate trees that depend on get_column 2021-03-11 14:46:13 -08:00
Max Brunsfeld
774ae5e3d5 In parse tables, store production ids as 16 bits
Also remove the use of bitfields from the parse table format.
In all cases, bitfields were not necessary to achieve the
current binary sizes. Avoiding them makes the binaries more
portable.

There was no way to make this change backward-compatible,
so we have finally dropped support for parsers generated
with an earlier version of Tree-sitter.

At some point, when Atom adopts this version of Tree-sitter,
this change will affect Atom users who have installed packages
using third-party Tree-sitter parsers. The packages will need
to be updated to use a regenerated version of the parsers.
2021-02-25 16:12:31 -08:00
Andrew Hlynskyi
6448053efa parser.c - added backslash escaping in LOG_LOOKAHEAD to make it clear in case of \n 2021-02-08 13:33:54 +02:00
Andrew Hlynskyi
51f8b16bb0 parser.c - make LOG_LOKAHEAD simpler according to disscussion in PR #917 2021-02-08 13:32:15 +02:00
Andrew Hlynskyi
f92ad04824 parser.c - added LOG_LOOKAHEAD macro to escape \n in logging 2021-02-08 06:23:55 +02:00