Commit graph

507 commits

Author SHA1 Message Date
Max Brunsfeld
6b87326470
Merge pull request #1787 from kianmeng/fix-typos
Fix typos
2022-08-25 10:25:39 -07:00
Max Brunsfeld
53ed4cf037 Tolerate tree edits whose old range extends beyond the end of the tree 2022-08-25 10:20:40 -07:00
Max Brunsfeld
477b667753 Add ts_tree_root_node_with_offset API 2022-08-18 13:48:47 -07:00
Max Brunsfeld
548c12fb88 Fix bug where patterns with top-level alternatives were not considered 'rooted' 2022-07-07 17:53:54 -07:00
Max Brunsfeld
1401767689 query: Don't attempt to match top-level sibling patterns directly in ERROR nodes
Co-authored-by: Keith Simmons <keith@zed.dev>
2022-07-07 15:27:00 -07:00
Kian-Meng Ang
b8552ec6c4 Fix typos 2022-06-28 19:57:42 +08:00
Max Brunsfeld
8b5ff2ec8f Loosen randomized changed range assertions on newline characters
For now, don't error if the scope changes for a newline character.
2022-06-25 17:13:46 -07:00
Max Brunsfeld
58b719541b Fix failure to match queries with wildcard at root with range restrictions 2022-06-22 15:54:06 -07:00
Aleksei Bavshin
b27b4665ac
test: fix incorrect uses of i8 instead of c_char
Fixes build on aarch64, ppc64le and other platforms that have c_char
defined as u8.
2022-03-23 00:00:26 -07:00
Max Brunsfeld
7170ec7c96 Improve randomized testing setup
* Allow iterations to be specified via an env var
* Randomly decide the edit count, with a maximum
  specified via an env var.
* Instead of separate env vars for starting seed + trial, just accept a seed
* Remove some noisy output
2022-03-02 17:12:25 -08:00
Max Brunsfeld
a494d6aa28
Merge pull request #1668 from tree-sitter/remember-lookahead-bytes-on-error-detection
Remember lookahead bytes on error detection
2022-02-26 14:59:40 -08:00
Max Brunsfeld
c697ebfb27 Add explicit unit test for error detection lookahead bug 2022-02-26 14:33:09 -08:00
Max Brunsfeld
5eb0a3090f
Merge pull request #1547 from the-mikedavis/md-test-tags
test tags queries in 'tree-sitter test'
2022-02-24 15:22:15 -08:00
Max Brunsfeld
cb4317ba8e Change goto_first_child_for_{byte,point} to compare nodes' ranges inclusively
Co-Authored-By: Antonio Scandurra <me@as-cii.com>
2022-02-04 12:38:33 -08:00
Michael Davis
02abc2a063
add test for removals in eager query matches 2022-01-18 20:54:55 -06:00
Hendrik van Antwerpen
ec9b00e5c6 Handle multiple top-level alternations correctly 2022-01-11 18:33:36 +01:00
Hendrik van Antwerpen
a1a241b013 Expose quantifiers per pattern, instead of merging for all patterns in a query 2022-01-11 18:33:36 +01:00
Hendrik van Antwerpen
1f1a449c76 Improve capture quantifier computation
Compute quantifiers in a bottom-up manner, which allows more precise
results for alternations, where the quantifiers are now precisly joined.
2022-01-11 18:33:36 +01:00
Max Brunsfeld
622359b400 Simplify allocation-recording in test suite using new ts_set_allocator API 2021-12-30 16:09:07 -08:00
Michael Davis
e7f0cc4e24
add python test case for tag testing 2021-12-18 11:07:46 -06:00
Michael Davis
47dc88b647
add tags config fixture test helper 2021-12-18 11:07:18 -06:00
Max Brunsfeld
26dac9b2dd Fix query bugs revealed by randomized tests
* Fix bugs related to named wildcard patterns vs regular wildcard patterns.
* Fix handling of extra nodes during query analysis. Previously, the
expected child_index was updated incorrectly after an extra node,
leading to false "impossible pattern" errors.
* Refine logic for avoiding unnecessary state-splitting due to fallible steps.
Compute *two* different analysis results related to step fallibility:
  * `root_pattern_guaranteed` which, like before, summarizes whether the
    entire pattern is guaranteed to match once this step is reached.
  * `parent_pattern_guaranteed` - which just indicates whether the
    immediate parent pattern is guaranteed. This is now used when
    deciding whether it's necessary to split a match state.
2021-11-21 12:02:58 -08:00
Max Brunsfeld
142f4b6438 Rename Query::step_is_definite -> is_pattern_guaranteed_at_step 2021-11-21 11:37:52 -08:00
Max Brunsfeld
f69c4861c3 Add a randomized test for query matching 2021-11-21 11:29:36 -08:00
Max Brunsfeld
2f9b7ac465 Fix logging of seed in randomized test 2021-11-19 12:16:54 -08:00
Max Brunsfeld
d05c665863 Convert some of the fixture grammars from JSON to JS
These tests are easier to write and maintain if the grammars are just JS,
like grammars normally are. It doesn't slow the tests down significantly
to shell out to `node` for each of these grammars.
2021-10-22 18:47:23 -06:00
Max Brunsfeld
ddb12dc0c6 query: Return error on unclosed tree pattern in alternation
Fixes #1436
2021-10-12 09:20:43 -07:00
Max Brunsfeld
e78413832b Restructure test suite's allocation recording so that tests can run in parallel 2021-10-11 17:24:37 -07:00
Andrew Helwer
e8e4dafa92 Fixed cargo fmt suggestions 2021-09-28 18:31:04 -04:00
Andrew Helwer
3d1f760e31 Simplified corpus test output comparisons 2021-09-28 18:27:25 -04:00
Andrew Helwer
0c3fb6e6a2 Fixed cargo fmt issues 2021-09-24 22:49:25 -04:00
Andrew Helwer
0dc1bd806f Fixed some test failures due to CRLF line endings 2021-09-24 22:42:38 -04:00
Max Brunsfeld
8fa875b1a4 Fix possible infinite loop when running syntax highlighting tests
Fixes #1347
2021-08-29 15:04:23 -07:00
Andrew Hlynskyi
7f538170bf fix(parser): count rows in the debug log from 0 2021-07-15 11:47:14 +03:00
Max Brunsfeld
9cb732859f Update unit test to reflect HTML grammar change 2021-07-12 12:21:10 -07:00
Cameron Forbis
1ea797fbd2 run fmt 2021-06-29 13:59:21 -07:00
Cameron Forbis
ae5340901e address code review comments 2021-06-29 13:49:43 -07:00
Cameron Forbis
9182ebef86 update set_included_ranges to modify extent if the current position is at the very beginning of the included range 2021-06-17 16:42:25 -07:00
Edgar
9feca80b08
Make HighlightConfiguration::configure use into iterator instead of a Vec<String> 2021-06-16 19:14:45 +02:00
Douglas Creager
e841fcfa1b cli: Extract CLI configuration into separate crate
This patch adds the `tree-sitter-config` crate, which manages
tree-sitter's configuration file.  This new setup allows different
components to define their own serializable configuration types, instead
of having to create a single monolithic configuration type.  But the
configuration itself is still stored in a single JSON file.

Before, the default location for the configuration file was
`~/.tree-sitter/config.json`.  This patch updates the default location
to follow the XDG Base Directory spec (or other relevant platform-
specific spec).  So on Linux, for instance, the new default location is
`~/.config/tree-sitter/config.json`.  We will look in the new location
_first_, and fall back on reading from the legacy location if we can't
find anything.
2021-06-10 10:21:22 -04:00
Douglas Creager
66c30648c2 rust: Extract runtime language detection into separate crate
This patch adds a new `tree-sitter-loader` crate, which holds the CLI's
logic for finding and building local grammar definitions at runtime.
This allows other command-line tools to use this logic too!
2021-06-09 17:42:30 -04:00
Douglas Creager
d2d01e77e3 cli: Use anyhow and thiserror for errors
This patch updates the CLI to use anyhow and thiserror for error
management.  The main feature that our custom `Error` type was providing
was a _list_ of messages, which would allow us to annotate "lower-level"
errors with more contextual information.  This is exactly what's
provided by anyhow's `Context` trait.

(This is setup work for a future PR that will pull the `config` and
`loader` modules out into separate crates; by using `anyhow` we wouldn't
have to deal with a circular dependency between with the new crates.)
2021-06-09 16:17:23 -04:00
Max Brunsfeld
f3ea60e23f Merge branch 'master' into query-cursor-api 2021-06-02 11:51:26 -07:00
Douglas Creager
cd96552448 query: Allow configurable match limit
The default is now a whopping 64K matches, which "should be enough for
everyone".  You can use the new `ts_query_cursor_set_match_limit`
function to set this to a lower limit, such as the previous default of
32.
2021-06-02 11:30:55 -04:00
Max Brunsfeld
d72771a19f Make ::set_{byte,point}_range methods take a Range
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
2021-05-28 14:21:11 -07:00
Max Brunsfeld
851f55afce Report non-rooted matches that intersect cursor's range restriction
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
2021-05-28 11:58:38 -07:00
Max Brunsfeld
919e9745a6 Add ts_tree_cursor_goto_first_child_for_point function
This function (and the similar `ts_tree_cursor_goto_first_child_for_byte`)
allows you to efficiently seek the tree cursor to a given position,
exploiting the tree's internal balancing, without having to visit
all of the preceding siblings of each node.
2021-05-27 12:30:19 -07:00
Max Brunsfeld
fda35894d4 Stop matching new patterns past the end of QueryCursor's range
This restores the original signatures of the `set_byte_range` and
`set_point_range` functions. Now, the QueryCursor will properly report
matches that intersect, but are not fully contained by its range.

Co-Authored-By: Nathan Sobo <nathan@zed.dev>
2021-05-25 18:02:35 -07:00
Max Brunsfeld
f597cc6a75 Preserve matches that contain the QueryCursor's start byte
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
Co-Authored-By: Antonio Scandurra <me@as-cii.com>
2021-05-25 13:06:24 -07:00
Max Brunsfeld
a61f25bc58 Add APIs for advancing a QueryCursor to an arbitrary position 2021-05-24 21:07:59 -07:00