Commit graph

286 commits

Author SHA1 Message Date
Max Brunsfeld
26dac9b2dd Fix query bugs revealed by randomized tests
* Fix bugs related to named wildcard patterns vs regular wildcard patterns.
* Fix handling of extra nodes during query analysis. Previously, the
expected child_index was updated incorrectly after an extra node,
leading to false "impossible pattern" errors.
* Refine logic for avoiding unnecessary state-splitting due to fallible steps.
Compute *two* different analysis results related to step fallibility:
  * `root_pattern_guaranteed` which, like before, summarizes whether the
    entire pattern is guaranteed to match once this step is reached.
  * `parent_pattern_guaranteed` - which just indicates whether the
    immediate parent pattern is guaranteed. This is now used when
    deciding whether it's necessary to split a match state.
2021-11-21 12:02:58 -08:00
Max Brunsfeld
142f4b6438 Rename Query::step_is_definite -> is_pattern_guaranteed_at_step 2021-11-21 11:37:52 -08:00
Max Brunsfeld
f69c4861c3 Add a randomized test for query matching 2021-11-21 11:29:36 -08:00
Max Brunsfeld
2f9b7ac465 Fix logging of seed in randomized test 2021-11-19 12:16:54 -08:00
Max Brunsfeld
d05c665863 Convert some of the fixture grammars from JSON to JS
These tests are easier to write and maintain if the grammars are just JS,
like grammars normally are. It doesn't slow the tests down significantly
to shell out to `node` for each of these grammars.
2021-10-22 18:47:23 -06:00
Max Brunsfeld
ddb12dc0c6 query: Return error on unclosed tree pattern in alternation
Fixes #1436
2021-10-12 09:20:43 -07:00
Max Brunsfeld
e78413832b Restructure test suite's allocation recording so that tests can run in parallel 2021-10-11 17:24:37 -07:00
Andrew Helwer
e8e4dafa92 Fixed cargo fmt suggestions 2021-09-28 18:31:04 -04:00
Andrew Helwer
3d1f760e31 Simplified corpus test output comparisons 2021-09-28 18:27:25 -04:00
Andrew Helwer
0c3fb6e6a2 Fixed cargo fmt issues 2021-09-24 22:49:25 -04:00
Andrew Helwer
0dc1bd806f Fixed some test failures due to CRLF line endings 2021-09-24 22:42:38 -04:00
Max Brunsfeld
8fa875b1a4 Fix possible infinite loop when running syntax highlighting tests
Fixes #1347
2021-08-29 15:04:23 -07:00
Andrew Hlynskyi
7f538170bf fix(parser): count rows in the debug log from 0 2021-07-15 11:47:14 +03:00
Max Brunsfeld
9cb732859f Update unit test to reflect HTML grammar change 2021-07-12 12:21:10 -07:00
Cameron Forbis
1ea797fbd2 run fmt 2021-06-29 13:59:21 -07:00
Cameron Forbis
ae5340901e address code review comments 2021-06-29 13:49:43 -07:00
Cameron Forbis
9182ebef86 update set_included_ranges to modify extent if the current position is at the very beginning of the included range 2021-06-17 16:42:25 -07:00
Edgar
9feca80b08
Make HighlightConfiguration::configure use into iterator instead of a Vec<String> 2021-06-16 19:14:45 +02:00
Douglas Creager
e841fcfa1b cli: Extract CLI configuration into separate crate
This patch adds the `tree-sitter-config` crate, which manages
tree-sitter's configuration file.  This new setup allows different
components to define their own serializable configuration types, instead
of having to create a single monolithic configuration type.  But the
configuration itself is still stored in a single JSON file.

Before, the default location for the configuration file was
`~/.tree-sitter/config.json`.  This patch updates the default location
to follow the XDG Base Directory spec (or other relevant platform-
specific spec).  So on Linux, for instance, the new default location is
`~/.config/tree-sitter/config.json`.  We will look in the new location
_first_, and fall back on reading from the legacy location if we can't
find anything.
2021-06-10 10:21:22 -04:00
Douglas Creager
66c30648c2 rust: Extract runtime language detection into separate crate
This patch adds a new `tree-sitter-loader` crate, which holds the CLI's
logic for finding and building local grammar definitions at runtime.
This allows other command-line tools to use this logic too!
2021-06-09 17:42:30 -04:00
Douglas Creager
d2d01e77e3 cli: Use anyhow and thiserror for errors
This patch updates the CLI to use anyhow and thiserror for error
management.  The main feature that our custom `Error` type was providing
was a _list_ of messages, which would allow us to annotate "lower-level"
errors with more contextual information.  This is exactly what's
provided by anyhow's `Context` trait.

(This is setup work for a future PR that will pull the `config` and
`loader` modules out into separate crates; by using `anyhow` we wouldn't
have to deal with a circular dependency between with the new crates.)
2021-06-09 16:17:23 -04:00
Max Brunsfeld
f3ea60e23f Merge branch 'master' into query-cursor-api 2021-06-02 11:51:26 -07:00
Douglas Creager
cd96552448 query: Allow configurable match limit
The default is now a whopping 64K matches, which "should be enough for
everyone".  You can use the new `ts_query_cursor_set_match_limit`
function to set this to a lower limit, such as the previous default of
32.
2021-06-02 11:30:55 -04:00
Max Brunsfeld
d72771a19f Make ::set_{byte,point}_range methods take a Range
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
2021-05-28 14:21:11 -07:00
Max Brunsfeld
851f55afce Report non-rooted matches that intersect cursor's range restriction
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
2021-05-28 11:58:38 -07:00
Max Brunsfeld
919e9745a6 Add ts_tree_cursor_goto_first_child_for_point function
This function (and the similar `ts_tree_cursor_goto_first_child_for_byte`)
allows you to efficiently seek the tree cursor to a given position,
exploiting the tree's internal balancing, without having to visit
all of the preceding siblings of each node.
2021-05-27 12:30:19 -07:00
Max Brunsfeld
fda35894d4 Stop matching new patterns past the end of QueryCursor's range
This restores the original signatures of the `set_byte_range` and
`set_point_range` functions. Now, the QueryCursor will properly report
matches that intersect, but are not fully contained by its range.

Co-Authored-By: Nathan Sobo <nathan@zed.dev>
2021-05-25 18:02:35 -07:00
Max Brunsfeld
f597cc6a75 Preserve matches that contain the QueryCursor's start byte
Co-Authored-By: Nathan Sobo <nathan@zed.dev>
Co-Authored-By: Antonio Scandurra <me@as-cii.com>
2021-05-25 13:06:24 -07:00
Max Brunsfeld
a61f25bc58 Add APIs for advancing a QueryCursor to an arbitrary position 2021-05-24 21:07:59 -07:00
Douglas Creager
78010722a4 query: Allow unlimited pending matches
Well, not completely unlimited — we're still using a 16-bit counter to
keep track of them.  But we longer have a static maximum of 32 pending
matches when executing a query.
2021-05-24 11:02:58 -04:00
Max Brunsfeld
8c3d1466ec Allow QueryCursor's text callback to return an iterator 2021-05-23 21:05:26 -07:00
Niranjan Hasabnis
c0e2b43d34 Fixing Rust formatting issue 2021-05-21 15:43:10 +00:00
Niranjan Hasabnis
c31acb8fec Changing API name; Adding unit test and Rust bindings 2021-05-21 01:50:10 +00:00
Andrew Hlynskyi
3c0152a331 chore(fmt): Apply 'cargo fmt' to the whole code base 2021-05-19 23:21:43 +03:00
Andrew Hlynskyi
b57266cbb8 test(binding_rust): Use some redundancy to trigger more clear comliler error on the master 2021-04-22 21:00:58 +03:00
Andrew Hlynskyi
2d9a2a1855 test(binding_rust): test query captures/matches lifetime is separate from nodes lifetime 2021-04-22 21:00:58 +03:00
Max Brunsfeld
72143b17ff Fix handling of queries with field names at the roots of patterns
Fixes #1018
2021-03-27 09:36:13 -07:00
Max Brunsfeld
89e1157a29 Fix handling of repetitions in query analysis
Fixes #1007
2021-03-19 11:00:31 -07:00
Andrew Hlynskyi
192f51324e Added more test for byte and point ranges 2021-03-17 19:11:20 +02:00
Max Brunsfeld
bc0ef5f373 Add negated field patterns to queries 2021-03-12 14:38:02 -08:00
Max Brunsfeld
a40045a419 When editing, properly invalidate trees that depend on get_column 2021-03-11 14:46:13 -08:00
Max Brunsfeld
9dc88061d5 query: Fix last child operator with multiple children
Fixes #806
2021-03-09 13:40:15 -08:00
Max Brunsfeld
6ba11d7778 Fix query analysis with top-level non-terminal extras
Fixes https://github.com/tree-sitter/tree-sitter/issues/968#issuecomment-792314281
2021-03-08 09:33:35 -08:00
Max Brunsfeld
5a2534ac4a Fix crash on unterminated alternation in query
Fixes #968
2021-03-07 08:48:03 -08:00
Max Brunsfeld
f18c36ca62 Fix handling of extra nodes in query analysis
Fixes #967
2021-03-06 14:35:41 -08:00
Max Brunsfeld
db6e1d9bdc Add ts_query_cursor_did_exceed_match_limit API 2021-03-05 15:02:49 -08:00
Max Brunsfeld
047d33eecf Remove bad logic for detecting later siblings with the same field
Co-authored-by: Douglas Creager <dcreager@dcreager.net>
2021-03-05 14:50:16 -08:00
hvithrafn
05f79f0f90 Update dependencies; fix breakage 2021-02-26 10:55:20 -07:00
Max Brunsfeld
c25fa9910e Fix query match failure when indefinite steps have no captures
Fixes #937
2021-02-25 21:41:49 -08:00
Max Brunsfeld
e49a56ea4c
Merge pull request #939 from tree-sitter/partial-order-precedence
Allow precedences to be specified using strings and a partial ordering relation
2021-02-25 13:16:01 -08:00