traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
3468 commits 16 branches 98 tags 37 MiB
Commit graph

211 commits

Author SHA1 Message Date
Andrew Hlynskyi
a331607f4e dsl.js: Reuse sym() in RuleBuilder 2021-03-10 23:06:53 +02:00
Max Brunsfeld
9e50befcf8 For node-types.json, process supertypes in a stable order 2021-03-08 12:02:01 -08:00
Max Brunsfeld
8e894ff3f1 Add --no-bindings flag to generate subcommand 2021-03-08 12:01:45 -08:00
Max Brunsfeld
7300249d20 Fix incorrect merging of states with different inherited fields 
Co-authored-by: Douglas Creager <dcreager@dcreager.net>
 2021-03-05 14:49:28 -08:00
Max Brunsfeld
e20aff9a9c Fix templates for rust binding files 2021-03-04 14:22:31 -08:00
Max Brunsfeld
e12093e8df Fix regression introduced in CharacterSet optimization 2021-03-04 13:50:27 -08:00
Max Brunsfeld
dd4cba2625 Allow symbols to be used in precedence lists 2021-03-03 13:11:05 -08:00
Max Brunsfeld
592fd8678d Organize TSLanguage fields 
Due to the breaking ABI change in #943, this is our chance
to reorder the fields in a more logical way.
 2021-03-01 10:27:22 -08:00
Max Brunsfeld
d56f9ebe4e Re-enable --prev-abi flag to generate command 2021-02-26 14:51:01 -08:00
Max Brunsfeld
075bf2bd5c In generate, create rust bindings 
Also, migrate node binding files into the same 'bindings' folder.
 2021-02-26 13:24:21 -08:00
Max Brunsfeld
c1639cc456 Add production_id_count field to Language objects 
I think this is the last additional field that's needed so
that every array member of TSLanguage has a length that
can be calculated at runtime.
 2021-02-25 16:32:05 -08:00
Max Brunsfeld
d8a235faa1 Add further static validation of named precedences 2021-02-25 11:54:21 -08:00
Max Brunsfeld
344797c110 Implement named precedence comparison 2021-02-24 16:02:56 -08:00
Max Brunsfeld
d40f118370 Generalize precedence datatype to include strings 
Right now, the strings are not used in comparisons, but they
are passed through the grammar processing pipeline, and are
available to the parse table construction algorithm.

This also cleans up a confusing aspect of the parse table
construction, in which precedences and associativities were
temporarily stored in the parse table data structure itself.
 2021-02-23 20:48:39 -08:00
Max Brunsfeld
2f28a35e1b Handle unicode property escapes inside bracketed char classes 
Refs #906
 2021-02-18 22:27:44 -08:00
Max Brunsfeld
29bc26ecd5 Fix test failure after non-terminal extras change 2021-02-18 15:43:01 -08:00
Max Brunsfeld
86a891fa63 Fix bugs in parser generation for non-terminal extras 
Previously, we attempted to completely separate the parse states
for item sets with non-terminal extras from the parse states
for other rules. But there was not a complete separation.

It actually isn't necessary to separate the parse states in this way.
The only special behavior for parse states with non-terminal extra rules
is what happens at the *end* of the rule: these parse states need to
perform an unconditional reduction.

Luckily, it's possible to distinguish these *non-terminal extra ending*
states from other states just based on their normal structure, with
no additional state.
 2021-02-18 14:14:22 -08:00
Max Brunsfeld
b46d51f224 Add a unit test for all unicode character escape forms 2021-02-17 17:49:01 -08:00
Max Brunsfeld
5b630054c6 Handle negated unicode property escapes in regexes 
Refs #380
 2021-02-17 17:22:33 -08:00
Max Brunsfeld
6ae04051e7 Tweak whitespace in generated character set functions 2021-02-17 16:32:49 -08:00
Max Brunsfeld
9d9eb2234f
Merge pull request #906 from tree-sitter/unicode-property-escapes 
Handle simple unicode property escapes in regexes
 2021-02-17 16:14:42 -08:00
Max Brunsfeld
dad8546776 Generate more compact code for character set binary search 2021-02-17 13:52:23 -08:00
Max Brunsfeld
6132a10b1c Use binary search in generated character set functions 2021-02-17 13:08:56 -08:00
Max Brunsfeld
f5a4c14dbe Add some doc comments to CharacterSet 2021-02-16 21:37:52 -08:00
Max Brunsfeld
2b0de9dfec Fix small bugs in conflict reporting 
* Negative precedence values were not displayed
* Rule names were repeated in resolution suggestions
 2021-02-01 13:30:06 -08:00
Max Brunsfeld
e3ba701344 Start work on handling unicode property escapes in regexes 2021-01-29 16:37:45 -08:00
Max Brunsfeld
38444ea7f9
Merge pull request #904 from tree-sitter/character-set-ranges 
Represent CharacterSet internally as a vector of ranges
 2021-01-29 13:35:48 -08:00
Andrew Hlynskyi
2b9e5f6c4b Fix hiding problems in ./build/Debug/tree_sitter_*_binding 
In debug building modules also may happen errors and a current implementation
completely hides them, so errors like 'undefined symbol' can't be
easily identified due to wrong traceback and error message.
 2021-01-29 15:54:10 +02:00
Max Brunsfeld
ab78ab3f9b Represent CharacterSet internally as a vector of ranges 2021-01-28 16:10:39 -08:00
Max Brunsfeld
026231e93d Merge branch 'master' into HEAD 2020-12-03 09:44:33 -08:00
Max Brunsfeld
3497f34dd7 Fix parser-generation bugs introduced in #782 2020-11-02 13:43:28 -08:00
Arthur Baars
d62e7f7d75 Add test case with extra_symbols 2020-10-30 10:58:41 +01:00
Arthur Baars
f07dda692e Ensure "extras" symbols are included in the node-types.json file 
The symbols marked as "extras" are the start symbols of secondary
languages. These should be included in the aliases map just as done
for start symbol of the main language to ensure their node type and
field information is included in the node-types.json file.
 2020-10-29 18:05:24 +01:00
Max Brunsfeld
071f4e40f1 Fix generate error when there are aliases in unused rules 2020-10-28 12:34:16 -07:00
Max Brunsfeld
a2d760e426 Ensure nodes are aliased consistently within syntax error nodes 
Co-Authored-By: Rick Winfrey <rewinfrey@github.com>
 2020-10-27 15:46:09 -07:00
Max Brunsfeld
8bb8e9b8b3 Initialize TSLanguage fields in order of their declaration 
This makes parser.c valid under the C++20 standard
 2020-10-15 07:20:12 -07:00
Patrick Thomson
683a2da055 Fix crash when extras function doesn't return an array. 
Fixes #745, which failed due to attempting to call `map` on a
non-array. This bails out at the same spot, but with a more
illuminating error message.
 2020-09-30 16:21:20 -04:00
Max Brunsfeld
ffd3bdc4c1 Escape ? in C string literals 
Fixes #714
 2020-09-23 13:06:06 -07:00
Max Brunsfeld
5003064da7 Make supertypes automatically hidden, without underscore prefix 2020-09-23 09:35:14 -07:00
Max Brunsfeld
b5a9adb555 Allow queries to match on supertypes 
Co-authored-by: Ayman Nadeem <aymannadeem@github.com>
 2020-09-21 12:34:48 -07:00
Max Brunsfeld
ff488f89c9 Make the --prev-abi flag work w/ the newest abi change 2020-09-08 10:58:20 -07:00
Max Brunsfeld
2eb04094f8 Handle aliased parent nodes in query analysis 2020-08-21 14:12:04 -07:00
Max Brunsfeld
1ea29053e1 Merge branch 'master' into query-pattern-is-definite 2020-08-14 09:31:55 -07:00
Max Brunsfeld
81bbdf19f4 Fix handling of non-terminal extras that share non-extra rules 
Fixes #701
 2020-07-29 09:50:13 -07:00
Max Brunsfeld
32099050d6 node_types: Fix panic when field is associated with a hidden token 
Fixes #695
 2020-07-24 09:26:56 -07:00
Max Brunsfeld
82aa1462fd Clean up get_variable_info function 2020-07-17 15:12:13 -07:00
Max Brunsfeld
c4fca5f73e node types: Fix handling of repetitions inside of fields 
Fixes #676
 2020-07-17 14:19:59 -07:00
Max Brunsfeld
f4adf0269a Propagate dynamic precedence correctly for inlined rules 
Fixes #683
 2020-07-17 09:53:01 -07:00
Max Brunsfeld
4c2f36a07b Mark steps as definite on query construction 
* Add a ts_query_pattern_is_definite API, just for debugging this
* Store state_count on TSLanguage structs, to allow for scanning parse tables
 2020-06-25 15:06:27 -07:00
Max Brunsfeld
a6f71328fe Avoid whitelist/blacklist terminology in test comments 2020-06-16 09:22:34 -07:00