traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
2962 commits 16 branches 98 tags 37 MiB
Commit graph

60 commits

Author SHA1 Message Date
Max Brunsfeld
3814babfa8 Update error recovery test to reflect JS grammar changes 2020-01-28 09:16:36 -08:00
Max Brunsfeld
0cb2ef1082 Fix code paths that still conflated null characters with EOF 2019-12-06 15:29:03 -08:00
Max Brunsfeld
660efd8c0f Fix handling of token rules directly inside of alias, prec, or field. 
Fixes #483
 2019-11-15 10:53:17 -08:00
Max Brunsfeld
39ceff1908 Restore json corpus tests, which accidentally weren't running 2019-10-31 14:32:50 -07:00
Max Brunsfeld
fcaabea0cf Allow non-terminal extras 2019-10-21 16:08:59 -07:00
Max Brunsfeld
0955c5b3d7 Handle named nodes aliased as anonymous nodes 
Fixes #401
 2019-08-29 14:28:44 -07:00
Max Brunsfeld
5e04daf483 Avoid non-deterministic set iteration order when handling conflicts 2019-08-13 15:57:42 -07:00
Max Brunsfeld
443ac1c1e5 Update python tests to reflect grammar changes 2019-07-22 09:55:30 -07:00
Max Brunsfeld
c90a532d8f Update tests to reflect javascript grammar changes 2019-07-10 14:30:43 -05:00
Max Brunsfeld
bc5e71bcc2 Fix handling of zero-width external tokens at beginning of document 
Fixes #316
 2019-04-08 15:01:52 -07:00
Max Brunsfeld
6490b1e4f2 Fix non-deterministic conflict message order 2019-03-29 17:36:19 -07:00
Max Brunsfeld
65d1ce8593 lib: Include fields in ts_node_string output 
This allows you to assert about fields in tests. But if your test 
s-expression does *not* include fields, the fields will be stripped from 
the regexp before comparison.
 2019-02-13 09:47:21 -08:00
Max Brunsfeld
e579e09569 Ensure interpretations are in a predictable order in conflict messages 2019-02-08 17:11:44 -08:00
Max Brunsfeld
6105bf9909 Include error recovery examples in test suite 2019-01-20 16:58:49 -08:00
Max Brunsfeld
5c3c1dd0bd Get logging flags working properly with test script 2019-01-15 13:21:48 -08:00
Max Brunsfeld
6592fdd24c Fix parser generation error messages 2019-01-11 17:26:45 -08:00
Max Brunsfeld
e64f7a64a1 Start work on running test corpus tests 2019-01-11 13:31:06 -08:00
Max Brunsfeld
00a94ac040 Fix javascript error fixture 2018-10-03 21:38:03 -07:00
Max Brunsfeld
7fc64ed25a Account for simple aliases in unit reduction elimination 2018-08-30 09:57:22 -07:00
Max Brunsfeld
41fe564a90 Update error recovery fixture 2018-08-01 15:09:45 -07:00
Max Brunsfeld
cb784975a4 Add IMMEDIATE_TOKEN rule type, for enforcing no preceding extras 2018-08-01 14:00:57 -07:00
Max Brunsfeld
6ebb9195b1 Flesh out integration test for precedence within tokens 2018-07-31 10:18:49 -07:00
Max Brunsfeld
23e4596ec1 Add test for handling of precedence within tokens 2018-07-26 17:06:09 -07:00
Max Brunsfeld
379a2fd121 Incrementally build a tree of skipped tokens 
Rather than pushing them to the stack individually
 2018-04-09 12:29:22 -07:00
Max Brunsfeld
80f856cef5 Maintain a total node count on every tree 
This simplifies (and fixes bugs in) the parse stack's tracking of its
total node count since the last error, which is needed for error
recovery.
 2018-04-06 13:26:18 -07:00
Max Brunsfeld
e59558c83b Allow stack versions to be temporarily paused 
This way, when detecting an error, we can defer the decision about
whether to bail or recover until all stack versions are processed.
 2018-04-06 13:26:18 -07:00
Max Brunsfeld
7183f8d3e7 Fix unit reduction elimination bugs 
* Handle 'chains' of unit reductions starting in a single state
* Avoid eliminating rules which will later receive aliases
 2018-03-12 07:54:18 -07:00
Max Brunsfeld
df2430b94c Remove a C error recovery test temporarily 2018-03-08 14:18:12 -08:00
Max Brunsfeld
2daae48fe0 Handle conflicts in repeat rules after external tokens 
Signed-off-by: Rick Winfrey <rewinfrey@github.com>
 2018-02-14 11:24:51 -08:00
Max Brunsfeld
2e4f76c164 Don't allow an epsilon start rule if it is used in other rules 2018-01-23 17:05:28 -08:00
Max Brunsfeld
13adfe4927 Update error corpus 2017-12-29 16:21:04 -08:00
Max Brunsfeld
e5851fd9b9 Don't use non-existent \a syntax in test grammars 2017-12-13 12:21:28 -08:00
Max Brunsfeld
f426b61e7c Fix expectation around preproc directive in C error test 2017-12-13 12:21:13 -08:00
Max Brunsfeld
fbcefe25f7 Avoid creating external tokens that start after they end 2017-12-07 11:50:27 -08:00
Max Brunsfeld
c0073c5b72 Update error corpus to reflect C grammar changes 2017-10-04 15:06:12 -07:00
Max Brunsfeld
d291af9a31 Refactor error comparisons 
* Deal with mergeability outside of error comparison function
* Make `better_version_exists` function pure (don't halt other versions
as a side effect).
* Tweak error comparison logic

Signed-off-by: Rick Winfrey <rewinfrey@github.com>
 2017-09-13 16:38:15 -07:00
Max Brunsfeld
99d048e016 Simplify error recovery; eliminate recovery states 
The previous approach to error recovery relied on special error-recovery
states in the parse table. For each token T, there was an error recovery
state in which the parser looked for *any* token that could follow T.
Unfortunately, sometimes the set of tokens that could follow T contained
conflicts. For example, in JS, the token '}' can be followed by the
open-ended 'template_chars' token, but also by ordinary tokens like
'identifier'. So with the old algorithm, when recovering from an
unexpected '}' token, the lexer had no way to distinguish identifiers
from template_chars.

This commit drops the error recovery states. Instead, when we encounter
an unexpected token T, we recover from the error by finding a previous
state S in the stack in which T would be valid, popping all of the nodes
after S, and wrapping them in an error.

This way, the lexer is always invoked in a normal parse state, in which
it is looking for a non-conflicting set of tokens. Eliminating the error
recovery states also shrinks the lex state machine significantly.

Signed-off-by: Rick Winfrey <rewinfrey@github.com>
 2017-09-11 15:22:52 -07:00
Max Brunsfeld
94dc703bfc Require that grammars' start rules be visible 2017-08-04 17:07:37 -07:00
Max Brunsfeld
1dca3a0b58 Simplify parse version reordering 2017-08-04 14:51:14 -07:00
Max Brunsfeld
e5c3bf742d Update fixture grammars 2017-08-03 16:32:39 -07:00
Max Brunsfeld
84e4114f79 Allow conflicts involving repeat rules to be whitelisted, via their parent rule 2017-08-03 15:18:29 -07:00
Max Brunsfeld
119c67dd78 Fix conflict reporting for shift/reduce conflicts w/ multiple reductions 
We were failing to rule out shift actions with lower precedence.

Signed-off-by: Philip Turnbull <philipturnbull@github.com>
 2017-08-02 15:13:30 -07:00
Max Brunsfeld
09f4796f6b Get tests passing w/ new alias API 2017-08-01 14:35:34 -07:00
Max Brunsfeld
cb5fe80348 Rename RENAME rule to ALIAS, allow it to create anonymous nodes 2017-07-31 16:41:11 -07:00
Max Brunsfeld
cbdfd89675 Mark reductions as fragile based on their final properties 
We previously maintained a set of individual productions that were
involved in conflicts, but that was subtly incorrect because
we don't compare productions themselves when comparing parse items;
we only compare the parse items properties that could affect the
final reduce actions.
 2017-07-21 09:54:24 -07:00
Max Brunsfeld
7d9d8bce79 Handle inlined rules that contain other inlined rules 2017-07-20 15:29:06 -07:00
Max Brunsfeld
45c40c8742 Update test grammars to use new serialization API 2017-07-17 17:46:46 -07:00
Max Brunsfeld
4b40a1ed6c Support anonymous tokens inside of RENAME rules 2017-07-14 10:19:58 -07:00
Max Brunsfeld
b3a72954ff Introduce RENAME rule type 2017-07-13 17:17:22 -07:00
Max Brunsfeld
e4f57d6fee Test more cases in fixture grammar with inline rules 2017-07-12 10:12:42 -07:00