Max Brunsfeld
a40045a419
When editing, properly invalidate trees that depend on get_column
2021-03-11 14:46:13 -08:00
Max Brunsfeld
e29d3714f7
Fix behavior of Lexer.get_column when at EOF
2021-03-11 12:11:33 -08:00
Max Brunsfeld
dd4cba2625
Allow symbols to be used in precedence lists
2021-03-03 13:11:05 -08:00
Max Brunsfeld
344797c110
Implement named precedence comparison
2021-02-24 16:02:56 -08:00
Max Brunsfeld
9abad258c3
Expand tests for unicode property escapes
2021-02-17 13:29:30 -08:00
Max Brunsfeld
e3ba701344
Start work on handling unicode property escapes in regexes
2021-01-29 16:37:45 -08:00
Max Brunsfeld
391fc8c340
Update unit tests to expect ruby grammar change
2020-12-20 19:34:45 -08:00
Max Brunsfeld
94c61de353
Update JS error recovery test to reflect grammar change
2020-12-03 10:09:58 -08:00
Max Brunsfeld
4e86b76e8c
Update ruby error recovery test to reflect grammar tweaks
2020-11-03 10:28:17 -08:00
Max Brunsfeld
a8d77001c2
Update c error recovery test to reflect behavior change
2020-09-21 13:20:50 -07:00
Max Brunsfeld
9a7fdd29c2
Add test for non-terminal extras that share non-extra rules
2020-07-29 09:53:07 -07:00
Max Brunsfeld
253f23c3d4
Fix error when parse error occurs after non-terminal extra
2020-07-28 13:33:13 -07:00
Max Brunsfeld
3814babfa8
Update error recovery test to reflect JS grammar changes
2020-01-28 09:16:36 -08:00
Max Brunsfeld
0cb2ef1082
Fix code paths that still conflated null characters with EOF
2019-12-06 15:29:03 -08:00
Max Brunsfeld
660efd8c0f
Fix handling of token rules directly inside of alias, prec, or field.
...
Fixes #483
2019-11-15 10:53:17 -08:00
Max Brunsfeld
39ceff1908
Restore json corpus tests, which accidentally weren't running
2019-10-31 14:32:50 -07:00
Max Brunsfeld
fcaabea0cf
Allow non-terminal extras
2019-10-21 16:08:59 -07:00
Max Brunsfeld
0955c5b3d7
Handle named nodes aliased as anonymous nodes
...
Fixes #401
2019-08-29 14:28:44 -07:00
Max Brunsfeld
5e04daf483
Avoid non-deterministic set iteration order when handling conflicts
2019-08-13 15:57:42 -07:00
Max Brunsfeld
443ac1c1e5
Update python tests to reflect grammar changes
2019-07-22 09:55:30 -07:00
Max Brunsfeld
c90a532d8f
Update tests to reflect javascript grammar changes
2019-07-10 14:30:43 -05:00
Max Brunsfeld
bc5e71bcc2
Fix handling of zero-width external tokens at beginning of document
...
Fixes #316
2019-04-08 15:01:52 -07:00
Max Brunsfeld
6490b1e4f2
Fix non-deterministic conflict message order
2019-03-29 17:36:19 -07:00
Max Brunsfeld
65d1ce8593
lib: Include fields in ts_node_string output
...
This allows you to assert about fields in tests. But if your test
s-expression does *not* include fields, the fields will be stripped from
the regexp before comparison.
2019-02-13 09:47:21 -08:00
Max Brunsfeld
e579e09569
Ensure interpretations are in a predictable order in conflict messages
2019-02-08 17:11:44 -08:00
Max Brunsfeld
6105bf9909
Include error recovery examples in test suite
2019-01-20 16:58:49 -08:00
Max Brunsfeld
5c3c1dd0bd
Get logging flags working properly with test script
2019-01-15 13:21:48 -08:00
Max Brunsfeld
6592fdd24c
Fix parser generation error messages
2019-01-11 17:26:45 -08:00
Max Brunsfeld
e64f7a64a1
Start work on running test corpus tests
2019-01-11 13:31:06 -08:00
Max Brunsfeld
00a94ac040
Fix javascript error fixture
2018-10-03 21:38:03 -07:00
Max Brunsfeld
7fc64ed25a
Account for simple aliases in unit reduction elimination
2018-08-30 09:57:22 -07:00
Max Brunsfeld
41fe564a90
Update error recovery fixture
2018-08-01 15:09:45 -07:00
Max Brunsfeld
cb784975a4
Add IMMEDIATE_TOKEN rule type, for enforcing no preceding extras
2018-08-01 14:00:57 -07:00
Max Brunsfeld
6ebb9195b1
Flesh out integration test for precedence within tokens
2018-07-31 10:18:49 -07:00
Max Brunsfeld
23e4596ec1
Add test for handling of precedence within tokens
2018-07-26 17:06:09 -07:00
Max Brunsfeld
379a2fd121
Incrementally build a tree of skipped tokens
...
Rather than pushing them to the stack individually
2018-04-09 12:29:22 -07:00
Max Brunsfeld
80f856cef5
Maintain a total node count on every tree
...
This simplifies (and fixes bugs in) the parse stack's tracking of its
total node count since the last error, which is needed for error
recovery.
2018-04-06 13:26:18 -07:00
Max Brunsfeld
e59558c83b
Allow stack versions to be temporarily paused
...
This way, when detecting an error, we can defer the decision about
whether to bail or recover until all stack versions are processed.
2018-04-06 13:26:18 -07:00
Max Brunsfeld
7183f8d3e7
Fix unit reduction elimination bugs
...
* Handle 'chains' of unit reductions starting in a single state
* Avoid eliminating rules which will later receive aliases
2018-03-12 07:54:18 -07:00
Max Brunsfeld
df2430b94c
Remove a C error recovery test temporarily
2018-03-08 14:18:12 -08:00
Max Brunsfeld
2daae48fe0
Handle conflicts in repeat rules after external tokens
...
Signed-off-by: Rick Winfrey <rewinfrey@github.com>
2018-02-14 11:24:51 -08:00
Max Brunsfeld
2e4f76c164
Don't allow an epsilon start rule if it is used in other rules
2018-01-23 17:05:28 -08:00
Max Brunsfeld
13adfe4927
Update error corpus
2017-12-29 16:21:04 -08:00
Max Brunsfeld
e5851fd9b9
Don't use non-existent \a syntax in test grammars
2017-12-13 12:21:28 -08:00
Max Brunsfeld
f426b61e7c
Fix expectation around preproc directive in C error test
2017-12-13 12:21:13 -08:00
Max Brunsfeld
fbcefe25f7
Avoid creating external tokens that start after they end
2017-12-07 11:50:27 -08:00
Max Brunsfeld
c0073c5b72
Update error corpus to reflect C grammar changes
2017-10-04 15:06:12 -07:00
Max Brunsfeld
d291af9a31
Refactor error comparisons
...
* Deal with mergeability outside of error comparison function
* Make `better_version_exists` function pure (don't halt other versions
as a side effect).
* Tweak error comparison logic
Signed-off-by: Rick Winfrey <rewinfrey@github.com>
2017-09-13 16:38:15 -07:00
Max Brunsfeld
99d048e016
Simplify error recovery; eliminate recovery states
...
The previous approach to error recovery relied on special error-recovery
states in the parse table. For each token T, there was an error recovery
state in which the parser looked for *any* token that could follow T.
Unfortunately, sometimes the set of tokens that could follow T contained
conflicts. For example, in JS, the token '}' can be followed by the
open-ended 'template_chars' token, but also by ordinary tokens like
'identifier'. So with the old algorithm, when recovering from an
unexpected '}' token, the lexer had no way to distinguish identifiers
from template_chars.
This commit drops the error recovery states. Instead, when we encounter
an unexpected token T, we recover from the error by finding a previous
state S in the stack in which T would be valid, popping all of the nodes
after S, and wrapping them in an error.
This way, the lexer is always invoked in a normal parse state, in which
it is looking for a non-conflicting set of tokens. Eliminating the error
recovery states also shrinks the lex state machine significantly.
Signed-off-by: Rick Winfrey <rewinfrey@github.com>
2017-09-11 15:22:52 -07:00
Max Brunsfeld
94dc703bfc
Require that grammars' start rules be visible
2017-08-04 17:07:37 -07:00
