Max Brunsfeld
773e50f26b
Update error recovery specs to reflect slightly different recoveries
2016-06-18 20:46:16 -07:00
Max Brunsfeld
94721c7ec0
Rewind and re-tokenize in error mode after detecting an error
2016-06-17 21:26:03 -07:00
Max Brunsfeld
76975f56ec
Display node positions in tooltips in debugging graphs
2016-06-15 11:06:00 -07:00
Max Brunsfeld
70d3cde775
Remove extra leading newline from corpus spec texts
2016-06-15 10:31:34 -07:00
Max Brunsfeld
f69d709650
Remove unused functions
2016-06-15 10:17:54 -07:00
Max Brunsfeld
ecc7399ed3
Fix stack breakdown procedure when there are trailing extra tokens
2016-06-14 20:25:33 -07:00
Max Brunsfeld
e70547cd11
Allow recoveries that skip leading children of invisible trees
...
Before this, errors could only be recovered by skipping internal children.
2016-06-14 14:48:35 -07:00
Max Brunsfeld
6dda23796b
Don't count extra tokens in error cost computation
2016-06-14 14:47:16 -07:00
Max Brunsfeld
2109f0ed74
Handle allocation failures when copying tree arrays
2016-06-14 14:46:49 -07:00
Max Brunsfeld
f77c08eff5
Check for better stack versions earlier when handling errors
2016-06-12 17:41:25 -07:00
Max Brunsfeld
2b80e66188
Don't merge stack versions with different error costs
2016-06-12 17:27:08 -07:00
Max Brunsfeld
ef2c3a10e3
Fix bug when reusing formerly extra trees as non-extra trees
2016-06-12 17:26:15 -07:00
Max Brunsfeld
e1c51556f2
Check that error repair reductions have the right child count
...
Also, avoid repetitively walking through the parse table to verify the
child trees in an error repair. Instead, walk through the parse table
once, and reuse the list of reductions that would be valid after that
sequence.
2016-06-10 13:15:45 -07:00
Max Brunsfeld
6d40e317df
Ensure that reductions are ordered by child count in parse table
2016-06-10 13:11:52 -07:00
Max Brunsfeld
00a0939504
Abort erroneous parse versions more eagerly
2016-06-02 14:04:48 -07:00
Max Brunsfeld
9b67b21dcd
Fix an outdated error corpus entry
2016-06-02 14:04:10 -07:00
Max Brunsfeld
96e2e49ef8
Fix false positive when marking nodes unreusable due to ambiguity
2016-05-30 14:16:55 -07:00
Max Brunsfeld
1e42e68098
Handle cases where both valid and incomplete reduction paths end at the same stack node
2016-05-30 14:08:42 -07:00
Max Brunsfeld
ea47fdc0fe
Rework logic for when to abandon parses with errors
2016-05-29 22:36:47 -07:00
Max Brunsfeld
6535704870
Replace stack_merge_new function with two simpler functions
...
- merge(version1, version2)
- split(version)
2016-05-28 21:22:10 -07:00
Max Brunsfeld
e686478ad2
Rename stack_merge function to stack_merge_all
2016-05-28 20:24:08 -07:00
Max Brunsfeld
e1a3a1daeb
Import error corpus entries from grammar repos
...
Now that error recovery requires no input for the grammar author, it shouldn't
be tested in the individual grammar repos.
2016-05-28 20:12:02 -07:00
Max Brunsfeld
1e353381ff
Don't create error node in lexer unless token is completely invalid
...
Before, any syntax error would cause the lexer to create an error
leaf node. This could happen even with a valid input, if the parse
stack had split and one particular version of the parse stack
failed to parse.
Now, an error leaf node is only created when the lexer cannot understand
part of the input stream at all. When a normal syntax error occurs,
the lexer just returns a token that is outside of the expected token
set, and the parser handles the unexpected token.
2016-05-26 14:15:10 -07:00
Max Brunsfeld
7c859a07bb
Allow null trees in the stack
2016-05-26 13:34:57 -07:00
Max Brunsfeld
a3679fbb1f
Distinguish separators from main tokens via a property on transitions
...
It was incorrect to store it as a property on the lexical states themselves
2016-05-19 16:27:25 -07:00
Max Brunsfeld
59712ec492
Clean up lex table generation
2016-05-19 13:25:46 -07:00
Max Brunsfeld
31cc6e6f9c
Remove unused InProgressSymbolEntry typedef
2016-05-16 12:46:29 -07:00
Max Brunsfeld
77e0e4bb16
Fix some leaks after allocation failures occur
2016-05-16 10:53:31 -07:00
Max Brunsfeld
88053cf723
In tests, don’t record allocations while printing debug graphs
2016-05-16 10:44:19 -07:00
Max Brunsfeld
d50f6a58cc
Abort parse versions w/ worse errors when repairing an error
2016-05-16 10:33:19 -07:00
Max Brunsfeld
507d5ad9f7
Include shift-extra actions alongside other actions in recovery states
2016-05-16 10:33:18 -07:00
Max Brunsfeld
19bd09b81d
Don't include accept actions in recovery states
2016-05-11 14:02:26 -07:00
Max Brunsfeld
22c550c9d6
Discard tokens after error detection to find the best repair
...
* Use GLR stack-splitting to try all numbers of tokens to
discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
rather than the statically-computed 'in-progress symbols' list
2016-05-11 13:49:43 -07:00
Max Brunsfeld
9d247e45b2
Deemphasize extra trees in stack debugging graphs
2016-05-01 15:24:50 -07:00
Max Brunsfeld
9ad1e36238
Rename out_of_context_states -> recovery_states
2016-04-27 14:14:56 -07:00
Max Brunsfeld
5b74813a5c
Refine logic for which tokens to use in error recovery
2016-04-27 14:09:19 -07:00
Max Brunsfeld
31f6b2e24a
Refactor construction of out-of-context states
2016-04-25 21:59:40 -07:00
Max Brunsfeld
e99a3925e0
Merge all versions created in a given reduce operation
2016-04-24 00:55:19 -07:00
Max Brunsfeld
fe74c6fb34
Explicitly mark stack versions in debugging graphs
2016-04-24 00:54:59 -07:00
Max Brunsfeld
fd4c33209e
Select ambiguous alternatives by minimizing error size
2016-04-24 00:54:20 -07:00
Max Brunsfeld
1fb6065f02
Move tree sexp function back to tree, for easier use in debugger
2016-04-24 00:09:32 -07:00
Max Brunsfeld
0d19f157ed
Adjust some spec assertions to reflect finer-grained error recoveries
2016-04-22 10:19:44 -07:00
Max Brunsfeld
cf19b2e58d
Make repeat rules left-recursive instead of right recursive
2016-04-18 12:40:14 -07:00
Max Brunsfeld
0ea2962fbe
Clean up consume_lookahead function
2016-04-18 11:17:07 -07:00
Max Brunsfeld
73c99e9926
Fix returned stack version when popping halts at an error
2016-04-18 11:17:07 -07:00
Max Brunsfeld
f63fcffe95
Fix incorrect cast in ts_language_symbol_is_in_progress
2016-04-18 11:17:07 -07:00
Max Brunsfeld
655d374d0c
Recompile test languages if parser.h changes
2016-04-18 11:17:06 -07:00
Max Brunsfeld
2f81a6f6c7
Don't recover from errors based on extra tokens
2016-04-15 23:14:31 -07:00
Max Brunsfeld
f62e0a3d49
Ensure reference count of new tree copy is 1
2016-04-15 23:01:36 -07:00
Max Brunsfeld
d55df29e49
Remove unused constant
2016-04-15 21:33:57 -07:00
