Max Brunsfeld
c3a242740b
Allow lookahead to be broken down further after performing reductions
2016-07-04 12:20:23 -07:00
Max Brunsfeld
9c37c06ec7
Remove unnecessary parser debugger accessors
2016-07-04 12:20:23 -07:00
Max Brunsfeld
d554fab5b5
Remove unused tree state constant
2016-06-27 14:39:12 -07:00
Max Brunsfeld
8c26d99353
Store error recovery actions in the normal parse table
2016-06-27 14:07:47 -07:00
Max Brunsfeld
bf245f016c
Check for better stack versions before skipping a node after reducing it
2016-06-26 22:51:48 -07:00
Max Brunsfeld
9538b5b879
Don't count extra trees toward stack versions' error costs
2016-06-26 22:46:40 -07:00
Max Brunsfeld
6fd3edceae
Fix logic for inserting leading & trailing extras into root node on acceptance
2016-06-26 11:57:42 -07:00
Max Brunsfeld
9972709e43
Allow error recovery to skip non-terminal nodes after error detection
2016-06-24 10:28:05 -07:00
Max Brunsfeld
76c6e47d49
Fix assignment of parser split condition
2016-06-23 13:41:38 -07:00
Max Brunsfeld
df09f37a6c
Tweak parse log messages
2016-06-23 11:42:43 -07:00
Max Brunsfeld
09b019c530
Fix test for invalid blank input
2016-06-23 09:24:26 -07:00
Max Brunsfeld
c6e9b32d3f
Print all the same parse log messages for both debugging methods
2016-06-22 22:36:11 -07:00
Max Brunsfeld
58b1d88950
Clean up breakdown_top_of_stack function
2016-06-22 22:32:38 -07:00
Max Brunsfeld
f425fbad18
Break down reused node on stack whenever lookahead can't be reused
2016-06-22 22:03:27 -07:00
Max Brunsfeld
16ca038556
Don't accept input with errors on the stack
2016-06-22 21:05:05 -07:00
Max Brunsfeld
b40c0326dc
Include parse tree rendering at end of debug output
2016-06-22 21:04:35 -07:00
Max Brunsfeld
57f669dfda
clang-format
2016-06-22 14:11:08 -07:00
Max Brunsfeld
3ceb3a1eb2
Clean up main parse loop
2016-06-22 14:10:54 -07:00
Max Brunsfeld
e7ccebc153
Avoid duplicate counting of trees when repairing errors
2016-06-21 22:58:06 -07:00
Max Brunsfeld
43ae8235fd
Remove the error action; a lack of actions implies an error.
2016-06-21 22:53:48 -07:00
Max Brunsfeld
6a7a5cfc3f
Remove nesting in parse action struct
2016-06-21 21:36:33 -07:00
Max Brunsfeld
634ee29d10
Clean up main consume_lookahead loop
2016-06-21 21:27:33 -07:00
Max Brunsfeld
38c144b4a3
Refine logic for deciding when tokens need to be re-lexed
...
* While generating the lex table, note which tokens can match the
same string. A token needs to be relexed when it has possible
homonyms in the current state.
* Also note which tokens can match substrings of each other tokens.
A token needs to be relexed when there are viable tokens that
could match longer strings in the current state and the next
token has been edited.
* Remove the logic for marking tokens as fragile on creation.
* Store the reusability/non-reusability of symbols off of individual
actions and onto the entire entry for the state & symbol.
2016-06-21 07:28:04 -07:00
Max Brunsfeld
94721c7ec0
Rewind and re-tokenize in error mode after detecting an error
2016-06-17 21:26:03 -07:00
Max Brunsfeld
76975f56ec
Display node positions in tooltips in debugging graphs
2016-06-15 11:06:00 -07:00
Max Brunsfeld
f69d709650
Remove unused functions
2016-06-15 10:17:54 -07:00
Max Brunsfeld
ecc7399ed3
Fix stack breakdown procedure when there are trailing extra tokens
2016-06-14 20:25:33 -07:00
Max Brunsfeld
e70547cd11
Allow recoveries that skip leading children of invisible trees
...
Before this, errors could only be recovered by skipping internal children.
2016-06-14 14:48:35 -07:00
Max Brunsfeld
6dda23796b
Don't count extra tokens in error cost computation
2016-06-14 14:47:16 -07:00
Max Brunsfeld
2109f0ed74
Handle allocation failures when copying tree arrays
2016-06-14 14:46:49 -07:00
Max Brunsfeld
f77c08eff5
Check for better stack versions earlier when handling errors
2016-06-12 17:41:25 -07:00
Max Brunsfeld
2b80e66188
Don't merge stack versions with different error costs
2016-06-12 17:27:08 -07:00
Max Brunsfeld
ef2c3a10e3
Fix bug when reusing formerly extra trees as non-extra trees
2016-06-12 17:26:15 -07:00
Max Brunsfeld
e1c51556f2
Check that error repair reductions have the right child count
...
Also, avoid repetitively walking through the parse table to verify the
child trees in an error repair. Instead, walk through the parse table
once, and reuse the list of reductions that would be valid after that
sequence.
2016-06-10 13:15:45 -07:00
Max Brunsfeld
00a0939504
Abort erroneous parse versions more eagerly
2016-06-02 14:04:48 -07:00
Max Brunsfeld
96e2e49ef8
Fix false positive when marking nodes unreusable due to ambiguity
2016-05-30 14:16:55 -07:00
Max Brunsfeld
1e42e68098
Handle cases where both valid and incomplete reduction paths end at the same stack node
2016-05-30 14:08:42 -07:00
Max Brunsfeld
ea47fdc0fe
Rework logic for when to abandon parses with errors
2016-05-29 22:36:47 -07:00
Max Brunsfeld
6535704870
Replace stack_merge_new function with two simpler functions
...
- merge(version1, version2)
- split(version)
2016-05-28 21:22:10 -07:00
Max Brunsfeld
e686478ad2
Rename stack_merge function to stack_merge_all
2016-05-28 20:24:08 -07:00
Max Brunsfeld
1e353381ff
Don't create error node in lexer unless token is completely invalid
...
Before, any syntax error would cause the lexer to create an error
leaf node. This could happen even with a valid input, if the parse
stack had split and one particular version of the parse stack
failed to parse.
Now, an error leaf node is only created when the lexer cannot understand
part of the input stream at all. When a normal syntax error occurs,
the lexer just returns a token that is outside of the expected token
set, and the parser handles the unexpected token.
2016-05-26 14:15:10 -07:00
Max Brunsfeld
7c859a07bb
Allow null trees in the stack
2016-05-26 13:34:57 -07:00
Max Brunsfeld
a3679fbb1f
Distinguish separators from main tokens via a property on transitions
...
It was incorrect to store it as a property on the lexical states themselves
2016-05-19 16:27:25 -07:00
Max Brunsfeld
77e0e4bb16
Fix some leaks after allocation failures occur
2016-05-16 10:53:31 -07:00
Max Brunsfeld
88053cf723
In tests, don’t record allocations while printing debug graphs
2016-05-16 10:44:19 -07:00
Max Brunsfeld
d50f6a58cc
Abort parse versions w/ worse errors when repairing an error
2016-05-16 10:33:19 -07:00
Max Brunsfeld
22c550c9d6
Discard tokens after error detection to find the best repair
...
* Use GLR stack-splitting to try all numbers of tokens to
discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
rather than the statically-computed 'in-progress symbols' list
2016-05-11 13:49:43 -07:00
Max Brunsfeld
9d247e45b2
Deemphasize extra trees in stack debugging graphs
2016-05-01 15:24:50 -07:00
Max Brunsfeld
9ad1e36238
Rename out_of_context_states -> recovery_states
2016-04-27 14:14:56 -07:00
Max Brunsfeld
31f6b2e24a
Refactor construction of out-of-context states
2016-04-25 21:59:40 -07:00
