Max Brunsfeld
6535704870
Replace stack_merge_new function with two simpler functions
...
- merge(version1, version2)
- split(version)
2016-05-28 21:22:10 -07:00
Max Brunsfeld
e686478ad2
Rename stack_merge function to stack_merge_all
2016-05-28 20:24:08 -07:00
Max Brunsfeld
1e353381ff
Don't create error node in lexer unless token is completely invalid
...
Before, any syntax error would cause the lexer to create an error
leaf node. This could happen even with a valid input, if the parse
stack had split and one particular version of the parse stack
failed to parse.
Now, an error leaf node is only created when the lexer cannot understand
part of the input stream at all. When a normal syntax error occurs,
the lexer just returns a token that is outside of the expected token
set, and the parser handles the unexpected token.
2016-05-26 14:15:10 -07:00
Max Brunsfeld
7c859a07bb
Allow null trees in the stack
2016-05-26 13:34:57 -07:00
Max Brunsfeld
a3679fbb1f
Distinguish separators from main tokens via a property on transitions
...
It was incorrect to store it as a property on the lexical states themselves
2016-05-19 16:27:25 -07:00
Max Brunsfeld
77e0e4bb16
Fix some leaks after allocation failures occur
2016-05-16 10:53:31 -07:00
Max Brunsfeld
88053cf723
In tests, don’t record allocations while printing debug graphs
2016-05-16 10:44:19 -07:00
Max Brunsfeld
d50f6a58cc
Abort parse versions w/ worse errors when repairing an error
2016-05-16 10:33:19 -07:00
Max Brunsfeld
22c550c9d6
Discard tokens after error detection to find the best repair
...
* Use GLR stack-splitting to try all numbers of tokens to
discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
rather than the statically-computed 'in-progress symbols' list
2016-05-11 13:49:43 -07:00
Max Brunsfeld
9d247e45b2
Deemphasize extra trees in stack debugging graphs
2016-05-01 15:24:50 -07:00
Max Brunsfeld
9ad1e36238
Rename out_of_context_states -> recovery_states
2016-04-27 14:14:56 -07:00
Max Brunsfeld
31f6b2e24a
Refactor construction of out-of-context states
2016-04-25 21:59:40 -07:00
Max Brunsfeld
e99a3925e0
Merge all versions created in a given reduce operation
2016-04-24 00:55:19 -07:00
Max Brunsfeld
fe74c6fb34
Explicitly mark stack versions in debugging graphs
2016-04-24 00:54:59 -07:00
Max Brunsfeld
fd4c33209e
Select ambiguous alternatives by minimizing error size
2016-04-24 00:54:20 -07:00
Max Brunsfeld
1fb6065f02
Move tree sexp function back to tree, for easier use in debugger
2016-04-24 00:09:32 -07:00
Max Brunsfeld
0ea2962fbe
Clean up consume_lookahead function
2016-04-18 11:17:07 -07:00
Max Brunsfeld
73c99e9926
Fix returned stack version when popping halts at an error
2016-04-18 11:17:07 -07:00
Max Brunsfeld
f63fcffe95
Fix incorrect cast in ts_language_symbol_is_in_progress
2016-04-18 11:17:07 -07:00
Max Brunsfeld
2f81a6f6c7
Don't recover from errors based on extra tokens
2016-04-15 23:14:31 -07:00
Max Brunsfeld
f62e0a3d49
Ensure reference count of new tree copy is 1
2016-04-15 23:01:36 -07:00
Max Brunsfeld
d55df29e49
Remove unused constant
2016-04-15 21:33:57 -07:00
Max Brunsfeld
316aba76ee
Reorder stack methods
2016-04-15 21:33:31 -07:00
Max Brunsfeld
cad663b144
Consider multiple error repairs on the same path of the stack
...
This changes the API to the stack_iterate function so that you can pop
from the stack without stopping iteration
2016-04-15 21:28:00 -07:00
Max Brunsfeld
3778c63277
Clean up stack pop code
2016-04-11 23:12:50 -07:00
Max Brunsfeld
9d87bc4517
Rename stack successors to links
2016-04-11 22:41:06 -07:00
Max Brunsfeld
695be5bc79
Merge equivalent stacks in a separate stage of parsing
...
* No more automatic merging every time a state is pushed to the stack
* When popping from the stack, the current version is always preserved
2016-04-10 14:12:24 -07:00
Max Brunsfeld
827573f1c7
Clean up reduce function
2016-04-07 10:40:33 -07:00
Max Brunsfeld
5ba40f15ad
Rename stack heads to versions
2016-04-04 12:25:57 -07:00
Max Brunsfeld
b1a696085a
Clean up stack pop functions
2016-04-04 11:59:10 -07:00
Max Brunsfeld
2f3e92c9be
Add function for popping all nodes from the stack
2016-04-04 11:44:45 -07:00
Max Brunsfeld
267092940d
Collapse redundant interior error nodes
2016-04-03 23:46:43 -07:00
Max Brunsfeld
eb47edb446
Add static function for stringifying nodes including hidden ones
2016-04-03 23:45:43 -07:00
Max Brunsfeld
849ecf0a13
Display pending stack links as dashed lines
2016-04-02 23:02:21 -07:00
Max Brunsfeld
91e3609fbf
Write to file directly from stack debugging function
2016-04-02 22:18:44 -07:00
Max Brunsfeld
d5c4dacb35
Clean up parser.c
2016-04-02 20:58:27 -07:00
Max Brunsfeld
956b3e68d7
Only store one reusable node, not one for each stack version
2016-04-01 21:17:23 -07:00
Max Brunsfeld
6bce6da1e6
Store verifying flag within parse stack
2016-03-31 12:03:21 -07:00
Max Brunsfeld
523923bd93
Ensure reduce only produces one slice when stopping at errors
2016-03-10 21:26:21 -08:00
Max Brunsfeld
be707970da
Fix always_inline attribute usage
2016-03-10 13:10:46 -08:00
Max Brunsfeld
07625808d3
Look for two consecutive valid tokens before resuming parsing after errors
2016-03-10 11:57:33 -08:00
Max Brunsfeld
a948cb3ea0
Allow normal reductions that pop to the initial error state
2016-03-10 11:56:10 -08:00
Max Brunsfeld
e7d3d40a59
Explicitly inform stack pop callback when the stack is exhausted
...
Also, pass non-extra tree count as a single value, rather than keeping
track of the extra count and the total separately.
2016-03-10 11:51:55 -08:00
Max Brunsfeld
288a164823
Retrieve current lookahead state before it can be removed in reduce
2016-03-10 11:44:23 -08:00
Max Brunsfeld
a8774831e8
Fix buffer overflow when limiting stack node's successor count
2016-03-10 11:43:13 -08:00
Max Brunsfeld
da4a3f6516
Don't abort on allocation failures while finishing parse
2016-03-10 11:42:10 -08:00
Max Brunsfeld
2e35587161
Use new stack_pop_until function for repairing errors
2016-03-07 20:06:46 -08:00
Max Brunsfeld
4348eb89d4
Expose lower stack nodes via pop_until() function
...
This callback-based API allows the parser to easily visit each interior node
of the stack when searching for an error repair. It also is a better abstraction
over the stack's DAG implementation than having the public functions for
accessing entries and their successor entries.
2016-03-07 16:09:34 -08:00
Max Brunsfeld
c0595c21c5
Halt stack pops at all error states, not just error trees
2016-03-03 11:05:37 -08:00
Max Brunsfeld
5f9be9d9b2
Make Failure enumerators have the value zero
2016-03-03 10:24:12 -08:00
