Max Brunsfeld
a3679fbb1f
Distinguish separators from main tokens via a property on transitions
...
It was incorrect to store it as a property on the lexical states themselves
2016-05-19 16:27:25 -07:00
Max Brunsfeld
59712ec492
Clean up lex table generation
2016-05-19 13:25:46 -07:00
Max Brunsfeld
88053cf723
In tests, don’t record allocations while printing debug graphs
2016-05-16 10:44:19 -07:00
Max Brunsfeld
22c550c9d6
Discard tokens after error detection to find the best repair
...
* Use GLR stack-splitting to try all numbers of tokens to
discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
rather than the statically-computed 'in-progress symbols' list
2016-05-11 13:49:43 -07:00
Max Brunsfeld
5b74813a5c
Refine logic for which tokens to use in error recovery
2016-04-27 14:09:19 -07:00
Max Brunsfeld
fd4c33209e
Select ambiguous alternatives by minimizing error size
2016-04-24 00:54:20 -07:00
Max Brunsfeld
0d19f157ed
Adjust some spec assertions to reflect finer-grained error recoveries
2016-04-22 10:19:44 -07:00
Max Brunsfeld
cf19b2e58d
Make repeat rules left-recursive instead of right recursive
2016-04-18 12:40:14 -07:00
Max Brunsfeld
655d374d0c
Recompile test languages if parser.h changes
2016-04-18 11:17:06 -07:00
Max Brunsfeld
cad663b144
Consider multiple error repairs on the same path of the stack
...
This changes the API to the stack_iterate function so that you can pop
from the stack without stopping iteration
2016-04-15 21:28:00 -07:00
Max Brunsfeld
695be5bc79
Merge equivalent stacks in a separate stage of parsing
...
* No more automatic merging every time a state is pushed to the stack
* When popping from the stack, the current version is always preserved
2016-04-10 14:12:24 -07:00
Max Brunsfeld
5ba40f15ad
Rename stack heads to versions
2016-04-04 12:25:57 -07:00
Max Brunsfeld
6bce6da1e6
Store verifying flag within parse stack
2016-03-31 12:03:21 -07:00
Max Brunsfeld
e7d3d40a59
Explicitly inform stack pop callback when the stack is exhausted
...
Also, pass non-extra tree count as a single value, rather than keeping
track of the extra count and the total separately.
2016-03-10 11:51:55 -08:00
Max Brunsfeld
240355b04c
Make test for allocation failure handling fail more gracefully
2016-03-10 11:36:26 -08:00
Max Brunsfeld
4f726da881
Fix logic for whether to regenerate parsers in specs
2016-03-10 11:35:59 -08:00
Max Brunsfeld
2e35587161
Use new stack_pop_until function for repairing errors
2016-03-07 20:06:46 -08:00
Max Brunsfeld
4348eb89d4
Expose lower stack nodes via pop_until() function
...
This callback-based API allows the parser to easily visit each interior node
of the stack when searching for an error repair. It also is a better abstraction
over the stack's DAG implementation than having the public functions for
accessing entries and their successor entries.
2016-03-07 16:09:34 -08:00
Max Brunsfeld
bc8df9f5c5
Avoid recompiling test languages when possible
2016-03-03 12:05:04 -08:00
Max Brunsfeld
c0595c21c5
Halt stack pops at all error states, not just error trees
2016-03-03 11:05:37 -08:00
Max Brunsfeld
3d516aeeec
Give StackPushResult enumerators shorter names
2016-03-03 10:20:05 -08:00
Max Brunsfeld
8a13b5d120
Rename StackPopResult -> StackSlice
2016-03-03 10:16:10 -08:00
Max Brunsfeld
aef7582a2a
Start using the forward move to recover from errors
...
Some unit tests passing. Corpus tests still failing
2016-03-02 21:08:42 -08:00
Max Brunsfeld
76d072545d
Include out-of-context states starting with non-terminals
2016-03-02 20:58:39 -08:00
Max Brunsfeld
e7abfdd373
Prevent string assertion failures from creating later memory leak errors
2016-03-02 20:58:39 -08:00
Max Brunsfeld
8c01b70ce7
Don't skip tokens that are not the start of any non-terminal
2016-03-02 20:56:05 -08:00
Max Brunsfeld
dee1f697c1
Compute the set of variables that can begin with each terminal symbol
2016-02-25 21:51:52 -08:00
Max Brunsfeld
5a34d74702
Clean up stack
2016-02-25 21:51:39 -08:00
Max Brunsfeld
da2ef7ad35
Store trees in the links between stack nodes, not in the nodes themselves
2016-02-23 17:35:50 -08:00
Max Brunsfeld
b99c257e9e
Make test symbol_names array accessible in specs
2016-02-23 00:08:20 -08:00
Max Brunsfeld
9d5a260e5b
🔥 unused variable in tree spec
2016-02-22 21:29:46 -08:00
Max Brunsfeld
f444a715fd
Clean up tree array assertions in stack spec
2016-02-22 09:23:25 -08:00
Max Brunsfeld
b113dc8b0f
Return a TreeArray from ts_stack_pop
...
Since the capacity is now included in the return value, the buffer
can be reused in the ts_parser__accept function. Also, it's just
cleaner to use Array consistently, rather than a separate buffer
and size.
2016-02-21 22:31:13 -08:00
Max Brunsfeld
17db46eded
Add test for node that has only one symbol
2016-02-20 13:38:39 -08:00
Max Brunsfeld
2b35890bbb
Add ts_node_symbols() function
2016-02-19 15:41:30 -08:00
Max Brunsfeld
df1d9b2416
Move sexp-generation code to node.c
2016-02-19 15:41:09 -08:00
Max Brunsfeld
3d7df851b5
Rename Vector -> Array
2016-02-17 20:41:29 -08:00
Max Brunsfeld
6fa7eca966
Make vector struct type-safe
2016-02-17 15:30:47 -08:00
Max Brunsfeld
3f08bfb264
Fix build warnings
2016-02-12 14:11:11 -08:00
Max Brunsfeld
e90a425618
Only return one result for each revealed head from ts_stack_pop
2016-02-08 12:08:15 -08:00
Max Brunsfeld
b80a330a74
Fix assorted memory leaks in test code
2016-02-05 12:23:54 -08:00
Max Brunsfeld
b1c8b74e9c
Avoid leak caused by earlier corpus spec failures
2016-02-05 11:35:40 -08:00
Max Brunsfeld
2f2ca401be
Fix last few leaks, add leak checking to all randomized specs
2016-02-05 11:18:22 -08:00
Max Brunsfeld
c96c4a08e6
Use an object pool for stack nodes, to reduce allocations
...
Also, fix some leaks in the case where memory allocation failed during parsing
2016-02-04 11:19:42 -08:00
Max Brunsfeld
4a7312e514
Fix memory leaks when editing
2016-02-02 12:22:29 -08:00
Max Brunsfeld
e3d65d5cbd
Change name of generated parser files
...
The previous name, parser.c made it harder to set breakpoints inside of
src/runtime/parser.c in LLDB
2016-01-29 17:33:15 -08:00
Max Brunsfeld
7c44b0e387
Fix leaked lookahead trees in normal parsing
2016-01-29 17:31:43 -08:00
Max Brunsfeld
0cf59913ae
Fix double retain of child trees
2016-01-28 23:15:22 -08:00
Max Brunsfeld
5f27550a7a
Fix leaked nodes in stack
2016-01-28 21:18:57 -08:00
Max Brunsfeld
6401a065ae
Use different types for advance and accept-token actions
...
Unlike with parse actions, lexical actions of different types never appear
in the same places in the table
2016-01-22 22:24:11 -07:00
