traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1818 commits 16 branches 98 tags 37 MiB
Commit graph

177 commits

Author SHA1 Message Date
Max Brunsfeld
27c9cb4175 Show anonymous tokens in quotes in dot graphs 
Signed-off-by: Nathan Sobo <nathan@github.com>
 2016-08-29 16:39:14 -07:00
Max Brunsfeld
8c26d99353 Store error recovery actions in the normal parse table 2016-06-27 14:07:47 -07:00
Max Brunsfeld
9538b5b879 Don't count extra trees toward stack versions' error costs 2016-06-26 22:46:40 -07:00
Max Brunsfeld
76975f56ec Display node positions in tooltips in debugging graphs 2016-06-15 11:06:00 -07:00
Max Brunsfeld
f69d709650 Remove unused functions 2016-06-15 10:17:54 -07:00
Max Brunsfeld
ecc7399ed3 Fix stack breakdown procedure when there are trailing extra tokens 2016-06-14 20:25:33 -07:00
Max Brunsfeld
2109f0ed74 Handle allocation failures when copying tree arrays 2016-06-14 14:46:49 -07:00
Max Brunsfeld
f77c08eff5 Check for better stack versions earlier when handling errors 2016-06-12 17:41:25 -07:00
Max Brunsfeld
2b80e66188 Don't merge stack versions with different error costs 2016-06-12 17:27:08 -07:00
Max Brunsfeld
00a0939504 Abort erroneous parse versions more eagerly 2016-06-02 14:04:48 -07:00
Max Brunsfeld
1e42e68098 Handle cases where both valid and incomplete reduction paths end at the same stack node 2016-05-30 14:08:42 -07:00
Max Brunsfeld
ea47fdc0fe Rework logic for when to abandon parses with errors 2016-05-29 22:36:47 -07:00
Max Brunsfeld
6535704870 Replace stack_merge_new function with two simpler functions 
- merge(version1, version2)
- split(version)
 2016-05-28 21:22:10 -07:00
Max Brunsfeld
e686478ad2 Rename stack_merge function to stack_merge_all 2016-05-28 20:24:08 -07:00
Max Brunsfeld
1e353381ff Don't create error node in lexer unless token is completely invalid 
Before, any syntax error would cause the lexer to create an error
leaf node. This could happen even with a valid input, if the parse
stack had split and one particular version of the parse stack
failed to parse.

Now, an error leaf node is only created when the lexer cannot understand
part of the input stream at all. When a normal syntax error occurs,
the lexer just returns a token that is outside of the expected token
set, and the parser handles the unexpected token.
 2016-05-26 14:15:10 -07:00
Max Brunsfeld
7c859a07bb Allow null trees in the stack 2016-05-26 13:34:57 -07:00
Max Brunsfeld
a3679fbb1f Distinguish separators from main tokens via a property on transitions 
It was incorrect to store it as a property on the lexical states themselves
 2016-05-19 16:27:25 -07:00
Max Brunsfeld
77e0e4bb16 Fix some leaks after allocation failures occur 2016-05-16 10:53:31 -07:00
Max Brunsfeld
88053cf723 In tests, don’t record allocations while printing debug graphs 2016-05-16 10:44:19 -07:00
Max Brunsfeld
d50f6a58cc Abort parse versions w/ worse errors when repairing an error 2016-05-16 10:33:19 -07:00
Max Brunsfeld
22c550c9d6 Discard tokens after error detection to find the best repair 
* Use GLR stack-splitting to try all numbers of tokens to
  discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
  rather than the statically-computed 'in-progress symbols' list
 2016-05-11 13:49:43 -07:00
Max Brunsfeld
9d247e45b2 Deemphasize extra trees in stack debugging graphs 2016-05-01 15:24:50 -07:00
Max Brunsfeld
31f6b2e24a Refactor construction of out-of-context states 2016-04-25 21:59:40 -07:00
Max Brunsfeld
e99a3925e0 Merge all versions created in a given reduce operation 2016-04-24 00:55:19 -07:00
Max Brunsfeld
fe74c6fb34 Explicitly mark stack versions in debugging graphs 2016-04-24 00:54:59 -07:00
Max Brunsfeld
fd4c33209e Select ambiguous alternatives by minimizing error size 2016-04-24 00:54:20 -07:00
Max Brunsfeld
73c99e9926 Fix returned stack version when popping halts at an error 2016-04-18 11:17:07 -07:00
Max Brunsfeld
d55df29e49 Remove unused constant 2016-04-15 21:33:57 -07:00
Max Brunsfeld
316aba76ee Reorder stack methods 2016-04-15 21:33:31 -07:00
Max Brunsfeld
cad663b144 Consider multiple error repairs on the same path of the stack 
This changes the API to the stack_iterate function so that you can pop
from the stack without stopping iteration
 2016-04-15 21:28:00 -07:00
Max Brunsfeld
3778c63277 Clean up stack pop code 2016-04-11 23:12:50 -07:00
Max Brunsfeld
9d87bc4517 Rename stack successors to links 2016-04-11 22:41:06 -07:00
Max Brunsfeld
695be5bc79 Merge equivalent stacks in a separate stage of parsing 
* No more automatic merging every time a state is pushed to the stack
* When popping from the stack, the current version is always preserved
 2016-04-10 14:12:24 -07:00
Max Brunsfeld
5ba40f15ad Rename stack heads to versions 2016-04-04 12:25:57 -07:00
Max Brunsfeld
b1a696085a Clean up stack pop functions 2016-04-04 11:59:10 -07:00
Max Brunsfeld
2f3e92c9be Add function for popping all nodes from the stack 2016-04-04 11:44:45 -07:00
Max Brunsfeld
267092940d Collapse redundant interior error nodes 2016-04-03 23:46:43 -07:00
Max Brunsfeld
849ecf0a13 Display pending stack links as dashed lines 2016-04-02 23:02:21 -07:00
Max Brunsfeld
91e3609fbf Write to file directly from stack debugging function 2016-04-02 22:18:44 -07:00
Max Brunsfeld
6bce6da1e6 Store verifying flag within parse stack 2016-03-31 12:03:21 -07:00
Max Brunsfeld
523923bd93 Ensure reduce only produces one slice when stopping at errors 2016-03-10 21:26:21 -08:00
Max Brunsfeld
be707970da Fix always_inline attribute usage 2016-03-10 13:10:46 -08:00
Max Brunsfeld
e7d3d40a59 Explicitly inform stack pop callback when the stack is exhausted 
Also, pass non-extra tree count as a single value, rather than keeping
track of the extra count and the total separately.
 2016-03-10 11:51:55 -08:00
Max Brunsfeld
a8774831e8 Fix buffer overflow when limiting stack node's successor count 2016-03-10 11:43:13 -08:00
Max Brunsfeld
2e35587161 Use new stack_pop_until function for repairing errors 2016-03-07 20:06:46 -08:00
Max Brunsfeld
4348eb89d4 Expose lower stack nodes via pop_until() function 
This callback-based API allows the parser to easily visit each interior node
of the stack when searching for an error repair. It also is a better abstraction
over the stack's DAG implementation than having the public functions for
accessing entries and their successor entries.
 2016-03-07 16:09:34 -08:00
Max Brunsfeld
c0595c21c5 Halt stack pops at all error states, not just error trees 2016-03-03 11:05:37 -08:00
Max Brunsfeld
3d516aeeec Give StackPushResult enumerators shorter names 2016-03-03 10:20:05 -08:00
Max Brunsfeld
8a13b5d120 Rename StackPopResult -> StackSlice 2016-03-03 10:16:10 -08:00
Max Brunsfeld
aef7582a2a Start using the forward move to recover from errors 
Some unit tests passing. Corpus tests still failing
 2016-03-02 21:08:42 -08:00