traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
983 commits 16 branches 98 tags 37 MiB
Commit graph

157 commits

Author SHA1 Message Date
Max Brunsfeld
13d8de3eb3 Remove stack head after it is accepted 2015-12-24 22:04:20 -08:00
Max Brunsfeld
f483178f83 Clean up main parser loop 2015-12-24 10:21:42 -08:00
Max Brunsfeld
7f1c5f7568 Don't use preprocesser for special tree state values 2015-12-22 14:37:29 -08:00
Max Brunsfeld
da1bc038e5 Remove nested options structs in Tree 2015-12-22 14:20:58 -08:00
Max Brunsfeld
30961143fe Only perform breakdown if next terminal node has changes 2015-12-22 14:00:22 -08:00
Max Brunsfeld
6b11890d68 Reuse fragile parent nodes that were reduced in the current state 2015-12-22 13:59:04 -08:00
Max Brunsfeld
2bcd2e4d00 Reuse fragile tokens that came from the current lex state 2015-12-21 16:04:11 -08:00
Max Brunsfeld
c495076adb Record in parse table which actions can hide splits 
Suppose a parse state S has multiple actions for a terminal lookahead symbol A.
Then during incremental parsing, while in state S, the parser should not
reuse a non-terminal lookahead B where FIRST(B) contains A, because reusing B
might prematurely discard one of the possible actions that a batch parser
would have attempted in state S, upon seeing A as a lookahead.
 2015-12-17 13:11:56 -08:00
Max Brunsfeld
7fbb628c78 Remove TreeSelectionCallback struct 
Just make a typedef for the function type
 2015-12-17 12:09:06 -08:00
Max Brunsfeld
66144dc28e Treat tokens that are sometimes extra as fragile 2015-12-16 20:04:45 -08:00
Max Brunsfeld
3c3d95345f Don't mutate shared lookahead nodes when setting the extra flag 2015-12-15 22:28:50 -08:00
Max Brunsfeld
71059f1d66 Don't reuse fragile tokens across different parse stack heads 2015-12-15 22:26:42 -08:00
Max Brunsfeld
4ca965019f Break down unverified reductions eagerly when the next node has changes 2015-12-10 21:06:22 -08:00
Max Brunsfeld
6abce098c0 Track whether parse stack was split before consuming each lookahead 2015-12-10 13:00:27 -08:00
Max Brunsfeld
924ce8c1b3 Fix tracking of lookahead positions for different parse stack heads 2015-12-08 13:02:25 -08:00
Max Brunsfeld
26e34ddcbb Allow all parse stack heads to finish 2015-12-08 13:01:33 -08:00
Max Brunsfeld
a7ca28403e Log when selecting a tree in an ambiguity 2015-12-08 12:25:41 -08:00
Max Brunsfeld
10286f307f Pass reference to parser in stack's tree selection callback 2015-12-08 12:21:27 -08:00
Max Brunsfeld
4abb415113 Handle parse stack splitting while breaking down unverified reduction 2015-12-06 21:10:47 -08:00
Max Brunsfeld
98f190d3ba When the parse stack is split, mark all reductions as fragile 2015-12-06 20:30:48 -08:00
Max Brunsfeld
738c49abc8 Log each step when breaking down the parse stack 2015-12-04 20:57:33 -08:00
Max Brunsfeld
d69fef11f9 Break down top of parse stack when needed 
Also, start randomizing the edits in the corpus specs
 2015-12-04 20:57:33 -08:00
Max Brunsfeld
08d50c25ae clang-format 2015-12-04 20:56:33 -08:00
Max Brunsfeld
d2bf88d5fe Include rows and columns in TSLength 
This way, we don't have to have separate 1D and 2D versions for so many values
 2015-12-04 20:20:29 -08:00
Max Brunsfeld
fe5286f863 Merge pull request #11 from maxbrunsfeld/move-ubiquitous-tokens-outward 
Put ubiquitous tokens as high in the tree as possible
 2015-12-03 10:06:50 -08:00
Max Brunsfeld
8a146a9bef Reset lexer correctly when old input was blank 2015-12-03 10:00:39 -08:00
Max Brunsfeld
863cabc827 Don't include trailing ubiquitous tokens as children when reducing 2015-12-02 15:31:15 -08:00
Max Brunsfeld
f08554e958 Replace NodeType enum with SymbolMetadata bitfield 
This will allow storing other metadata about symbols, like if they
only appear as ubiquitous tokens
 2015-12-02 15:10:24 -08:00
joshvera
5e748fdf63 s/ts_tree_offset_point/ts_tree_total_size_point 2015-12-02 16:41:56 -05:00
joshvera
4af3b7d0fd Add offset_point to LookaheadState 2015-11-30 12:50:16 -05:00
joshvera
f5fc247c8b Merge remote-tracking branch 'origin/master' into line-numbers 2015-11-30 12:36:11 -05:00
joshvera
ad58b752e6 Rename ts_tree_total_size_point to ts_tree_offset_point 2015-11-25 11:28:28 -05:00
Max Brunsfeld
c28db44cd9 🎨 2015-11-20 13:10:11 -08:00
Max Brunsfeld
32b1088823 Rename HeadState -> LookaheadState, remove parser.lookahead field 2015-11-20 12:55:01 -08:00
Max Brunsfeld
467930a785 Move some ts_language functions to the right file 2015-11-20 12:00:49 -08:00
Max Brunsfeld
5c95d02bd0 Move ts_tree_compare to the right file 2015-11-20 11:53:03 -08:00
Max Brunsfeld
7aba2a0716 Rename DEBUG macro to LOG 
DEBUG is already used as the symbol to enable/disable assert() calls
 2015-11-20 11:50:50 -08:00
Max Brunsfeld
8d1b9501ab Fix integer comparison warnings 2015-11-20 11:47:39 -08:00
Max Brunsfeld
c88e9044d5 Make stack popping more robust 2015-11-20 00:04:21 -08:00
Max Brunsfeld
64874449e4 Allow different parse stack heads to lex differently 2015-11-19 20:55:18 -08:00
joshvera
b0f6bac3ab replace start and end with padding and size 2015-11-18 16:34:50 -08:00
Max Brunsfeld
484721b0c2 Assign tree parent pointers after parse is complete 2015-11-15 12:21:16 -08:00
Max Brunsfeld
0824d3e1f3 Only use first parse stack path during error recovery 2015-11-14 12:37:21 -08:00
Max Brunsfeld
46e99d3e42 Retrieve the shift action correctly when reducing and recovering 2015-11-11 17:53:45 -08:00
Max Brunsfeld
40a90b551a Allow error recovery to look all the way to the bottom of the stack 
Previously, there was a bug where the first node on the stack
would never be popped
 2015-11-11 16:59:41 -08:00
Max Brunsfeld
1a5d5b3156 Make ambiguities resolve deterministically 
In the future, they should resolve according to some kind of dynamic
precedence annotations provided in the grammars. For now, this at least makes
them fully deterministic, so that tests won't fail due to ambiguities resolving
differently after undone edits.
 2015-11-11 16:54:03 -08:00
Max Brunsfeld
30b6530fd1 Account for parse stack merges when shifting 
Previously, when the parse stack was split into 3 or more heads, it was
possible for head 3 to be accidentally skipped if head 2 merged with head 1.
 2015-11-05 21:21:18 -08:00
Max Brunsfeld
216ce8c80b Rename receiver parameter to 'self' 2015-10-14 22:14:08 -07:00
Max Brunsfeld
3ee58461d7 Clean up main parser loop 2015-10-07 13:05:36 -07:00
Max Brunsfeld
8ef25ffef3 Try lexing using each parse stack head's state 
This fixes the case where the parse stack is split and the top states
have different valid lookahead symbols
 2015-10-06 16:22:58 -07:00