traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1313 commits 16 branches 98 tags 37 MiB
Commit graph

279 commits

Author SHA1 Message Date
Max Brunsfeld
6cf4ccb840 Represent rule metadata as a struct, not a map 2016-11-19 13:59:34 -08:00
Max Brunsfeld
cab1bd3ac5 Make conflict messages explicit about precedence combinations 2016-11-18 17:05:16 -08:00
Max Brunsfeld
5924285e69 🎨 2016-11-18 16:14:05 -08:00
Max Brunsfeld
32387400c6 Rework LR conflict resolution 
* Unify precedence/associativity-based resolution with the
  search for a whitelisted conflict
* Improve conflict error messages
 2016-11-18 13:50:55 -08:00
Max Brunsfeld
6935f1d26f Use hash_combine everywhere 2016-11-16 11:46:22 -08:00
Max Brunsfeld
6cfd009503 Compute parse state group signature based on the item set 2016-11-16 10:21:30 -08:00
Max Brunsfeld
42d37656ea Optimize remove_duplicate_parse_states method 
Signed-off-by: Nathan Sobo <nathan@github.com>
 2016-11-15 17:51:52 -08:00
Max Brunsfeld
1118a9142a Introduce Symbol::Index type alias 2016-11-14 10:25:26 -08:00
Max Brunsfeld
fad7294ba4 Store shift states for non-terminals directly in the main parse table 2016-11-14 08:36:06 -08:00
Max Brunsfeld
8d9c261e3a Don't include reduce actions for nonterminal lookaheads 2016-11-10 11:33:37 -08:00
Max Brunsfeld
255bc2427c 🎨 build_parse_table 2016-11-09 20:47:47 -08:00
Timothy Clem
693c6d40dd Move setup of mergeable_symbols to constructor, use set throughout 2016-10-18 15:18:33 -07:00
Timothy Clem
14bae584d4 WIP: New check for mergable symbols in merge_state 2016-10-18 13:03:41 -07:00
Max Brunsfeld
e149d94ff5 Remove generated parsers' dependency on runtime.h 2016-10-05 14:02:49 -07:00
Max Brunsfeld
b76574e01c Handle ambiguities between extra and non-extra tokens using normal GLR splitting 2016-09-06 10:22:16 -07:00
Max Brunsfeld
0ee1994078 Don't have both shift and shift-extra actions in recovery states 2016-07-17 13:35:58 -07:00
Max Brunsfeld
0e2bbbd7ee Compress parse table by allowing reductions w/ unexpected lookaheads 2016-07-04 12:20:23 -07:00
Max Brunsfeld
8c26d99353 Store error recovery actions in the normal parse table 2016-06-27 14:07:47 -07:00
Max Brunsfeld
38c144b4a3 Refine logic for deciding when tokens need to be re-lexed 
* While generating the lex table, note which tokens can match the
  same string. A token needs to be relexed when it has possible
  homonyms in the current state.
* Also note which tokens can match substrings of each other tokens.
  A token needs to be relexed when there are viable tokens that
  could match longer strings in the current state and the next
  token has been edited.
* Remove the logic for marking tokens as fragile on creation.
* Store the reusability/non-reusability of symbols off of individual
  actions and onto the entire entry for the state & symbol.
 2016-06-21 07:28:04 -07:00
Max Brunsfeld
45f7cee0c8 Handle extra tokens properly during error recovery 2016-06-18 20:46:25 -07:00
Max Brunsfeld
6d40e317df Ensure that reductions are ordered by child count in parse table 2016-06-10 13:11:52 -07:00
Max Brunsfeld
a3679fbb1f Distinguish separators from main tokens via a property on transitions 
It was incorrect to store it as a property on the lexical states themselves
 2016-05-19 16:27:25 -07:00
Max Brunsfeld
59712ec492 Clean up lex table generation 2016-05-19 13:25:46 -07:00
Max Brunsfeld
507d5ad9f7 Include shift-extra actions alongside other actions in recovery states 2016-05-16 10:33:18 -07:00
Max Brunsfeld
19bd09b81d Don't include accept actions in recovery states 2016-05-11 14:02:26 -07:00
Max Brunsfeld
22c550c9d6 Discard tokens after error detection to find the best repair 
* Use GLR stack-splitting to try all numbers of tokens to
  discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
  rather than the statically-computed 'in-progress symbols' list
 2016-05-11 13:49:43 -07:00
Max Brunsfeld
5b74813a5c Refine logic for which tokens to use in error recovery 2016-04-27 14:09:19 -07:00
Max Brunsfeld
31f6b2e24a Refactor construction of out-of-context states 2016-04-25 21:59:40 -07:00
Max Brunsfeld
cad663b144 Consider multiple error repairs on the same path of the stack 
This changes the API to the stack_iterate function so that you can pop
from the stack without stopping iteration
 2016-04-15 21:28:00 -07:00
Max Brunsfeld
9657dfcfc3 Compute in-progress symbols for out-of-context states 2016-03-10 11:39:44 -08:00
Max Brunsfeld
b733b0cc81 Remove duplicate parse actions 
This has only come up for out-of-context states, but it seems possible now that there
could be duplicate actions for any state, because of the possibility of multiple
actions with different precedence or associativity that are otherwise the same
 2016-03-02 20:58:40 -08:00
Max Brunsfeld
b68f7212c8 Do not consider any symbols to be 'in-progress' in out-of-context states 2016-03-02 20:58:39 -08:00
Max Brunsfeld
76d072545d Include out-of-context states starting with non-terminals 2016-03-02 20:58:39 -08:00
Max Brunsfeld
e0c24e3be6 Remove old error recovery code 2016-03-02 20:58:39 -08:00
Max Brunsfeld
ffcd8b5c49 Generate C code for the in-progress symbols in each parse state 2016-03-02 20:56:05 -08:00
Max Brunsfeld
00d953f507 Generate C code for out-of-context states 2016-03-02 20:56:05 -08:00
Max Brunsfeld
8c01b70ce7 Don't skip tokens that are not the start of any non-terminal 2016-03-02 20:56:05 -08:00
Max Brunsfeld
b4f2407a49 Add forward move states for each terminal symbol 2016-03-02 20:56:04 -08:00
Max Brunsfeld
dee1f697c1 Compute the set of variables that can begin with each terminal symbol 2016-02-25 21:51:52 -08:00
Max Brunsfeld
3f08bfb264 Fix build warnings 2016-02-12 14:11:11 -08:00
Max Brunsfeld
6401a065ae Use different types for advance and accept-token actions 
Unlike with parse actions, lexical actions of different types never appear
in the same places in the table
 2016-01-22 22:24:11 -07:00
Max Brunsfeld
49f393b75e Merge pull request #22 from maxbrunsfeld/c-compiler-api 
Simplify the compiler API
 2016-01-13 21:08:41 -08:00
Max Brunsfeld
d4632ab9a9 Make the compile function plain C and take a JSON grammar 2016-01-11 12:33:48 -08:00
Max Brunsfeld
e59f6294cb Fix bug in lexical state de-duping 2015-12-30 11:15:36 -08:00
Max Brunsfeld
939476c947 When removing duplicate lex states, update the error state too 
Now, instead of being stored as a separate field on the parse table, the error
state is just the first state in the states vector.
 2015-12-29 21:02:24 -08:00
Max Brunsfeld
a8f50986e0 clang-format 2015-12-24 22:05:54 -08:00
Max Brunsfeld
386b124866 Ensure that there are no duplicate lex states 2015-12-20 15:46:13 -08:00
Max Brunsfeld
1c6ad5f7e4 Rename ubiquitous_tokens -> extra_tokens in compiler API 
They were already called this in the runtime code.
'Extra' is just easier to say.
 2015-12-17 15:50:50 -08:00
Max Brunsfeld
f065eb0480 Remove unused parameter to LexConflictManager 2015-12-17 15:45:47 -08:00
Max Brunsfeld
a8d2585330 Fix resolution of shift-extra vs reduce actions 2015-12-17 15:19:58 -08:00