traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
973 commits 16 branches 98 tags 37 MiB
Commit graph

74 commits

Author SHA1 Message Date
Max Brunsfeld
1c6ad5f7e4 Rename ubiquitous_tokens -> extra_tokens in compiler API 
They were already called this in the runtime code.
'Extra' is just easier to say.
 2015-12-17 15:50:50 -08:00
Max Brunsfeld
351b4f4aaa Remove unused parameters to ParseConflictManager 2015-12-17 15:19:00 -08:00
Max Brunsfeld
c495076adb Record in parse table which actions can hide splits 
Suppose a parse state S has multiple actions for a terminal lookahead symbol A.
Then during incremental parsing, while in state S, the parser should not
reuse a non-terminal lookahead B where FIRST(B) contains A, because reusing B
might prematurely discard one of the possible actions that a batch parser
would have attempted in state S, upon seeing A as a lookahead.
 2015-12-17 13:11:56 -08:00
Max Brunsfeld
77a94a2929 Use ::count to check if sets and maps contain elements 2015-12-17 10:05:42 -08:00
Max Brunsfeld
75f31a79a3 Treat reduce actions with different production IDs as distinct 2015-12-10 13:00:26 -08:00
Max Brunsfeld
26dad87299 Always mark reduce actions as fragile when they're discarded due to precedence 2015-12-06 14:09:24 -08:00
Max Brunsfeld
ad619d95f6 Add 'extra' field to symbol metadata 
This stores whether a symbol is only ever used as a ubiquitous token. This will
allow ubiquitous nodes to be reused more effectively: if they are always
ubiquitous, then they can be reused immediately, and otherwise, they must be
broken down in case they need to be used structurally.
 2015-12-02 15:10:24 -08:00
Max Brunsfeld
c8be143f65 🔥 get_metadata function 2015-10-30 16:22:25 -07:00
Max Brunsfeld
73b3280fbb Include precedence calculation in LexItemSet::transitions 2015-10-30 16:07:29 -07:00
Max Brunsfeld
e9be0ff24e Make completion_status() a method on ParseItem 2015-10-30 14:07:33 -07:00
Max Brunsfeld
4850384b78 Include precedence calculation in ParseItemSet::transitions 2015-10-30 13:54:11 -07:00
Max Brunsfeld
b7e0cb1fc6 Revert 'Add reduce-extra actions for all symbols' 
It only makes sense to add reduce-extra actions for those symbols that
have actions in the state the will be reduced to
 2015-10-26 17:19:04 -07:00
Max Brunsfeld
1babdc43e1 Add reduce-extra actions for all symbols 2015-10-24 12:49:26 -07:00
Max Brunsfeld
44d3e4228e Show precedences and associativities in parse conflicts 2015-10-24 12:49:04 -07:00
Max Brunsfeld
1983bcfb60 Fix conflation of finished items w/ different precedence 2015-10-18 12:51:32 -07:00
Max Brunsfeld
8725e96a65 Fix item-set-closure bug w/ empty productions 2015-10-15 23:59:47 -07:00
Max Brunsfeld
02e549202f Improve parse conflict reporting 2015-10-14 22:14:08 -07:00
Max Brunsfeld
9959fe35b0 Allow associativity to be specified in rules w/o precedence 2015-10-13 11:25:28 -07:00
Max Brunsfeld
5e4bdcbaf8 Simplify handling of precedence & associativity in productions 2015-10-05 16:56:11 -07:00
Max Brunsfeld
6d748a6714 Store parse actions' precedences as ranges, not sets 2015-10-05 16:05:19 -07:00
Max Brunsfeld
ef2acf9496 Make ParseItemSet & LexItemSet classes 2015-10-05 15:13:43 -07:00
Max Brunsfeld
f01972c64e Reorganize ParseItemSet and LexItemSet 2015-10-05 15:09:06 -07:00
Max Brunsfeld
c4ef228397 Share common lookahead sets between parse item sets 2015-10-04 21:33:54 -07:00
Max Brunsfeld
a0bf3d0bd8 Compute closures of item sets lazily 2015-10-04 00:21:29 -07:00
Max Brunsfeld
ebc52f109d Merge branch 'flatten-rules-into-productions' 
This branch had diverged considerably, so merging it required changing a lot
of code.

Conflicts:
	project.gyp
	spec/compiler/build_tables/action_takes_precedence_spec.cc
	spec/compiler/build_tables/build_conflict_spec.cc
	spec/compiler/build_tables/build_parse_table_spec.cc
	spec/compiler/build_tables/first_symbols_spec.cc
	spec/compiler/build_tables/item_set_closure_spec.cc
	spec/compiler/build_tables/item_set_transitions_spec.cc
	spec/compiler/build_tables/rule_can_be_blank_spec.cc
	spec/compiler/helpers/containers.h
	spec/compiler/prepare_grammar/expand_repeats_spec.cc
	spec/compiler/prepare_grammar/extract_tokens_spec.cc
	src/compiler/build_tables/action_takes_precedence.h
	src/compiler/build_tables/build_parse_table.cc
	src/compiler/build_tables/first_symbols.cc
	src/compiler/build_tables/first_symbols.h
	src/compiler/build_tables/item_set_closure.cc
	src/compiler/build_tables/item_set_transitions.cc
	src/compiler/build_tables/parse_item.cc
	src/compiler/build_tables/parse_item.h
	src/compiler/build_tables/rule_can_be_blank.cc
	src/compiler/build_tables/rule_can_be_blank.h
	src/compiler/prepare_grammar/expand_repeats.cc
	src/compiler/prepare_grammar/extract_tokens.cc
	src/compiler/prepare_grammar/extract_tokens.h
	src/compiler/prepare_grammar/prepare_grammar.cc
	src/compiler/rules/built_in_symbols.cc
	src/compiler/rules/built_in_symbols.h
	src/compiler/syntax_grammar.cc
	src/compiler/syntax_grammar.h
 2015-10-02 23:46:39 -07:00
Max Brunsfeld
673ca411b1 Fix lint errors 2015-09-19 13:19:49 -07:00
Max Brunsfeld
5982b77c97 In compiler, distinguish between anonymous tokens and hidden rules 2015-09-05 22:28:55 -07:00
Max Brunsfeld
97bb7a26cf Fix precedence calculations when building parse table 
* Recurse into choice rules
* Compute reduction precedence differently than shift precedence
 2015-09-02 13:05:54 -07:00
Max Brunsfeld
21258e6a9e Remove 'document' wrapper node 2015-08-22 10:48:34 -07:00
Max Brunsfeld
f9b057f3a9 clang-format everything 2015-07-27 18:29:48 -07:00
Max Brunsfeld
0b1d70db34 Always resolve ambiguities immediately 
No more ambiguity nodes.
Also, when merging parse stacks, merge their successors if needed.
 2015-07-15 13:15:11 -07:00
Max Brunsfeld
929897a78e Fix missing include 2015-07-09 23:09:46 -07:00
Max Brunsfeld
aabcb10cfb Respect expected_conflicts field when building parse table 2015-06-28 16:22:31 -05:00
Max Brunsfeld
381f89f8ba Create ambiguity nodes when joining stack heads 2015-06-18 17:03:16 -07:00
Max Brunsfeld
a54d293317 Include 'goal' non-terminal names in conflict description 2015-04-16 17:43:30 -07:00
Max Brunsfeld
80ec303b10 Replace prec rule w/ left_assoc and right_assoc 
Consider shift/reduce conflicts to be compilation errors unless
they are resolved by a specified associativity.
 2015-03-16 23:12:34 -07:00
Max Brunsfeld
c59c98251b Fix misplaced include and using directive 2015-03-16 18:33:11 -07:00
Max Brunsfeld
7872ddd21b Improve reduce/reduce conflict error message 2015-03-16 11:59:45 -07:00
Max Brunsfeld
9a198562e0 Treat parse conflicts as errors in grammar compilation 
For now, only reduce/reduce conflicts w/ no tie-breaking precedence
are treated as errors. The rest are dropped, because shift/reduce
conflicts are currently very common because we don't have a way
of specifying associativity along w/ precedence.
 2015-03-15 20:31:41 -07:00
Max Brunsfeld
2d436cf141 Identify fragile reductions at compile time 2015-02-21 15:11:03 -08:00
Max Brunsfeld
ccb0d0d043 Store parse items' list of consumed symbols 
Not just the consumed symbol count.
 2015-02-20 22:53:27 -08:00
Max Brunsfeld
8ac4b9fc17 Store productions' end rule ids in the vector 2015-02-16 22:11:03 -08:00
Max Brunsfeld
1ba8701ada Compute fewer item set closures in item set transitions function 2015-02-16 22:11:03 -08:00
Max Brunsfeld
52daffb3f3 Separate syntax rules into flat lists of symbols 
This way, every ParseItem can be associated with a particular production
for its non-terminal. That lets us keep track of which productions are
involved in shift/reduce conflicts.
 2015-02-16 22:11:03 -08:00
Max Brunsfeld
109b5616d3 Remove unused arg to action_takes_precedence 2015-01-17 14:14:49 -08:00
Max Brunsfeld
34d96909d1 Move {Syntax,Lexical}Grammar into separate files 2015-01-14 21:10:41 -08:00
Max Brunsfeld
5dc08ccce9 Include names of in-progress rules in shift/reduce conflicts 2014-11-05 18:39:50 -08:00
Max Brunsfeld
8379d9387c Tidy up build_parse_table function 2014-10-12 12:56:04 -07:00
Max Brunsfeld
aae6f6de14 Remove whitespace between template closing tags 2014-10-12 11:51:12 -07:00
Max Brunsfeld
3bcb221379 Include non-terminal lookahead symbols for reduction actions 
This is necessary for re-using the right subtree after an edit
 2014-10-10 12:06:16 -07:00