traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1600 commits 16 branches 98 tags 37 MiB
Commit graph

65 commits

Author SHA1 Message Date
Max Brunsfeld
cb5fe80348 Rename RENAME rule to ALIAS, allow it to create anonymous nodes 2017-07-31 16:41:11 -07:00
Max Brunsfeld
cbdfd89675 Mark reductions as fragile based on their final properties 
We previously maintained a set of individual productions that were
involved in conflicts, but that was subtly incorrect because
we don't compare productions themselves when comparing parse items;
we only compare the parse items properties that could affect the
final reduce actions.
 2017-07-21 09:54:24 -07:00
Max Brunsfeld
7d9d8bce79 Handle inlined rules that contain other inlined rules 2017-07-20 15:29:06 -07:00
Max Brunsfeld
b3a72954ff Introduce RENAME rule type 2017-07-13 17:17:22 -07:00
Max Brunsfeld
561821d011 Remove precedence and associativity methods from ParseAction 2017-07-13 13:41:56 -07:00
Max Brunsfeld
d8e9d04fe7 Add PREC_DYNAMIC rule for resolving runtime ambiguities 2017-07-06 15:24:45 -07:00
Phil Turnbull
c58f6401d0 Non-terminal entries always have valid state-ids 2017-06-14 08:49:38 -04:00
Phil Turnbull
577e43f653 shift-extra actions do not have valid state_ids 2017-06-09 16:26:01 -04:00
Phil Turnbull
18ba6ebbd7 Move state_id check into each_referenced_state 2017-06-09 16:25:59 -04:00
Phil Turnbull
dee86f908a Correctly check type is ParseActionTypeRecover 2017-06-07 17:05:39 -04:00
Max Brunsfeld
db4b9ebc7c Implement Rule as a union rather than an abstract base class 2017-03-17 13:29:31 -07:00
Max Brunsfeld
abf8a4f2c2 🎨 2017-03-01 22:15:26 -08:00
Max Brunsfeld
3c8e6f9987 Restructure parse state merging logic 
* Remove remnants of templatized remove_duplicate_states function
* Rename recovery_tokens function to get_compatible_tokens and augment it
  also compute pairs of tokens which could potentially be incompatible
 2017-02-26 12:23:48 -08:00
Max Brunsfeld
c966af0412 Start work on external tokens 2016-12-02 16:24:19 -08:00
Max Brunsfeld
5924285e69 🎨 2016-11-18 16:14:05 -08:00
Max Brunsfeld
32387400c6 Rework LR conflict resolution 
* Unify precedence/associativity-based resolution with the
  search for a whitelisted conflict
* Improve conflict error messages
 2016-11-18 13:50:55 -08:00
Max Brunsfeld
1118a9142a Introduce Symbol::Index type alias 2016-11-14 10:25:26 -08:00
Max Brunsfeld
fad7294ba4 Store shift states for non-terminals directly in the main parse table 2016-11-14 08:36:06 -08:00
Max Brunsfeld
3b3fddd64d Relax overly conservative parse state mergeability check 
Built-in symbols (e.g. EOF, ERROR) should not prevent parse states from being
merged. Neither should non-token productions.
 2016-10-26 21:58:15 -07:00
Timothy Clem
14bae584d4 WIP: New check for mergable symbols in merge_state 2016-10-18 13:03:41 -07:00
Max Brunsfeld
e149d94ff5 Remove generated parsers' dependency on runtime.h 2016-10-05 14:02:49 -07:00
Max Brunsfeld
b76574e01c Handle ambiguities between extra and non-extra tokens using normal GLR splitting 2016-09-06 10:22:16 -07:00
Max Brunsfeld
0e2bbbd7ee Compress parse table by allowing reductions w/ unexpected lookaheads 2016-07-04 12:20:23 -07:00
Max Brunsfeld
8c26d99353 Store error recovery actions in the normal parse table 2016-06-27 14:07:47 -07:00
Max Brunsfeld
877fe1f682 Fix incorrect exta entry in symbol metadata table 2016-06-26 22:14:31 -07:00
Max Brunsfeld
38c144b4a3 Refine logic for deciding when tokens need to be re-lexed 
* While generating the lex table, note which tokens can match the
  same string. A token needs to be relexed when it has possible
  homonyms in the current state.
* Also note which tokens can match substrings of each other tokens.
  A token needs to be relexed when there are viable tokens that
  could match longer strings in the current state and the next
  token has been edited.
* Remove the logic for marking tokens as fragile on creation.
* Store the reusability/non-reusability of symbols off of individual
  actions and onto the entire entry for the state & symbol.
 2016-06-21 07:28:04 -07:00
Max Brunsfeld
b4f2407a49 Add forward move states for each terminal symbol 2016-03-02 20:56:04 -08:00
Max Brunsfeld
6401a065ae Use different types for advance and accept-token actions 
Unlike with parse actions, lexical actions of different types never appear
in the same places in the table
 2016-01-22 22:24:11 -07:00
Max Brunsfeld
f0b1d851ce Fix uninitialized instance variable in ParseAction 2016-01-21 23:52:05 -07:00
Max Brunsfeld
386b124866 Ensure that there are no duplicate lex states 2015-12-20 15:46:13 -08:00
Max Brunsfeld
c495076adb Record in parse table which actions can hide splits 
Suppose a parse state S has multiple actions for a terminal lookahead symbol A.
Then during incremental parsing, while in state S, the parser should not
reuse a non-terminal lookahead B where FIRST(B) contains A, because reusing B
might prematurely discard one of the possible actions that a batch parser
would have attempted in state S, upon seeing A as a lookahead.
 2015-12-17 13:11:56 -08:00
Max Brunsfeld
75f31a79a3 Treat reduce actions with different production IDs as distinct 2015-12-10 13:00:26 -08:00
Max Brunsfeld
ad619d95f6 Add 'extra' field to symbol metadata 
This stores whether a symbol is only ever used as a ubiquitous token. This will
allow ubiquitous nodes to be reused more effectively: if they are always
ubiquitous, then they can be reused immediately, and otherwise, they must be
broken down in case they need to be used structurally.
 2015-12-02 15:10:24 -08:00
Max Brunsfeld
1983bcfb60 Fix conflation of finished items w/ different precedence 2015-10-18 12:51:32 -07:00
Max Brunsfeld
9959fe35b0 Allow associativity to be specified in rules w/o precedence 2015-10-13 11:25:28 -07:00
Max Brunsfeld
4b817dc07c Fix linter errors 2015-10-12 19:22:05 -07:00
Max Brunsfeld
6d748a6714 Store parse actions' precedences as ranges, not sets 2015-10-05 16:05:19 -07:00
Max Brunsfeld
ebc52f109d Merge branch 'flatten-rules-into-productions' 
This branch had diverged considerably, so merging it required changing a lot
of code.

Conflicts:
	project.gyp
	spec/compiler/build_tables/action_takes_precedence_spec.cc
	spec/compiler/build_tables/build_conflict_spec.cc
	spec/compiler/build_tables/build_parse_table_spec.cc
	spec/compiler/build_tables/first_symbols_spec.cc
	spec/compiler/build_tables/item_set_closure_spec.cc
	spec/compiler/build_tables/item_set_transitions_spec.cc
	spec/compiler/build_tables/rule_can_be_blank_spec.cc
	spec/compiler/helpers/containers.h
	spec/compiler/prepare_grammar/expand_repeats_spec.cc
	spec/compiler/prepare_grammar/extract_tokens_spec.cc
	src/compiler/build_tables/action_takes_precedence.h
	src/compiler/build_tables/build_parse_table.cc
	src/compiler/build_tables/first_symbols.cc
	src/compiler/build_tables/first_symbols.h
	src/compiler/build_tables/item_set_closure.cc
	src/compiler/build_tables/item_set_transitions.cc
	src/compiler/build_tables/parse_item.cc
	src/compiler/build_tables/parse_item.h
	src/compiler/build_tables/rule_can_be_blank.cc
	src/compiler/build_tables/rule_can_be_blank.h
	src/compiler/prepare_grammar/expand_repeats.cc
	src/compiler/prepare_grammar/extract_tokens.cc
	src/compiler/prepare_grammar/extract_tokens.h
	src/compiler/prepare_grammar/prepare_grammar.cc
	src/compiler/rules/built_in_symbols.cc
	src/compiler/rules/built_in_symbols.h
	src/compiler/syntax_grammar.cc
	src/compiler/syntax_grammar.h
 2015-10-02 23:46:39 -07:00
Max Brunsfeld
e6f3239bef Move stream operator definitions to spec helpers 
This is one less thing for users to worry about when compiling and linking
the library itself
 2015-09-10 10:12:11 -07:00
Max Brunsfeld
bd77ab1ac9 Move public rule functions out of rule namespace 
This way, there's only one public namespace: tree_sitter
 2015-09-03 17:49:20 -07:00
Max Brunsfeld
c18351772a Auto-format: no single-line functions 2015-07-31 16:32:24 -07:00
Max Brunsfeld
f9b057f3a9 clang-format everything 2015-07-27 18:29:48 -07:00
Max Brunsfeld
aabcb10cfb Respect expected_conflicts field when building parse table 2015-06-28 16:22:31 -05:00
Max Brunsfeld
80ec303b10 Replace prec rule w/ left_assoc and right_assoc 
Consider shift/reduce conflicts to be compilation errors unless
they are resolved by a specified associativity.
 2015-03-16 23:12:34 -07:00
Max Brunsfeld
3458fa6e50 Fix non-deterministic order in conflict description 2015-03-07 11:02:21 -08:00
Max Brunsfeld
2d436cf141 Identify fragile reductions at compile time 2015-02-21 15:11:03 -08:00
Max Brunsfeld
2e7ffb4d14 Tweak auto-format settings 
Prefer lines that exceed 80 characters by a small margin to
line breaks in argument lists
 2014-09-09 13:15:40 -07:00
Max Brunsfeld
98cc2f2264 Auto-format all source code with clang-format 2014-07-21 13:20:00 -07:00
Max Brunsfeld
f4287c07d0 Fix ParseStateId / size_t confusion in parse table 2014-07-07 13:21:30 -07:00
Max Brunsfeld
67590eddc7 Fix stream operator for new parse action types 2014-07-03 08:15:36 -07:00