Max Brunsfeld
5924285e69
🎨
2016-11-18 16:14:05 -08:00
Max Brunsfeld
32387400c6
Rework LR conflict resolution
...
* Unify precedence/associativity-based resolution with the
search for a whitelisted conflict
* Improve conflict error messages
2016-11-18 13:50:55 -08:00
Max Brunsfeld
1118a9142a
Introduce Symbol::Index type alias
2016-11-14 10:25:26 -08:00
Max Brunsfeld
fad7294ba4
Store shift states for non-terminals directly in the main parse table
2016-11-14 08:36:06 -08:00
Max Brunsfeld
3b3fddd64d
Relax overly conservative parse state mergeability check
...
Built-in symbols (e.g. EOF, ERROR) should not prevent parse states from being
merged. Neither should non-token productions.
2016-10-26 21:58:15 -07:00
Timothy Clem
14bae584d4
WIP: New check for mergable symbols in merge_state
2016-10-18 13:03:41 -07:00
Max Brunsfeld
e149d94ff5
Remove generated parsers' dependency on runtime.h
2016-10-05 14:02:49 -07:00
Max Brunsfeld
b76574e01c
Handle ambiguities between extra and non-extra tokens using normal GLR splitting
2016-09-06 10:22:16 -07:00
Max Brunsfeld
0e2bbbd7ee
Compress parse table by allowing reductions w/ unexpected lookaheads
2016-07-04 12:20:23 -07:00
Max Brunsfeld
8c26d99353
Store error recovery actions in the normal parse table
2016-06-27 14:07:47 -07:00
Max Brunsfeld
877fe1f682
Fix incorrect exta entry in symbol metadata table
2016-06-26 22:14:31 -07:00
Max Brunsfeld
38c144b4a3
Refine logic for deciding when tokens need to be re-lexed
...
* While generating the lex table, note which tokens can match the
same string. A token needs to be relexed when it has possible
homonyms in the current state.
* Also note which tokens can match substrings of each other tokens.
A token needs to be relexed when there are viable tokens that
could match longer strings in the current state and the next
token has been edited.
* Remove the logic for marking tokens as fragile on creation.
* Store the reusability/non-reusability of symbols off of individual
actions and onto the entire entry for the state & symbol.
2016-06-21 07:28:04 -07:00
Max Brunsfeld
b4f2407a49
Add forward move states for each terminal symbol
2016-03-02 20:56:04 -08:00
Max Brunsfeld
6401a065ae
Use different types for advance and accept-token actions
...
Unlike with parse actions, lexical actions of different types never appear
in the same places in the table
2016-01-22 22:24:11 -07:00
Max Brunsfeld
f0b1d851ce
Fix uninitialized instance variable in ParseAction
2016-01-21 23:52:05 -07:00
Max Brunsfeld
386b124866
Ensure that there are no duplicate lex states
2015-12-20 15:46:13 -08:00
Max Brunsfeld
c495076adb
Record in parse table which actions can hide splits
...
Suppose a parse state S has multiple actions for a terminal lookahead symbol A.
Then during incremental parsing, while in state S, the parser should not
reuse a non-terminal lookahead B where FIRST(B) contains A, because reusing B
might prematurely discard one of the possible actions that a batch parser
would have attempted in state S, upon seeing A as a lookahead.
2015-12-17 13:11:56 -08:00
Max Brunsfeld
75f31a79a3
Treat reduce actions with different production IDs as distinct
2015-12-10 13:00:26 -08:00
Max Brunsfeld
ad619d95f6
Add 'extra' field to symbol metadata
...
This stores whether a symbol is only ever used as a ubiquitous token. This will
allow ubiquitous nodes to be reused more effectively: if they are always
ubiquitous, then they can be reused immediately, and otherwise, they must be
broken down in case they need to be used structurally.
2015-12-02 15:10:24 -08:00
Max Brunsfeld
1983bcfb60
Fix conflation of finished items w/ different precedence
2015-10-18 12:51:32 -07:00
Max Brunsfeld
9959fe35b0
Allow associativity to be specified in rules w/o precedence
2015-10-13 11:25:28 -07:00
Max Brunsfeld
4b817dc07c
Fix linter errors
2015-10-12 19:22:05 -07:00
Max Brunsfeld
6d748a6714
Store parse actions' precedences as ranges, not sets
2015-10-05 16:05:19 -07:00
Max Brunsfeld
ebc52f109d
Merge branch 'flatten-rules-into-productions'
...
This branch had diverged considerably, so merging it required changing a lot
of code.
Conflicts:
project.gyp
spec/compiler/build_tables/action_takes_precedence_spec.cc
spec/compiler/build_tables/build_conflict_spec.cc
spec/compiler/build_tables/build_parse_table_spec.cc
spec/compiler/build_tables/first_symbols_spec.cc
spec/compiler/build_tables/item_set_closure_spec.cc
spec/compiler/build_tables/item_set_transitions_spec.cc
spec/compiler/build_tables/rule_can_be_blank_spec.cc
spec/compiler/helpers/containers.h
spec/compiler/prepare_grammar/expand_repeats_spec.cc
spec/compiler/prepare_grammar/extract_tokens_spec.cc
src/compiler/build_tables/action_takes_precedence.h
src/compiler/build_tables/build_parse_table.cc
src/compiler/build_tables/first_symbols.cc
src/compiler/build_tables/first_symbols.h
src/compiler/build_tables/item_set_closure.cc
src/compiler/build_tables/item_set_transitions.cc
src/compiler/build_tables/parse_item.cc
src/compiler/build_tables/parse_item.h
src/compiler/build_tables/rule_can_be_blank.cc
src/compiler/build_tables/rule_can_be_blank.h
src/compiler/prepare_grammar/expand_repeats.cc
src/compiler/prepare_grammar/extract_tokens.cc
src/compiler/prepare_grammar/extract_tokens.h
src/compiler/prepare_grammar/prepare_grammar.cc
src/compiler/rules/built_in_symbols.cc
src/compiler/rules/built_in_symbols.h
src/compiler/syntax_grammar.cc
src/compiler/syntax_grammar.h
2015-10-02 23:46:39 -07:00
Max Brunsfeld
e6f3239bef
Move stream operator definitions to spec helpers
...
This is one less thing for users to worry about when compiling and linking
the library itself
2015-09-10 10:12:11 -07:00
Max Brunsfeld
bd77ab1ac9
Move public rule functions out of rule namespace
...
This way, there's only one public namespace: tree_sitter
2015-09-03 17:49:20 -07:00
Max Brunsfeld
c18351772a
Auto-format: no single-line functions
2015-07-31 16:32:24 -07:00
Max Brunsfeld
f9b057f3a9
clang-format everything
2015-07-27 18:29:48 -07:00
Max Brunsfeld
aabcb10cfb
Respect expected_conflicts field when building parse table
2015-06-28 16:22:31 -05:00
Max Brunsfeld
80ec303b10
Replace prec rule w/ left_assoc and right_assoc
...
Consider shift/reduce conflicts to be compilation errors unless
they are resolved by a specified associativity.
2015-03-16 23:12:34 -07:00
Max Brunsfeld
3458fa6e50
Fix non-deterministic order in conflict description
2015-03-07 11:02:21 -08:00
Max Brunsfeld
2d436cf141
Identify fragile reductions at compile time
2015-02-21 15:11:03 -08:00
Max Brunsfeld
2e7ffb4d14
Tweak auto-format settings
...
Prefer lines that exceed 80 characters by a small margin to
line breaks in argument lists
2014-09-09 13:15:40 -07:00
Max Brunsfeld
98cc2f2264
Auto-format all source code with clang-format
2014-07-21 13:20:00 -07:00
Max Brunsfeld
f4287c07d0
Fix ParseStateId / size_t confusion in parse table
2014-07-07 13:21:30 -07:00
Max Brunsfeld
67590eddc7
Fix stream operator for new parse action types
2014-07-03 08:15:36 -07:00
Max Brunsfeld
83a1b9439e
Fix handling of ubiquitous tokens used in grammar rules
2014-07-01 20:47:35 -07:00
Max Brunsfeld
9686c57e90
Allow ubiquitous tokens to also be used in grammar rules
2014-06-26 08:52:42 -07:00
Max Brunsfeld
b010e1667e
Fix parse action equality method
2014-05-06 12:51:38 -07:00
Max Brunsfeld
25eda9d889
ISymbol -> Symbol
...
Interned symbols are now the main type of symbol in use
2014-04-28 20:43:27 -07:00
Max Brunsfeld
68d44fd565
Intern symbols during grammar preparation
2014-04-22 23:38:26 -07:00
Max Brunsfeld
a437d39773
Add rule precedence construct
...
Still need to add some way of expressing left and right
associativity
2014-04-15 08:40:46 -07:00
Max Brunsfeld
6a0a28f4b3
WIP - try to fix travis build
2014-04-08 21:41:38 -07:00
Max Brunsfeld
3982b73ed6
Start work on recording parse action conflicts
2014-04-08 08:19:55 -07:00
Max Brunsfeld
129d2b9314
Remove extra EOF actions in lexer
2014-04-04 08:44:35 -07:00
Max Brunsfeld
3f770ff3c3
Remove unused consumed_symbols vector from parse items
2014-03-26 21:04:11 -07:00
Max Brunsfeld
aac0786449
Resolve token conflicts by tokens' order in grammar
2014-03-24 19:18:06 -07:00
Max Brunsfeld
3aaa08b948
Fix some egregiously long lines
2014-03-09 23:58:25 -07:00
Max Brunsfeld
a4120f36d4
Remove all uses of 'using namespace'
2014-03-09 22:45:33 -07:00
Max Brunsfeld
de5deeeb69
Remove spurious semicolons
2014-03-09 22:24:52 -07:00
