Max Brunsfeld
11a1ea8dfe
Reuse separator-rule in LexTableBuilder
2015-10-05 17:02:59 -07:00
Max Brunsfeld
5e4bdcbaf8
Simplify handling of precedence & associativity in productions
2015-10-05 16:56:11 -07:00
Max Brunsfeld
ef2acf9496
Make ParseItemSet & LexItemSet classes
2015-10-05 15:13:43 -07:00
Max Brunsfeld
f01972c64e
Reorganize ParseItemSet and LexItemSet
2015-10-05 15:09:06 -07:00
Max Brunsfeld
ebc52f109d
Merge branch 'flatten-rules-into-productions'
...
This branch had diverged considerably, so merging it required changing a lot
of code.
Conflicts:
project.gyp
spec/compiler/build_tables/action_takes_precedence_spec.cc
spec/compiler/build_tables/build_conflict_spec.cc
spec/compiler/build_tables/build_parse_table_spec.cc
spec/compiler/build_tables/first_symbols_spec.cc
spec/compiler/build_tables/item_set_closure_spec.cc
spec/compiler/build_tables/item_set_transitions_spec.cc
spec/compiler/build_tables/rule_can_be_blank_spec.cc
spec/compiler/helpers/containers.h
spec/compiler/prepare_grammar/expand_repeats_spec.cc
spec/compiler/prepare_grammar/extract_tokens_spec.cc
src/compiler/build_tables/action_takes_precedence.h
src/compiler/build_tables/build_parse_table.cc
src/compiler/build_tables/first_symbols.cc
src/compiler/build_tables/first_symbols.h
src/compiler/build_tables/item_set_closure.cc
src/compiler/build_tables/item_set_transitions.cc
src/compiler/build_tables/parse_item.cc
src/compiler/build_tables/parse_item.h
src/compiler/build_tables/rule_can_be_blank.cc
src/compiler/build_tables/rule_can_be_blank.h
src/compiler/prepare_grammar/expand_repeats.cc
src/compiler/prepare_grammar/extract_tokens.cc
src/compiler/prepare_grammar/extract_tokens.h
src/compiler/prepare_grammar/prepare_grammar.cc
src/compiler/rules/built_in_symbols.cc
src/compiler/rules/built_in_symbols.h
src/compiler/syntax_grammar.cc
src/compiler/syntax_grammar.h
2015-10-02 23:46:39 -07:00
Max Brunsfeld
673ca411b1
Fix lint errors
2015-09-19 13:19:49 -07:00
Max Brunsfeld
5982b77c97
In compiler, distinguish between anonymous tokens and hidden rules
2015-09-05 22:28:55 -07:00
Max Brunsfeld
bd77ab1ac9
Move public rule functions out of rule namespace
...
This way, there's only one public namespace: tree_sitter
2015-09-03 17:49:20 -07:00
Max Brunsfeld
97bb7a26cf
Fix precedence calculations when building parse table
...
* Recurse into choice rules
* Compute reduction precedence differently than shift precedence
2015-09-02 13:05:54 -07:00
Max Brunsfeld
f9b057f3a9
clang-format everything
2015-07-27 18:29:48 -07:00
Max Brunsfeld
7872ddd21b
Improve reduce/reduce conflict error message
2015-03-16 11:59:45 -07:00
Max Brunsfeld
074a7884aa
Fix uninitialized variable error
2015-02-16 22:09:30 -08:00
Max Brunsfeld
160fca6579
Refactor avoidance of redundant repeat rules
2015-01-14 21:11:19 -08:00
Max Brunsfeld
a0d9da9d5c
Rename static 'Build' methods to 'build'
2015-01-14 21:11:05 -08:00
Max Brunsfeld
34d96909d1
Move {Syntax,Lexical}Grammar into separate files
2015-01-14 21:10:41 -08:00
Max Brunsfeld
0d267e41aa
Replace LexConflictManager class with action_takes_precedence function
2015-01-11 19:50:58 -08:00
Max Brunsfeld
ef2084d3c8
Tweak parse debugging
2014-10-13 21:20:08 -07:00
Max Brunsfeld
9fd2821389
Fix TODO comment in build_lex_table
2014-10-12 13:53:31 -07:00
Max Brunsfeld
1fb52eacab
Tidy up build_lex_table function
2014-10-12 12:44:16 -07:00
Max Brunsfeld
2e7ffb4d14
Tweak auto-format settings
...
Prefer lines that exceed 80 characters by a small margin to
line breaks in argument lists
2014-09-09 13:15:40 -07:00
Max Brunsfeld
1ff7cedf40
Unify ubiquitous tokens and lexical separators in API
2014-09-07 22:16:45 -07:00
Max Brunsfeld
545e575508
Revert "Remove the separator characters construct"
...
This reverts commit 5cd07648fd
2014-09-02 08:03:51 -07:00
Max Brunsfeld
e941f8c175
Fix error in document editing
...
When breaking down the stack in parser.c, the previous code
would not account for ubiquitous tokens. This was a problem
for a long time, but wasn't noticed until ubiquitous tokens
started being used to represent separator characters
2014-09-01 21:32:29 -07:00
Max Brunsfeld
5cd07648fd
Remove the separator characters construct
...
Now, grammars can handle whitespace by making it another ubiquitous
token, like comments.
For now, this has the side effect of whitespace being included in the
tree that precedes it. This was already an issue for other ubiquitous
tokens though, so it needs to be fixed anyway.
2014-09-01 20:19:43 -07:00
Max Brunsfeld
0bb5663f0f
Refactor - represent char sets in terms of inclusions and exclusions
2014-08-23 14:25:45 -07:00
Max Brunsfeld
98cc2f2264
Auto-format all source code with clang-format
2014-07-21 13:20:00 -07:00
Max Brunsfeld
a9dff20658
Make grammars' separator characters configurable
2014-06-26 07:31:08 -07:00
Max Brunsfeld
7df35f9b8d
Make separate types for syntax and lexical grammars
...
This way, the separator characters can be added as a field to
lexical grammars only
2014-06-25 13:27:16 -07:00
Max Brunsfeld
d203c15911
Remove unneeded blank rule in lex table builder
2014-06-11 17:43:07 -07:00
Max Brunsfeld
e93e254518
In lexer, prefer tokens to skipped separator characters
...
This was causing newlines in go and javascript to be parsed as
meaningless separator characters instead of statement terminators
2014-05-30 13:29:54 -07:00
Max Brunsfeld
220e081c49
Remove unnecessary methods on LexTable
2014-05-29 18:28:23 -07:00
Max Brunsfeld
4700e33746
Introduce 'ubiquitous_tokens' concept, for parsing comments and such
2014-05-06 12:54:04 -07:00
Max Brunsfeld
63c0e27501
Clean up table builder functions
2014-05-04 23:04:34 -07:00
Max Brunsfeld
1d314d71c2
Separate functions for building parse and lex tables
...
Now, instead of adding states to the lex table as they are needed
by the parse states, we iterate over the parse states after the fact
and set up their corresponding lex states. This has the nice side
effect that the lex states are in a more readable order.
2014-05-04 22:07:52 -07:00
