Max Brunsfeld
d5ce268074
Fix handling of changing precedence within lexical rules.
...
A precedence annotation wrapping a sequence of characters now only affects how
tightly those characters bind to *each other*, not how tightly they bind to the
preceding character.
This bug surfaced because a generated lexer was failing to recognize a '\n' character
as a token, instead treating it as ubiquitous whitespace. It made this error
because, even though anonymous ubiquitous tokens have the lowest precedence, the
character immediately *after* the '\n' was part of a normal token, which had
*normal* precedence (0). Advancing into that following token was incorrectly
prioritized above accepting the line-break token.
2015-11-08 13:36:15 -08:00
Max Brunsfeld
ebc52f109d
Merge branch 'flatten-rules-into-productions'
...
This branch had diverged considerably, so merging it required changing a lot
of code.
Conflicts:
project.gyp
spec/compiler/build_tables/action_takes_precedence_spec.cc
spec/compiler/build_tables/build_conflict_spec.cc
spec/compiler/build_tables/build_parse_table_spec.cc
spec/compiler/build_tables/first_symbols_spec.cc
spec/compiler/build_tables/item_set_closure_spec.cc
spec/compiler/build_tables/item_set_transitions_spec.cc
spec/compiler/build_tables/rule_can_be_blank_spec.cc
spec/compiler/helpers/containers.h
spec/compiler/prepare_grammar/expand_repeats_spec.cc
spec/compiler/prepare_grammar/extract_tokens_spec.cc
src/compiler/build_tables/action_takes_precedence.h
src/compiler/build_tables/build_parse_table.cc
src/compiler/build_tables/first_symbols.cc
src/compiler/build_tables/first_symbols.h
src/compiler/build_tables/item_set_closure.cc
src/compiler/build_tables/item_set_transitions.cc
src/compiler/build_tables/parse_item.cc
src/compiler/build_tables/parse_item.h
src/compiler/build_tables/rule_can_be_blank.cc
src/compiler/build_tables/rule_can_be_blank.h
src/compiler/prepare_grammar/expand_repeats.cc
src/compiler/prepare_grammar/extract_tokens.cc
src/compiler/prepare_grammar/extract_tokens.h
src/compiler/prepare_grammar/prepare_grammar.cc
src/compiler/rules/built_in_symbols.cc
src/compiler/rules/built_in_symbols.h
src/compiler/syntax_grammar.cc
src/compiler/syntax_grammar.h
2015-10-02 23:46:39 -07:00
Max Brunsfeld
5982b77c97
In compiler, distinguish between anonymous tokens and hidden rules
2015-09-05 22:28:55 -07:00
Max Brunsfeld
9338249075
Remove implicit CharacterRange constructors
...
Also fix misc smaller lint errors
2014-08-23 14:52:44 -07:00
Max Brunsfeld
0bb5663f0f
Refactor - represent char sets in terms of inclusions and exclusions
2014-08-23 14:25:45 -07:00
Max Brunsfeld
b155994491
Fix indentation in specs
2014-08-07 08:11:21 -07:00
Max Brunsfeld
e93e254518
In lexer, prefer tokens to skipped separator characters
...
This was causing newlines in go and javascript to be parsed as
meaningless separator characters instead of statement terminators
2014-05-30 13:29:54 -07:00
Max Brunsfeld
649f200831
Expand regex/string rules as part of grammar preparation
...
This makes it possible to report errors in regex parsing
2014-05-19 20:54:59 -07:00
Max Brunsfeld
25eda9d889
ISymbol -> Symbol
...
Interned symbols are now the main type of symbol in use
2014-04-28 20:43:27 -07:00
Max Brunsfeld
93df5579b4
Trim whitespace
2014-04-25 22:17:23 -07:00
Max Brunsfeld
68d44fd565
Intern symbols during grammar preparation
2014-04-22 23:38:26 -07:00
