Commit graph

16 commits

Author SHA1 Message Date
Max Brunsfeld
e9dad529f5 Make descriptions more consistent in compiler specs 2014-09-09 13:01:18 -07:00
Max Brunsfeld
1ff7cedf40 Unify ubiquitous tokens and lexical separators in API 2014-09-07 22:16:45 -07:00
Max Brunsfeld
545e575508 Revert "Remove the separator characters construct"
This reverts commit 5cd07648fd.

The separators construct is useful as an optimization. It turns out that
constructing a node for every chunk of whitespace in a document causes a
significant performance regression.

Conflicts:
	src/compiler/build_tables/build_lex_table.cc
	src/compiler/grammar.cc
	src/runtime/parser.c
2014-09-02 08:03:51 -07:00
Max Brunsfeld
5cd07648fd Remove the separator characters construct
Now, grammars can handle whitespace by making it another ubiquitous
token, like comments.

For now, this has the side effect of whitespace being included in the
tree that precedes it. This was already an issue for other ubiquitous
tokens though, so it needs to be fixed anyway.
2014-09-01 20:19:43 -07:00
Max Brunsfeld
b155994491 Fix indentation in specs 2014-08-07 08:11:21 -07:00
Max Brunsfeld
83a1b9439e Fix handling of ubiquitous tokens used in grammar rules 2014-07-01 20:47:35 -07:00
Max Brunsfeld
7df35f9b8d Make separate types for syntax and lexical grammars
This way, the separator characters can be added as a field to
lexical grammars only
2014-06-25 13:27:16 -07:00
Max Brunsfeld
81880e000e Tweak header include paths in tests 2014-06-23 18:50:03 -07:00
Max Brunsfeld
e105f5cebc Remove inheritance link btwn PreparedGrammar and Grammar 2014-06-10 10:34:37 -07:00
Max Brunsfeld
54a555168d Add accessor methods on Grammar 2014-06-09 21:05:25 -07:00
Max Brunsfeld
4700e33746 Introduce 'ubiquitous_tokens' concept, for parsing comments and such 2014-05-06 12:54:04 -07:00
Max Brunsfeld
3a50171249 Expose all grammar compilation errors 2014-05-01 23:28:40 -07:00
Max Brunsfeld
25eda9d889 ISymbol -> Symbol
Interned symbols are now the main type of symbol in use
2014-04-28 20:43:27 -07:00
Max Brunsfeld
faf80aadac Symbol -> NamedSymbol 2014-04-28 20:15:49 -07:00
Max Brunsfeld
93df5579b4 Trim whitespace 2014-04-25 22:17:23 -07:00
Max Brunsfeld
68d44fd565 Intern symbols during grammar preparation 2014-04-22 23:38:26 -07:00