24 commits

Author	SHA1	Message	Date
Max Brunsfeld	e9dad529f5	Make descriptions more consistent in compiler specs	2014-09-09 13:01:18 -07:00
Max Brunsfeld	1ff7cedf40	Unify ubiquitous tokens and lexical separators in API	2014-09-07 22:16:45 -07:00
Max Brunsfeld	545e575508	Revert "Remove the separator characters construct" ... This reverts commit 5cd07648fd. The separators construct is useful as an optimization. It turns out that constructing a node for every chunk of whitespace in a document causes a significant performance regression. Conflicts: src/compiler/build_tables/build_lex_table.cc src/compiler/grammar.cc src/runtime/parser.c	2014-09-02 08:03:51 -07:00
Max Brunsfeld	5cd07648fd	Remove the separator characters construct ... Now, grammars can handle whitespace by making it another ubiquitous token, like comments. For now, this has the side effect of whitespace being included in the tree that precedes it. This was already an issue for other ubiquitous tokens though, so it needs to be fixed anyway.	2014-09-01 20:19:43 -07:00
Max Brunsfeld	b155994491	Fix indentation in specs	2014-08-07 08:11:21 -07:00
Max Brunsfeld	7dfaba2954	Fix conflict manager spec	2014-07-13 18:10:32 -07:00
Max Brunsfeld	b217cd38fb	Handle built-in symbols correctly in conflict manager	2014-07-13 17:59:57 -07:00
Max Brunsfeld	77df7fe511	In lexer, always prefer the longest match ... Only use rules' precedence to decide between two tokens that match the same string	2014-07-03 08:57:35 -07:00
Max Brunsfeld	83a1b9439e	Fix handling of ubiquitous tokens used in grammar rules	2014-07-01 20:47:35 -07:00
Max Brunsfeld	7df35f9b8d	Make separate types for syntax and lexical grammars ... This way, the separator characters can be added as a field to lexical grammars only	2014-06-25 13:27:16 -07:00
Max Brunsfeld	81880e000e	Tweak header include paths in tests	2014-06-23 18:50:03 -07:00
Max Brunsfeld	e93e254518	In lexer, prefer tokens to skipped separator characters ... This was causing newlines in go and javascript to be parsed as meaningless separator characters instead of statement terminators	2014-05-30 13:29:54 -07:00
Max Brunsfeld	d91bc718a0	Add basic test for parse table builder	2014-05-04 23:28:53 -07:00
Max Brunsfeld	1d314d71c2	Separate functions for building parse and lex tables ... Now, instead of adding states to the lex table as they are needed by the parse states, we iterate over the parse states after the fact and set up their corresponding lex states. This has the nice side effect that the lex states are in a more readable order.	2014-05-04 22:07:52 -07:00
Max Brunsfeld	3a50171249	Expose all grammar compilation errors	2014-05-01 23:28:40 -07:00
Max Brunsfeld	93620b3ed1	Add keyword helper for making higher-priority string tokens	2014-05-01 13:25:20 -07:00
Max Brunsfeld	25eda9d889	ISymbol -> Symbol ... Interned symbols are now the main type of symbol in use	2014-04-28 20:43:27 -07:00
Max Brunsfeld	68d44fd565	Intern symbols during grammar preparation	2014-04-22 23:38:26 -07:00
Max Brunsfeld	33d781f492	Refactor bookkeeping of token starts in lexical rules ... - Move lex items and parse items into their own files	2014-04-17 13:33:34 -07:00
Max Brunsfeld	a437d39773	Add rule precedence construct ... Still need to add some way of expressing left and right associativity	2014-04-15 08:40:46 -07:00
Max Brunsfeld	67243c7e2f	cpplint	2014-04-14 08:38:44 -07:00
Max Brunsfeld	e1e0cc6278	Make sure conflicts returned by compile are unique	2014-04-10 08:38:14 -07:00
Max Brunsfeld	f71d7bae16	Record parse conflicts when compiling grammars ... Need to remove duplicate conflicts	2014-04-08 18:47:42 -07:00
Max Brunsfeld	3982b73ed6	Start work on recording parse action conflicts	2014-04-08 08:19:55 -07:00