Max Brunsfeld
1daaf4485f
Refactor item set transition functions
2014-06-16 13:37:34 -07:00
Max Brunsfeld
17040e32ec
Remove unused version of first_set function
2014-06-16 13:25:30 -07:00
Max Brunsfeld
39c1ab2d50
Refactor item_set_closure
...
Inline unnecessary function
2014-06-16 13:20:39 -07:00
Max Brunsfeld
7a2c2c1c90
Store ParseItemSets as maps, w/ core items as keys
...
ParseItem no longer has a lookahead_sym field; it now represents
the 'core' of a parse item. The lookahead context is stored separately,
as a set per core item. This makes iterating, copying and merging item
sets more efficient, because before, the core items were repeated for each
different lookahead symbol.
Also, the memoization in sym_transitions(ParseItemSet) has been removed.
Maybe I'll add it back later.
2014-06-16 08:35:20 -07:00
Max Brunsfeld
d203c15911
Remove unneeded blank rule in lex table builder
2014-06-11 17:43:07 -07:00
Max Brunsfeld
cad6e0ed5f
Fix expected symbol count in lr parser
2014-06-11 16:44:38 -07:00
Max Brunsfeld
bb4d83ce47
Add regex postfix flags to javascript grammar
...
- Refactor statement terminators in javascript grammar
- Reorganize javascript language tests
2014-06-11 16:43:27 -07:00
Max Brunsfeld
3cd031af38
Add keypattern rule helper
...
This way, pattern rules (e.g. golang's comment) can be easily given the
same precedence as keyword rules.
2014-06-11 12:40:49 -07:00
Max Brunsfeld
174f306e2a
Fix precedence of comments vs '/' operator
2014-06-11 12:27:58 -07:00
Max Brunsfeld
a42f498c59
Optimize merge_sym_transitions and merge_char_transitions
2014-06-10 14:01:32 -07:00
Max Brunsfeld
123d3b26d8
Add more expressions, statements to golang grammar
2014-06-10 11:33:05 -07:00
Max Brunsfeld
e105f5cebc
Remove inheritance link btwn PreparedGrammar and Grammar
2014-06-10 10:34:37 -07:00
Max Brunsfeld
11acc7d087
Fix missing initializer warnings
2014-06-09 21:47:57 -07:00
Max Brunsfeld
21c259df9c
Clean up lint errors
2014-06-09 21:14:38 -07:00
Max Brunsfeld
54a555168d
Add accessor methods on Grammar
2014-06-09 21:05:25 -07:00
Max Brunsfeld
12331d66f5
Fix memory leaks
2014-06-09 13:12:44 -07:00
Max Brunsfeld
9bc7d51074
Always initialize tree node size and offset
2014-06-09 13:12:44 -07:00
Max Brunsfeld
deaa7e2755
Fix missing return warning on gcc
2014-06-04 13:39:12 -07:00
Max Brunsfeld
9a4889176e
Move lr_parser implementation into a separate .c file
2014-06-04 13:34:37 -07:00
Max Brunsfeld
63cde3967c
Add unit test for stack
...
- Also, fix bug where trees pushed onto the stack were not retained
2014-06-03 13:19:49 -07:00
Max Brunsfeld
baec9f2c9a
Move computation of tree size/offset into tree constructor
2014-06-02 13:32:36 -07:00
Max Brunsfeld
e93e254518
In lexer, prefer tokens to skipped separator characters
...
This was causing newlines in go and javascript to be parsed as
meaningless separator characters instead of statement terminators
2014-05-30 13:29:54 -07:00
Max Brunsfeld
220e081c49
Remove unnecessary methods on LexTable
2014-05-29 18:28:23 -07:00
Max Brunsfeld
2988cc5aa2
Show offending lookahead chars when pretty-printing trees w/ errors
2014-05-26 21:50:01 -07:00
Max Brunsfeld
c30055ba18
Fix symbol names for extracted tokens
2014-05-20 08:30:58 -07:00
Max Brunsfeld
649f200831
Expand regex/string rules as part of grammar preparation
...
This makes it possible to report errors in regex parsing
2014-05-19 20:54:59 -07:00
Max Brunsfeld
5245bc01fe
Backfill tests for token extraction in auxiliary rules
2014-05-19 19:05:54 -07:00
Max Brunsfeld
608b5ce02b
Cleanup - extract method in parse table builder
2014-05-12 08:53:28 -07:00
Max Brunsfeld
10d3801d7e
Fix missing symbol names for keywords
2014-05-09 16:14:48 -07:00
Max Brunsfeld
3e0debf814
Fix compile error on gcc
2014-05-09 15:37:30 -07:00
Max Brunsfeld
963768eb8a
Remove unnecessary parser helper functions
2014-05-09 15:03:29 -07:00
Max Brunsfeld
ccc1b41f2a
Make separate header files for stack and lexer
2014-05-09 13:32:12 -07:00
Max Brunsfeld
e4be585c43
Handle ubiquitous tokens at the beginning of programs
...
As a final step before returning the finished parse tree, check if
there are still multiple nodes on the stack. If so, make the inner
nodes children of the top node.
2014-05-09 12:46:36 -07:00
Max Brunsfeld
3f374c6547
Tidy up
2014-05-08 13:27:48 -07:00
Max Brunsfeld
34137be12d
Represent state ids as unsigned shorts
...
This fixes some signedness conversion warnings
2014-05-08 13:23:46 -07:00
Max Brunsfeld
0a21eee3f0
Remove magic number from generated symbols enums
...
The symbol numbers 0 and 1 are reserved for 'error' and 'eof',
so the grammar's start symbol is always 2.
2014-05-08 13:14:45 -07:00
Max Brunsfeld
4700e33746
Introduce 'ubiquitous_tokens' concept, for parsing comments and such
2014-05-06 12:54:04 -07:00
Max Brunsfeld
b010e1667e
Fix parse action equality method
2014-05-06 12:51:38 -07:00
Max Brunsfeld
63c0e27501
Clean up table builder functions
2014-05-04 23:04:34 -07:00
Max Brunsfeld
1d314d71c2
Separate functions for building parse and lex tables
...
Now, instead of adding states to the lex table as they are needed
by the parse states, we iterate over the parse states after the fact
and set up their corresponding lex states. This has the nice side
effect that the lex states are in a more readable order.
2014-05-04 22:07:52 -07:00
Max Brunsfeld
b998bb35af
Don't use initializer lists for tuples
...
gcc doesn't allow it.
2014-05-04 19:49:09 -07:00
Max Brunsfeld
b97e4f1de8
Clean up intern symbols function
2014-05-01 23:40:04 -07:00
Max Brunsfeld
3a50171249
Expose all grammar compilation errors
2014-05-01 23:28:40 -07:00
Max Brunsfeld
93620b3ed1
Add keyword helper for making higher-priority string tokens
2014-05-01 13:25:20 -07:00
Max Brunsfeld
a2c125998e
Add single quoted strings and regexes to javascript grammar
2014-05-01 12:43:53 -07:00
Max Brunsfeld
6d40dcf881
Add token helper for building token rules
...
Now you can specify the structure of tokens using
all of the rule functions, not just `str` and `pattern`
2014-05-01 12:43:29 -07:00
Max Brunsfeld
0d763d229d
cpplint
2014-04-28 21:46:43 -07:00
Max Brunsfeld
25eda9d889
ISymbol -> Symbol
...
Interned symbols are now the main type of symbol in use
2014-04-28 20:43:27 -07:00
Max Brunsfeld
faf80aadac
Symbol -> NamedSymbol
2014-04-28 20:15:49 -07:00
Max Brunsfeld
66c033e411
Optimize parse item equality
2014-04-28 18:31:03 -07:00
