Max Brunsfeld
e105f5cebc
Remove inheritance link btwn PreparedGrammar and Grammar
2014-06-10 10:34:37 -07:00
Max Brunsfeld
e93e254518
In lexer, prefer tokens to skipped separator characters
...
This was causing newlines in go and javascript to be parsed as
meaningless separator characters instead of statement terminators
2014-05-30 13:29:54 -07:00
Max Brunsfeld
220e081c49
Remove unnecessary methods on LexTable
2014-05-29 18:28:23 -07:00
Max Brunsfeld
649f200831
Expand regex/string rules as part of grammar preparation
...
This makes it possible to report errors in regex parsing
2014-05-19 20:54:59 -07:00
Max Brunsfeld
608b5ce02b
Cleanup - extract method in parse table builder
2014-05-12 08:53:28 -07:00
Max Brunsfeld
4700e33746
Introduce 'ubiquitous_tokens' concept, for parsing comments and such
2014-05-06 12:54:04 -07:00
Max Brunsfeld
63c0e27501
Clean up table builder functions
2014-05-04 23:04:34 -07:00
Max Brunsfeld
1d314d71c2
Separate functions for building parse and lex tables
...
Now, instead of adding states to the lex table as they are needed
by the parse states, we iterate over the parse states after the fact
and set up their corresponding lex states. This has the nice side
effect that the lex states are in a more readable order.
2014-05-04 22:07:52 -07:00
Max Brunsfeld
b998bb35af
Don't use initializer lists for tuples
...
gcc doesn't allow it.
2014-05-04 19:49:09 -07:00
Max Brunsfeld
3a50171249
Expose all grammar compilation errors
2014-05-01 23:28:40 -07:00
Max Brunsfeld
93620b3ed1
Add keyword helper for making higher-priority string tokens
2014-05-01 13:25:20 -07:00
Max Brunsfeld
6d40dcf881
Add token helper for building token rules
...
Now you can specify the structure of tokens using
all of the rule functions, not just `str` and `pattern`
2014-05-01 12:43:29 -07:00
Max Brunsfeld
0d763d229d
cpplint
2014-04-28 21:46:43 -07:00
Max Brunsfeld
25eda9d889
ISymbol -> Symbol
...
Interned symbols are now the main type of symbol in use
2014-04-28 20:43:27 -07:00
Max Brunsfeld
faf80aadac
Symbol -> NamedSymbol
2014-04-28 20:15:49 -07:00
Max Brunsfeld
66c033e411
Optimize parse item equality
2014-04-28 18:31:03 -07:00
Max Brunsfeld
70ba76762c
Optimize item set transitions function further
2014-04-28 18:30:50 -07:00
Max Brunsfeld
ae2450d282
Fix dangling reference error on gcc
2014-04-28 13:06:16 -07:00
Max Brunsfeld
4dd1f9fbce
Cache state transition computations by grammar rule
...
This results in a huge speed increase
2014-04-27 23:03:37 -07:00
Max Brunsfeld
4eece88e18
Remove duplicate set insert
2014-04-27 22:08:40 -07:00
Max Brunsfeld
076a27be4a
Fix compile error on gcc
2014-04-27 22:08:01 -07:00
Max Brunsfeld
b86203d205
Extract shared helper functions from example grammars
2014-04-27 21:45:05 -07:00
Max Brunsfeld
29bbff655c
Store choice rules using vectors, not pairs
2014-04-26 23:21:09 -07:00
Max Brunsfeld
c9e0d4bdf0
Optimize item set closure function
2014-04-26 19:06:34 -07:00
Max Brunsfeld
7be8d469b8
Add ternary expressions to javascript grammar
2014-04-23 22:15:07 -07:00
Max Brunsfeld
5313bb5257
Simplify item set closure function
2014-04-23 13:35:21 -07:00
Max Brunsfeld
2b883a72a2
Further reduce allocations in rule transitions functions
2014-04-23 13:12:00 -07:00
Max Brunsfeld
e8f2b788d4
Reduce allocations when computing rule transitions
2014-04-23 09:01:57 -07:00
Max Brunsfeld
3b388d66cd
Profile and optimize
...
- Eliminate unnecessary copies of grammar objects
- Do cheaper comparisons first in equality methods
2014-04-23 08:32:11 -07:00
Max Brunsfeld
68d44fd565
Intern symbols during grammar preparation
2014-04-22 23:38:26 -07:00
Max Brunsfeld
33d781f492
Refactor bookkeeping of token starts in lexical rules
...
- Move lex items and parse items into their own files
2014-04-17 13:33:34 -07:00
Max Brunsfeld
d32c8b7dd7
Use const references in a few more places
2014-04-16 18:30:21 -07:00
Max Brunsfeld
a437d39773
Add rule precedence construct
...
Still need to add some way of expressing left and right
associativity
2014-04-15 08:40:46 -07:00
Max Brunsfeld
67243c7e2f
cpplint
2014-04-14 08:38:44 -07:00
Max Brunsfeld
e1e0cc6278
Make sure conflicts returned by compile are unique
2014-04-10 08:38:14 -07:00
Max Brunsfeld
a5816a9624
Refactor rule visitors
2014-04-09 13:28:02 -07:00
Max Brunsfeld
6a0a28f4b3
WIP - try to fix travis build
2014-04-08 21:41:38 -07:00
Max Brunsfeld
f71d7bae16
Record parse conflicts when compiling grammars
...
Need to remove duplicate conflicts
2014-04-08 18:47:42 -07:00
Max Brunsfeld
3982b73ed6
Start work on recording parse action conflicts
2014-04-08 08:19:55 -07:00
Max Brunsfeld
1da9f1fdfd
Store rule metadata as a map, not a single number
...
Need to store more than just boolean values
2014-04-07 08:50:00 -07:00
Max Brunsfeld
5320cad065
Trim trailing whitespace
2014-04-04 13:10:55 -07:00
Max Brunsfeld
129d2b9314
Remove extra EOF actions in lexer
2014-04-04 08:44:35 -07:00
Max Brunsfeld
5a00ebbc99
Clean up parse table builder
2014-04-03 20:36:06 -07:00
Max Brunsfeld
1cc7e32e2d
Fix handling of tokens consisting of separator characters
...
The parser is no longer hard-coded to skip whitespace. Tokens
such as newlines, whose characters overlap with the separator
characters, can now be correctly recognized.
2014-04-03 19:10:09 -07:00
Max Brunsfeld
f39cb1890d
Refactor rule visitor objects
2014-04-01 13:38:02 -07:00
Max Brunsfeld
85b97c4f87
Fix error in merging character set transitions
2014-04-01 13:33:32 -07:00
Max Brunsfeld
13c4e6e648
Tweak format for example grammars
2014-03-28 13:51:32 -07:00
Max Brunsfeld
820b6f4020
Fix infinite loops during table generation for left-recursive rules
...
Add function calls for arbitrary expressions and dot property access to javascript grammar
2014-03-26 22:43:08 -07:00
Max Brunsfeld
3f770ff3c3
Remove unused consumed_symbols vector from parse items
2014-03-26 21:04:11 -07:00
Max Brunsfeld
05e16a8c46
Remove ordering of parse and lex items
...
Using an unordered_map and unordered_set to store these
speeds up grammar compilation significantly. The ordering
method on items was using the rules' .to_string method to
compute the order, which was not a meaningful comparison,
and was probably slow.
2014-03-26 13:13:22 -07:00
