Max Brunsfeld
346cf4fe5d
Remove LEX_PANIC macro
2014-08-26 13:12:12 -07:00
Max Brunsfeld
0bb5663f0f
Refactor - represent char sets in terms of inclusions and exclusions
2014-08-23 14:25:45 -07:00
Max Brunsfeld
1e79ed794b
Allow multiple top-level nodes
...
Now, the root node of a document is always a document node.
It will often have only one child node which corresponds to the grammar's
start symbol, but not always. Currently, it may have more than one child
if there are ubiquitous tokens such as comments at the beginning of the
document. In the future, it will also be possible be possible to have multiple
for the document to have multiple children if the document is partially parsed.
2014-08-09 00:00:20 -07:00
Max Brunsfeld
9366f11dcb
In generated C, only format printable characters as char literals
2014-08-07 08:12:15 -07:00
Max Brunsfeld
eecbcccee0
Remove generated parsers' dependency on the runtime library
...
Generated parsers no longer export a parser constructor function.
They now export an opaque Language object which can be set on
Documents directly. This way, the logic for constructing parsers
lives entirely in the runtime. The Languages are just structs which
have no load-time dependency on the runtime
2014-07-30 23:40:02 -07:00
Max Brunsfeld
4d14a65e22
In build_parse_table, switch recursion to explicit iteration
2014-07-13 18:06:37 -07:00
Max Brunsfeld
1c7d2d2d03
Add for-in loops and math assignment operators to js grammar
2014-07-07 13:35:55 -07:00
Max Brunsfeld
77df7fe511
In lexer, always prefer the longest match
...
Only use rules' precedence to decide between two tokens
that match the same string
2014-07-03 08:57:35 -07:00
Max Brunsfeld
c85841364e
Add throw statements to js grammar
2014-07-03 08:20:43 -07:00
Max Brunsfeld
83a1b9439e
Fix handling of ubiquitous tokens used in grammar rules
2014-07-01 20:47:35 -07:00
Max Brunsfeld
9686c57e90
Allow ubiquitous tokens to also be used in grammar rules
2014-06-26 08:52:42 -07:00
Max Brunsfeld
bb4d83ce47
Add regex postfix flags to javascript grammar
...
- Refactor statement terminators in javascript grammar
- Reorganize javascript language tests
2014-06-11 16:43:27 -07:00
Max Brunsfeld
082560dd6e
Fix operator precedence of '.' operator in js grammar
2014-06-11 14:01:38 -07:00
Max Brunsfeld
4ad6278334
Add finally, instance of, typeof, in to js grammar
2014-06-11 11:49:06 -07:00
Max Brunsfeld
e93e254518
In lexer, prefer tokens to skipped separator characters
...
This was causing newlines in go and javascript to be parsed as
meaningless separator characters instead of statement terminators
2014-05-30 13:29:54 -07:00
Max Brunsfeld
4c9ac3dada
Fix parsing of empty strings in javascript and golang
2014-05-20 09:47:26 -07:00
Max Brunsfeld
c30055ba18
Fix symbol names for extracted tokens
2014-05-20 08:30:58 -07:00
Max Brunsfeld
649f200831
Expand regex/string rules as part of grammar preparation
...
This makes it possible to report errors in regex parsing
2014-05-19 20:54:59 -07:00
Max Brunsfeld
5245bc01fe
Backfill tests for token extraction in auxiliary rules
2014-05-19 19:05:54 -07:00
Max Brunsfeld
10d3801d7e
Fix missing symbol names for keywords
2014-05-09 16:14:48 -07:00
Max Brunsfeld
34137be12d
Represent state ids as unsigned shorts
...
This fixes some signedness conversion warnings
2014-05-08 13:23:46 -07:00
Max Brunsfeld
0a21eee3f0
Remove magic number from generated symbols enums
...
The symbol numbers 0 and 1 are reserved for 'error' and 'eof',
so the grammar's start symbol is always 2.
2014-05-08 13:14:45 -07:00
Max Brunsfeld
4700e33746
Introduce 'ubiquitous_tokens' concept, for parsing comments and such
2014-05-06 12:54:04 -07:00
Max Brunsfeld
1d314d71c2
Separate functions for building parse and lex tables
...
Now, instead of adding states to the lex table as they are needed
by the parse states, we iterate over the parse states after the fact
and set up their corresponding lex states. This has the nice side
effect that the lex states are in a more readable order.
2014-05-04 22:07:52 -07:00
Max Brunsfeld
bae32adc7b
Add constructor calls, pre/postfix operators to js grammar
2014-05-04 13:36:19 -07:00
Max Brunsfeld
1bdd87535a
Add prefix math operators +, - to javascript grammar
2014-05-02 07:42:13 -07:00
Max Brunsfeld
b9393b5c1a
Refactor grammar helper functions
2014-05-01 20:02:06 -07:00
Max Brunsfeld
2906125824
Refactor javascript grammar to use new keyword helper
2014-05-01 13:25:50 -07:00
Max Brunsfeld
a2c125998e
Add single quoted strings and regexes to javascript grammar
2014-05-01 12:43:53 -07:00
Max Brunsfeld
4dd1f9fbce
Cache state transition computations by grammar rule
...
This results in a huge speed increase
2014-04-27 23:03:37 -07:00
Max Brunsfeld
29bbff655c
Store choice rules using vectors, not pairs
2014-04-26 23:21:09 -07:00
Max Brunsfeld
801f4bd0a8
Add returns, deletes and bool operators to js grammar
2014-04-25 22:08:11 -07:00
Max Brunsfeld
61692c8bb1
Add error recovery in function calls to javascript gramamr
2014-04-24 13:22:54 -07:00
Max Brunsfeld
52c338ed60
Add some infix math operators to javascript grammar
2014-04-23 22:25:48 -07:00
Max Brunsfeld
7be8d469b8
Add ternary expressions to javascript grammar
2014-04-23 22:15:07 -07:00
Max Brunsfeld
68d44fd565
Intern symbols during grammar preparation
2014-04-22 23:38:26 -07:00
Max Brunsfeld
33d781f492
Refactor bookkeeping of token starts in lexical rules
...
- Move lex items and parse items into their own files
2014-04-17 13:33:34 -07:00
Max Brunsfeld
5145bba53d
Silence missing-initializer warnings for gcc
2014-04-12 20:16:16 -07:00
Max Brunsfeld
53285f7d0c
Give rule-specific names to repeat helper rules
2014-04-09 18:41:43 -07:00
Max Brunsfeld
bd5ec68c96
Get generated parsers building under gcc
2014-04-08 22:11:20 -07:00
Max Brunsfeld
f71d7bae16
Record parse conflicts when compiling grammars
...
Need to remove duplicate conflicts
2014-04-08 18:47:42 -07:00
Max Brunsfeld
be1c8e0f17
Add dynamic property access to javascript grammar
2014-04-05 15:55:20 -07:00
Max Brunsfeld
2191a7d988
Add switch statements to javascript grammar
2014-04-04 13:10:33 -07:00
Max Brunsfeld
129d2b9314
Remove extra EOF actions in lexer
2014-04-04 08:44:35 -07:00
Max Brunsfeld
1cc7e32e2d
Fix handling of tokens consisting of separator characters
...
The parser is no longer hard-coded to skip whitespace. Tokens
such as newlines, whose characters overlap with the separator
characters, can now be correctly recognized.
2014-04-03 19:10:09 -07:00
Max Brunsfeld
a79a7435de
Remove remaining trailing whitespace from generated c code
2014-03-29 19:21:42 -07:00
Max Brunsfeld
8e1b78ca8e
Remove trailing whitespace from generated c code
2014-03-29 19:00:31 -07:00
Max Brunsfeld
99817a38c1
Improve trees generated by arithmetic grammar
...
This work should inform how to implement operator
precedence helper functions
2014-03-29 18:37:51 -07:00
Max Brunsfeld
7adb0bf34f
Add golang example grammar
...
Also, support '\a' character class shorthand in regexes,
for alphabetical characters
2014-03-29 16:29:34 -07:00
Max Brunsfeld
13c4e6e648
Tweak format for example grammars
2014-03-28 13:51:32 -07:00
