traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
737 commits 16 branches 98 tags 37 MiB
Commit graph

53 commits

Author SHA1 Message Date
Max Brunsfeld
acf9280eda Make expression and statement rules hidden in javascript grammar 2015-09-02 13:05:54 -07:00
Max Brunsfeld
14d7ebb7da Leave newlines in javascript test programs 2015-08-23 10:42:06 -07:00
Max Brunsfeld
21258e6a9e Remove 'document' wrapper node 2015-08-22 10:48:34 -07:00
Max Brunsfeld
3d0890eecf Preserve tokens within errors 2015-06-15 15:26:06 -07:00
Max Brunsfeld
80ec303b10 Replace prec rule w/ left_assoc and right_assoc 
Consider shift/reduce conflicts to be compilation errors unless
they are resolved by a specified associativity.
 2015-03-16 23:12:34 -07:00
Max Brunsfeld
91cf35b72c Rework javascript fixture grammar 2014-10-21 08:49:16 -07:00
Max Brunsfeld
8134b64d00 Remove repetition from language spec descriptions 2014-10-19 12:36:43 -07:00
Max Brunsfeld
22ee68e1a9 Make node for each var assignment in JS grammar 2014-10-15 15:04:57 -07:00
Max Brunsfeld
ed11ef557a Fix expansion of repeat rules into recursive rules 
Previously, the way repeat rules were expanded, the auxiliary
rule always needed to be reduced, even if the repeating content
was empty. This caused problems in parse states where some items
contained the repeat rule and some did not. To make those cases
work, the repeat rule had to explicitly be marked as optional.
With this change, that is no longer necessary.
 2014-09-07 09:39:14 -07:00
Max Brunsfeld
545e575508 Revert "Remove the separator characters construct" 
This reverts commit 5cd07648fd.

The separators construct is useful as an optimization. It turns out that
constructing a node for every chunk of whitespace in a document causes a
significant performance regression.

Conflicts:
	src/compiler/build_tables/build_lex_table.cc
	src/compiler/grammar.cc
	src/runtime/parser.c
 2014-09-02 08:03:51 -07:00
Max Brunsfeld
5cd07648fd Remove the separator characters construct 
Now, grammars can handle whitespace by making it another ubiquitous
token, like comments.

For now, this has the side effect of whitespace being included in the
tree that precedes it. This was already an issue for other ubiquitous
tokens though, so it needs to be fixed anyway.
 2014-09-01 20:19:43 -07:00
Max Brunsfeld
1e79ed794b Allow multiple top-level nodes 
Now, the root node of a document is always a document node.
It will often have only one child node which corresponds to the grammar's
start symbol, but not always. Currently, it may have more than one child
if there are ubiquitous tokens such as comments at the beginning of the
document. In the future, it will also be possible be possible to have multiple
for the document to have multiple children if the document is partially parsed.
 2014-08-09 00:00:20 -07:00
Max Brunsfeld
7ba3953f7e Simplify handling of ubiquitous tokens during reduce 2014-08-08 08:46:01 -07:00
Max Brunsfeld
779bf0d745 Don't store tree's hidden children in a separate array 
Just mark hidden trees as such, and skip them when
pretty-printing a tree
 2014-07-17 13:36:53 -07:00
Max Brunsfeld
6e551d6d9f Simplify handling of multiple top-level nodes after parsing 2014-07-14 20:46:20 -07:00
Max Brunsfeld
1c7d2d2d03 Add for-in loops and math assignment operators to js grammar 2014-07-07 13:35:55 -07:00
Max Brunsfeld
77df7fe511 In lexer, always prefer the longest match 
Only use rules' precedence to decide between two tokens
that match the same string
 2014-07-03 08:57:35 -07:00
Max Brunsfeld
c85841364e Add throw statements to js grammar 2014-07-03 08:20:43 -07:00
Max Brunsfeld
83a1b9439e Fix handling of ubiquitous tokens used in grammar rules 2014-07-01 20:47:35 -07:00
Max Brunsfeld
9686c57e90 Allow ubiquitous tokens to also be used in grammar rules 2014-06-26 08:52:42 -07:00
Max Brunsfeld
bb4d83ce47 Add regex postfix flags to javascript grammar 
- Refactor statement terminators in javascript grammar
- Reorganize javascript language tests
 2014-06-11 16:43:27 -07:00
Max Brunsfeld
082560dd6e Fix operator precedence of '.' operator in js grammar 2014-06-11 14:01:38 -07:00
Max Brunsfeld
4ad6278334 Add finally, instance of, typeof, in to js grammar 2014-06-11 11:49:06 -07:00
Max Brunsfeld
e93e254518 In lexer, prefer tokens to skipped separator characters 
This was causing newlines in go and javascript to be parsed as
meaningless separator characters instead of statement terminators
 2014-05-30 13:29:54 -07:00
Max Brunsfeld
2988cc5aa2 Show offending lookahead chars when pretty-printing trees w/ errors 2014-05-26 21:50:01 -07:00
Max Brunsfeld
4c9ac3dada Fix parsing of empty strings in javascript and golang 2014-05-20 09:47:26 -07:00
Max Brunsfeld
2d0f90c7d5 Add try and while statements to js grammar 2014-05-09 21:36:18 -07:00
Max Brunsfeld
e4be585c43 Handle ubiquitous tokens at the beginning of programs 
As a final step before returning the finished parse tree, check if
there are still multiple nodes on the stack. If so, make the inner
nodes children of the top node.
 2014-05-09 12:46:36 -07:00
Max Brunsfeld
4700e33746 Introduce 'ubiquitous_tokens' concept, for parsing comments and such 2014-05-06 12:54:04 -07:00
Max Brunsfeld
bae32adc7b Add constructor calls, pre/postfix operators to js grammar 2014-05-04 13:36:19 -07:00
Max Brunsfeld
1bdd87535a Add prefix math operators +, - to javascript grammar 2014-05-02 07:42:13 -07:00
Max Brunsfeld
a2c125998e Add single quoted strings and regexes to javascript grammar 2014-05-01 12:43:53 -07:00
Max Brunsfeld
801f4bd0a8 Add returns, deletes and bool operators to js grammar 2014-04-25 22:08:11 -07:00
Max Brunsfeld
61692c8bb1 Add error recovery in function calls to javascript gramamr 2014-04-24 13:22:54 -07:00
Max Brunsfeld
68c26a06b1 Add comments to javascript grammar 2014-04-24 13:22:23 -07:00
Max Brunsfeld
52c338ed60 Add some infix math operators to javascript grammar 2014-04-23 22:25:48 -07:00
Max Brunsfeld
7be8d469b8 Add ternary expressions to javascript grammar 2014-04-23 22:15:07 -07:00
Max Brunsfeld
be1c8e0f17 Add dynamic property access to javascript grammar 2014-04-05 15:55:20 -07:00
Max Brunsfeld
2191a7d988 Add switch statements to javascript grammar 2014-04-04 13:10:33 -07:00
Max Brunsfeld
129d2b9314 Remove extra EOF actions in lexer 2014-04-04 08:44:35 -07:00
Max Brunsfeld
1cc7e32e2d Fix handling of tokens consisting of separator characters 
The parser is no longer hard-coded to skip whitespace. Tokens
such as newlines, whose characters overlap with the separator
characters, can now be correctly recognized.
 2014-04-03 19:10:09 -07:00
Max Brunsfeld
13c4e6e648 Tweak format for example grammars 2014-03-28 13:51:32 -07:00
Max Brunsfeld
324f55f1ce Add error recovery to for loops in javascript grammar 2014-03-28 13:11:12 -07:00
Max Brunsfeld
2e11f60710 Add for loops to javascript grammar 2014-03-28 12:59:47 -07:00
Max Brunsfeld
2226234924 Add error recovery to javascript if statements 2014-03-26 23:02:57 -07:00
Max Brunsfeld
6d84e71ceb Clean up javascript language tests 2014-03-26 22:56:58 -07:00
Max Brunsfeld
820b6f4020 Fix infinite loops during table generation for left-recursive rules 
Add function calls for arbitrary expressions and dot property access to javascript grammar
 2014-03-26 22:43:08 -07:00
Max Brunsfeld
6a0e2c08e6 Add more features to javascript grammar 2014-03-26 08:23:13 -07:00
Max Brunsfeld
09e28e7859 Collapse nodes with only one child and no additional text content 2014-03-26 00:10:59 -07:00
Max Brunsfeld
059f7ff9d5 Add else blocks to javascript grammar 2014-03-25 23:40:53 -07:00