traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
506 commits 16 branches 98 tags 37 MiB
Commit graph

112 commits

Author SHA1 Message Date
Max Brunsfeld
346cf4fe5d Remove LEX_PANIC macro 2014-08-26 13:12:12 -07:00
Max Brunsfeld
0bb5663f0f Refactor - represent char sets in terms of inclusions and exclusions 2014-08-23 14:25:45 -07:00
Max Brunsfeld
4327f3ed26 Refactor parser and stack 2014-08-09 01:03:55 -07:00
Max Brunsfeld
1e79ed794b Allow multiple top-level nodes 
Now, the root node of a document is always a document node.
It will often have only one child node which corresponds to the grammar's
start symbol, but not always. Currently, it may have more than one child
if there are ubiquitous tokens such as comments at the beginning of the
document. In the future, it will also be possible be possible to have multiple
for the document to have multiple children if the document is partially parsed.
 2014-08-09 00:00:20 -07:00
Max Brunsfeld
9366f11dcb In generated C, only format printable characters as char literals 2014-08-07 08:12:15 -07:00
Max Brunsfeld
eecbcccee0 Remove generated parsers' dependency on the runtime library 
Generated parsers no longer export a parser constructor function.
They now export an opaque Language object which can be set on
Documents directly. This way, the logic for constructing parsers
lives entirely in the runtime. The Languages are just structs which
have no load-time dependency on the runtime
 2014-07-30 23:40:02 -07:00
Max Brunsfeld
fbdd6ee591 Update todo 2014-07-21 13:20:00 -07:00
Max Brunsfeld
98cc2f2264 Auto-format all source code with clang-format 2014-07-21 13:20:00 -07:00
Max Brunsfeld
4d14a65e22 In build_parse_table, switch recursion to explicit iteration 2014-07-13 18:06:37 -07:00
Max Brunsfeld
44c4bf5f5e Refactor add_ubiquitous_token_actions method 2014-07-11 13:21:44 -07:00
Max Brunsfeld
1c7d2d2d03 Add for-in loops and math assignment operators to js grammar 2014-07-07 13:35:55 -07:00
Max Brunsfeld
c5c25d30bb Fix golang grammar after latest lexer change 2014-07-03 13:28:08 -07:00
Max Brunsfeld
77df7fe511 In lexer, always prefer the longest match 
Only use rules' precedence to decide between two tokens
that match the same string
 2014-07-03 08:57:35 -07:00
Max Brunsfeld
c85841364e Add throw statements to js grammar 2014-07-03 08:20:43 -07:00
Max Brunsfeld
83a1b9439e Fix handling of ubiquitous tokens used in grammar rules 2014-07-01 20:47:35 -07:00
Max Brunsfeld
9686c57e90 Allow ubiquitous tokens to also be used in grammar rules 2014-06-26 08:52:42 -07:00
Max Brunsfeld
7df35f9b8d Make separate types for syntax and lexical grammars 
This way, the separator characters can be added as a field to
lexical grammars only
 2014-06-25 13:27:16 -07:00
Max Brunsfeld
bb4d83ce47 Add regex postfix flags to javascript grammar 
- Refactor statement terminators in javascript grammar
- Reorganize javascript language tests
 2014-06-11 16:43:27 -07:00
Max Brunsfeld
082560dd6e Fix operator precedence of '.' operator in js grammar 2014-06-11 14:01:38 -07:00
Max Brunsfeld
3cd031af38 Add keypattern rule helper 
This way, pattern rules (e.g. golang's comment) can be easily given the
same precedence as keyword rules.
 2014-06-11 12:40:49 -07:00
Max Brunsfeld
174f306e2a Fix precedence of comments vs '/' operator 2014-06-11 12:27:58 -07:00
Max Brunsfeld
4ad6278334 Add finally, instance of, typeof, in to js grammar 2014-06-11 11:49:06 -07:00
Max Brunsfeld
c91c5cb730 Add range statements to golang grammar 2014-06-10 14:11:25 -07:00
Max Brunsfeld
53bc633a22 Add var decl and if statements to golang grammar 2014-06-10 13:27:55 -07:00
Max Brunsfeld
1c93d5e1a6 Add declarations w/o initialization to golang grammar 2014-06-10 11:57:45 -07:00
Max Brunsfeld
123d3b26d8 Add more expressions, statements to golang grammar 2014-06-10 11:33:05 -07:00
Max Brunsfeld
54a555168d Add accessor methods on Grammar 2014-06-09 21:05:25 -07:00
Max Brunsfeld
e93e254518 In lexer, prefer tokens to skipped separator characters 
This was causing newlines in go and javascript to be parsed as
meaningless separator characters instead of statement terminators
 2014-05-30 13:29:54 -07:00
Max Brunsfeld
844f73c193 Remove unnecessary comment rule from JS grammar 2014-05-26 21:48:35 -07:00
Max Brunsfeld
4c9ac3dada Fix parsing of empty strings in javascript and golang 2014-05-20 09:47:26 -07:00
Max Brunsfeld
c30055ba18 Fix symbol names for extracted tokens 2014-05-20 08:30:58 -07:00
Max Brunsfeld
649f200831 Expand regex/string rules as part of grammar preparation 
This makes it possible to report errors in regex parsing
 2014-05-19 20:54:59 -07:00
Max Brunsfeld
5245bc01fe Backfill tests for token extraction in auxiliary rules 2014-05-19 19:05:54 -07:00
Max Brunsfeld
2d0f90c7d5 Add try and while statements to js grammar 2014-05-09 21:36:18 -07:00
Max Brunsfeld
10d3801d7e Fix missing symbol names for keywords 2014-05-09 16:14:48 -07:00
Max Brunsfeld
34137be12d Represent state ids as unsigned shorts 
This fixes some signedness conversion warnings
 2014-05-08 13:23:46 -07:00
Max Brunsfeld
0a21eee3f0 Remove magic number from generated symbols enums 
The symbol numbers 0 and 1 are reserved for 'error' and 'eof',
so the grammar's start symbol is always 2.
 2014-05-08 13:14:45 -07:00
Max Brunsfeld
4700e33746 Introduce 'ubiquitous_tokens' concept, for parsing comments and such 2014-05-06 12:54:04 -07:00
Max Brunsfeld
1d314d71c2 Separate functions for building parse and lex tables 
Now, instead of adding states to the lex table as they are needed
by the parse states, we iterate over the parse states after the fact
and set up their corresponding lex states. This has the nice side
effect that the lex states are in a more readable order.
 2014-05-04 22:07:52 -07:00
Max Brunsfeld
bae32adc7b Add constructor calls, pre/postfix operators to js grammar 2014-05-04 13:36:19 -07:00
Max Brunsfeld
1bdd87535a Add prefix math operators +, - to javascript grammar 2014-05-02 07:42:13 -07:00
Max Brunsfeld
3a50171249 Expose all grammar compilation errors 2014-05-01 23:28:40 -07:00
Max Brunsfeld
b9393b5c1a Refactor grammar helper functions 2014-05-01 20:02:06 -07:00
Max Brunsfeld
2906125824 Refactor javascript grammar to use new keyword helper 2014-05-01 13:25:50 -07:00
Max Brunsfeld
60e2728acc Refactor string and regex specification in javascript grammar 2014-05-01 12:48:01 -07:00
Max Brunsfeld
a2c125998e Add single quoted strings and regexes to javascript grammar 2014-05-01 12:43:53 -07:00
Max Brunsfeld
6ea4e6b2b0 Give rules::Visitor a virtual destructor 2014-04-27 23:19:11 -07:00
Max Brunsfeld
4dd1f9fbce Cache state transition computations by grammar rule 
This results in a huge speed increase
 2014-04-27 23:03:37 -07:00
Max Brunsfeld
b86203d205 Extract shared helper functions from example grammars 2014-04-27 21:45:05 -07:00
Max Brunsfeld
29bbff655c Store choice rules using vectors, not pairs 2014-04-26 23:21:09 -07:00