traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
6118 commits 16 branches 98 tags 37 MiB
Commit graph

6118 commits

This branch
This branch
All branches
Author SHA1 Message Date
Max Brunsfeld
4c9ac3dada Fix parsing of empty strings in javascript and golang 2014-05-20 09:47:26 -07:00
Max Brunsfeld
c30055ba18 Fix symbol names for extracted tokens 2014-05-20 08:30:58 -07:00
Max Brunsfeld
649f200831 Expand regex/string rules as part of grammar preparation 
This makes it possible to report errors in regex parsing
 2014-05-19 20:54:59 -07:00
Max Brunsfeld
5245bc01fe Backfill tests for token extraction in auxiliary rules 2014-05-19 19:05:54 -07:00
Max Brunsfeld
608b5ce02b Cleanup - extract method in parse table builder 2014-05-12 08:53:28 -07:00
Max Brunsfeld
2d0f90c7d5 Add try and while statements to js grammar 2014-05-09 21:36:18 -07:00
Max Brunsfeld
e8760eee34 Don't check in cpplint 2014-05-09 16:32:57 -07:00
Max Brunsfeld
10d3801d7e Fix missing symbol names for keywords 2014-05-09 16:14:48 -07:00
Max Brunsfeld
3e0debf814 Fix compile error on gcc 2014-05-09 15:37:30 -07:00
Max Brunsfeld
963768eb8a Remove unnecessary parser helper functions 2014-05-09 15:03:29 -07:00
Max Brunsfeld
292b753914 Move lr_parser into its own header file 2014-05-09 14:43:43 -07:00
Max Brunsfeld
ccc1b41f2a Make separate header files for stack and lexer 2014-05-09 13:32:12 -07:00
Max Brunsfeld
e4be585c43 Handle ubiquitous tokens at the beginning of programs 
As a final step before returning the finished parse tree, check if
there are still multiple nodes on the stack. If so, make the inner
nodes children of the top node.
 2014-05-09 12:46:36 -07:00
Max Brunsfeld
3f374c6547 Tidy up 2014-05-08 13:27:48 -07:00
Max Brunsfeld
34137be12d Represent state ids as unsigned shorts 
This fixes some signedness conversion warnings
 2014-05-08 13:23:46 -07:00
Max Brunsfeld
0a21eee3f0 Remove magic number from generated symbols enums 
The symbol numbers 0 and 1 are reserved for 'error' and 'eof',
so the grammar's start symbol is always 2.
 2014-05-08 13:14:45 -07:00
Max Brunsfeld
013572671f Use smaller integer types for parse table 2014-05-08 08:45:41 -07:00
Max Brunsfeld
4700e33746 Introduce 'ubiquitous_tokens' concept, for parsing comments and such 2014-05-06 12:54:04 -07:00
Max Brunsfeld
b010e1667e Fix parse action equality method 2014-05-06 12:51:38 -07:00
Max Brunsfeld
d91bc718a0 Add basic test for parse table builder 2014-05-04 23:28:53 -07:00
Max Brunsfeld
63c0e27501 Clean up table builder functions 2014-05-04 23:04:34 -07:00
Max Brunsfeld
1d314d71c2 Separate functions for building parse and lex tables 
Now, instead of adding states to the lex table as they are needed
by the parse states, we iterate over the parse states after the fact
and set up their corresponding lex states. This has the nice side
effect that the lex states are in a more readable order.
 2014-05-04 22:07:52 -07:00
Max Brunsfeld
b998bb35af Don't use initializer lists for tuples 
gcc doesn't allow it.
 2014-05-04 19:49:09 -07:00
Max Brunsfeld
bae32adc7b Add constructor calls, pre/postfix operators to js grammar 2014-05-04 13:36:19 -07:00
Max Brunsfeld
1bdd87535a Add prefix math operators +, - to javascript grammar 2014-05-02 07:42:13 -07:00
Max Brunsfeld
b97e4f1de8 Clean up intern symbols function 2014-05-01 23:40:04 -07:00
Max Brunsfeld
5708a181c2 Removed unused field on reduce parse actions 2014-05-01 23:29:01 -07:00
Max Brunsfeld
3a50171249 Expose all grammar compilation errors 2014-05-01 23:28:40 -07:00
Max Brunsfeld
e36d586817 Update todo 2014-05-01 20:05:03 -07:00
Max Brunsfeld
b9393b5c1a Refactor grammar helper functions 2014-05-01 20:02:06 -07:00
Max Brunsfeld
2906125824 Refactor javascript grammar to use new keyword helper 2014-05-01 13:25:50 -07:00
Max Brunsfeld
93620b3ed1 Add keyword helper for making higher-priority string tokens 2014-05-01 13:25:20 -07:00
Max Brunsfeld
60e2728acc Refactor string and regex specification in javascript grammar 2014-05-01 12:48:01 -07:00
Max Brunsfeld
a2c125998e Add single quoted strings and regexes to javascript grammar 2014-05-01 12:43:53 -07:00
Max Brunsfeld
6d40dcf881 Add token helper for building token rules 
Now you can specify the structure of tokens using
all of the rule functions, not just `str` and `pattern`
 2014-05-01 12:43:29 -07:00
Max Brunsfeld
d685edf015 Backfill test for item set transitions 2014-04-29 08:18:41 -07:00
Max Brunsfeld
0d763d229d cpplint 2014-04-28 21:46:43 -07:00
Max Brunsfeld
25eda9d889 ISymbol -> Symbol 
Interned symbols are now the main type of symbol in use
 2014-04-28 20:43:27 -07:00
Max Brunsfeld
faf80aadac Symbol -> NamedSymbol 2014-04-28 20:15:49 -07:00
Max Brunsfeld
66c033e411 Optimize parse item equality 2014-04-28 18:31:03 -07:00
Max Brunsfeld
70ba76762c Optimize item set transitions function further 2014-04-28 18:30:50 -07:00
Max Brunsfeld
ae2450d282 Fix dangling reference error on gcc 2014-04-28 13:06:16 -07:00
Max Brunsfeld
0323a696f1 Update todo 2014-04-28 09:03:29 -07:00
Max Brunsfeld
c53c0dfe37 Record running time in test script 2014-04-28 08:11:48 -07:00
Max Brunsfeld
6ea4e6b2b0 Give rules::Visitor a virtual destructor 2014-04-27 23:19:11 -07:00
Max Brunsfeld
4dd1f9fbce Cache state transition computations by grammar rule 
This results in a huge speed increase
 2014-04-27 23:03:37 -07:00
Max Brunsfeld
4eece88e18 Remove duplicate set insert 2014-04-27 22:08:40 -07:00
Max Brunsfeld
076a27be4a Fix compile error on gcc 2014-04-27 22:08:01 -07:00
Max Brunsfeld
b2cb78166e Give Rule a virtual destructor 
Not needed at the moment because rule pointers are are always wrapped
in shared_ptrs. Still, don't want to forget this if I stopped using shared_ptrs
at some point.
 2014-04-27 21:49:56 -07:00
Max Brunsfeld
b86203d205 Extract shared helper functions from example grammars 2014-04-27 21:45:05 -07:00