traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
368 commits 16 branches 98 tags 37 MiB
Commit graph

91 commits

Author SHA1 Message Date
Max Brunsfeld
649f200831 Expand regex/string rules as part of grammar preparation 
This makes it possible to report errors in regex parsing
 2014-05-19 20:54:59 -07:00
Max Brunsfeld
3e0debf814 Fix compile error on gcc 2014-05-09 15:37:30 -07:00
Max Brunsfeld
292b753914 Move lr_parser into its own header file 2014-05-09 14:43:43 -07:00
Max Brunsfeld
ccc1b41f2a Make separate header files for stack and lexer 2014-05-09 13:32:12 -07:00
Max Brunsfeld
e4be585c43 Handle ubiquitous tokens at the beginning of programs 
As a final step before returning the finished parse tree, check if
there are still multiple nodes on the stack. If so, make the inner
nodes children of the top node.
 2014-05-09 12:46:36 -07:00
Max Brunsfeld
3f374c6547 Tidy up 2014-05-08 13:27:48 -07:00
Max Brunsfeld
34137be12d Represent state ids as unsigned shorts 
This fixes some signedness conversion warnings
 2014-05-08 13:23:46 -07:00
Max Brunsfeld
0a21eee3f0 Remove magic number from generated symbols enums 
The symbol numbers 0 and 1 are reserved for 'error' and 'eof',
so the grammar's start symbol is always 2.
 2014-05-08 13:14:45 -07:00
Max Brunsfeld
013572671f Use smaller integer types for parse table 2014-05-08 08:45:41 -07:00
Max Brunsfeld
4700e33746 Introduce 'ubiquitous_tokens' concept, for parsing comments and such 2014-05-06 12:54:04 -07:00
Max Brunsfeld
5708a181c2 Removed unused field on reduce parse actions 2014-05-01 23:29:01 -07:00
Max Brunsfeld
3a50171249 Expose all grammar compilation errors 2014-05-01 23:28:40 -07:00
Max Brunsfeld
93620b3ed1 Add keyword helper for making higher-priority string tokens 2014-05-01 13:25:20 -07:00
Max Brunsfeld
6d40dcf881 Add token helper for building token rules 
Now you can specify the structure of tokens using
all of the rule functions, not just `str` and `pattern`
 2014-05-01 12:43:29 -07:00
Max Brunsfeld
a437d39773 Add rule precedence construct 
Still need to add some way of expressing left and right
associativity
 2014-04-15 08:40:46 -07:00
Max Brunsfeld
e23604ac52 Fix debugging macros in parser.h 2014-04-14 22:31:11 -07:00
Max Brunsfeld
5145bba53d Silence missing-initializer warnings for gcc 2014-04-12 20:16:16 -07:00
Max Brunsfeld
e1e0cc6278 Make sure conflicts returned by compile are unique 2014-04-10 08:38:14 -07:00
Max Brunsfeld
bd5ec68c96 Get generated parsers building under gcc 2014-04-08 22:11:20 -07:00
Max Brunsfeld
3982b73ed6 Start work on recording parse action conflicts 2014-04-08 08:19:55 -07:00
Max Brunsfeld
5320cad065 Trim trailing whitespace 2014-04-04 13:10:55 -07:00
Max Brunsfeld
1cc7e32e2d Fix handling of tokens consisting of separator characters 
The parser is no longer hard-coded to skip whitespace. Tokens
such as newlines, whose characters overlap with the separator
characters, can now be correctly recognized.
 2014-04-03 19:10:09 -07:00
Max Brunsfeld
13c4e6e648 Tweak format for example grammars 2014-03-28 13:51:32 -07:00
Max Brunsfeld
4454925b5a Clean up parser macros more 2014-03-26 13:03:12 -07:00
Max Brunsfeld
316adc7788 Represent tree symbols as unsigned integers 2014-03-25 23:47:25 -07:00
Max Brunsfeld
f601322956 Clean up macros in parser.h 2014-03-25 19:51:34 -07:00
Max Brunsfeld
80b19cbb83 Construct entire parse table statically 
This removes the need for the 'init_parse_table' function,
which was not really thread safe
 2014-03-25 19:34:17 -07:00
Max Brunsfeld
25861b7f03 Remove reduction-specific collapse flags in favor of globally hidden symbols 2014-03-25 09:05:55 -07:00
Max Brunsfeld
2df56f01c8 Remove public hidden symbol function 
Now, you can't a particular occurrence of a symbol
in a grammar. You can only hide a symbol globally
(right now, by beginning its name with an underscore).
 2014-03-25 08:16:26 -07:00
Max Brunsfeld
48baf056b4 Store grammars' rules in vectors, not maps 
This way, we can keep of the order in which the rules
were given, for resolving reduce/reduce conflicts.
 2014-03-24 13:17:38 -07:00
Max Brunsfeld
3a7c4bb5b1 Store AST nodes' non-hidden children 2014-03-24 01:03:32 -07:00
Max Brunsfeld
95188d84b6 Make tree struct private 2014-03-24 00:34:13 -07:00
Max Brunsfeld
c40411b4d1 Handle unexpected tokens properly 
Also, add null and boolean values to json grammar
 2014-03-22 15:46:58 -07:00
Karen Wang and Max Brunsfeld
f248ece3aa Use table-driven tests for specific parsers 2014-03-22 15:19:13 -07:00
Max Brunsfeld
5869c1ea18 Clean up stack breakdown function 2014-03-21 13:02:25 -07:00
Max Brunsfeld
bdbe69258a Fix bug in stack breakdown procedure 2014-03-21 12:46:23 -07:00
Max Brunsfeld
cbc66b359e Cleanup 2014-03-19 23:10:28 -07:00
Max Brunsfeld
fbe8b0a905 Fix incremental parsing 
Stop collapsing hidden symbols upon reducing them.
Sadly, this messes up the ability to re-use parse
trees. Instead, for now, hide these nodes when
stringifying parse trees
 2014-03-19 19:27:31 -07:00
Max Brunsfeld
7e94a4f1b2 Start work on reading input incrementally 2014-03-18 13:23:21 -07:00
Max Brunsfeld
fbb9b24d7b Refactor ts_tree_children 2014-03-18 12:47:26 -07:00
Max Brunsfeld
8e7f59db1c Allow room in parse table for error and EOF entries 2014-03-18 08:04:17 -07:00
Max Brunsfeld
67b33a615b Refactor generated parsers to used explicit table 
This is slightly slower than encoding the parse table in
flow control, but allows the parser to inspect the parse
table more flexibly. This is needed for incremental parsing.
 2014-03-17 18:43:17 -07:00
Max Brunsfeld
0d6435e24a Pass edit information into parser function 2014-03-15 16:55:35 -07:00
Max Brunsfeld
464a0e8dad Clean up parser header further 2014-03-15 15:15:59 -07:00
Max Brunsfeld
0dc3a95d0c Refactor parser header 
Make separate lexer, stack and parser structs.
 2014-03-15 14:43:50 -07:00
Max Brunsfeld
05a5f9c124 Add function for notifying documents of edits 2014-03-12 13:39:12 -07:00
Max Brunsfeld
ff0c8a98b8 Consolidate reading of input chunks in parser 2014-03-11 12:43:13 -07:00
Max Brunsfeld
42e9a264f3 In parser, read chunked input correctly 2014-03-10 13:25:31 -07:00
Max Brunsfeld
31a58bc7e4 Make include guards pass cpplint 2014-03-09 22:05:24 -07:00
Max Brunsfeld
39aa0ccc91 Add script to trim whitespace 2014-03-09 19:49:35 -07:00