traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1929 commits 16 branches 98 tags 37 MiB
Commit graph

54 commits

Author SHA1 Message Date
Max Brunsfeld
e17cd42e47 Perform keyword optimization using explicitly selected word token 
rather than trying to infer the word token automatically.

Co-Authored-By: Ashi Krishnan <queerviolet@github.com>
 2018-06-14 09:35:54 -07:00
Max Brunsfeld
6fca8f2f4d Make ts_compile_grammar take an optional log file, start logging to it 2018-05-24 16:01:14 -07:00
Max Brunsfeld
94dc703bfc Require that grammars' start rules be visible 2017-08-04 17:07:37 -07:00
Max Brunsfeld
db4b9ebc7c Implement Rule as a union rather than an abstract base class 2017-03-17 13:29:31 -07:00
Max Brunsfeld
4131e1c16e Return an error when external token name matches non-terminal rule 2017-01-31 11:36:51 -08:00
Max Brunsfeld
996ca91e70 Disallow syntax rules that match the empty string (for now) 2016-11-30 23:19:54 -08:00
Max Brunsfeld
b80a330a74 Fix assorted memory leaks in test code 2016-02-05 12:23:54 -08:00
Max Brunsfeld
d4632ab9a9 Make the compile function plain C and take a JSON grammar 2016-01-11 12:33:48 -08:00
Max Brunsfeld
b69e19c525 Add plain C API for compiling a JSON grammar 2016-01-10 13:44:22 -08:00
Max Brunsfeld
36870bfced Make Grammar a simple struct 2016-01-08 15:51:30 -08:00
Max Brunsfeld
1c6ad5f7e4 Rename ubiquitous_tokens -> extra_tokens in compiler API 
They were already called this in the runtime code.
'Extra' is just easier to say.
 2015-12-17 15:50:50 -08:00
Max Brunsfeld
9959fe35b0 Allow associativity to be specified in rules w/o precedence 2015-10-13 11:25:28 -07:00
Max Brunsfeld
82726ad53b Define repeat rule in terms of repeat1 rule 2015-10-12 19:22:05 -07:00
Max Brunsfeld
e6f3239bef Move stream operator definitions to spec helpers 
This is one less thing for users to worry about when compiling and linking
the library itself
 2015-09-10 10:12:11 -07:00
Max Brunsfeld
67241e3052 Don't use std::set in public compiler header 
Just use vectors
 2015-09-08 23:43:45 -07:00
Max Brunsfeld
bd77ab1ac9 Move public rule functions out of rule namespace 
This way, there's only one public namespace: tree_sitter
 2015-09-03 17:49:20 -07:00
Max Brunsfeld
e386c634aa Remove unused Grammar methods 2015-09-03 17:19:43 -07:00
Max Brunsfeld
97bb7a26cf Fix precedence calculations when building parse table 
* Recurse into choice rules
* Compute reduction precedence differently than shift precedence
 2015-09-02 13:05:54 -07:00
Max Brunsfeld
f9b057f3a9 clang-format everything 2015-07-27 18:29:48 -07:00
Max Brunsfeld
c9a482bbf3 Add expected_conflicts field to grammar 2015-06-26 16:14:08 -05:00
Max Brunsfeld
b1f8ba6202 Replace {left,right}_assoc w/ prec, with an associativity argument 2015-03-23 21:06:31 -07:00
Max Brunsfeld
a19b0e75ac 🔥 keyword and keypattern functions 
Just make strings have higher precedence than regexps.
 2015-03-22 16:00:26 -07:00
Max Brunsfeld
80ec303b10 Replace prec rule w/ left_assoc and right_assoc 
Consider shift/reduce conflicts to be compilation errors unless
they are resolved by a specified associativity.
 2015-03-16 23:12:34 -07:00
Max Brunsfeld
9a198562e0 Treat parse conflicts as errors in grammar compilation 
For now, only reduce/reduce conflicts w/ no tie-breaking precedence
are treated as errors. The rest are dropped, because shift/reduce
conflicts are currently very common because we don't have a way
of specifying associativity along w/ precedence.
 2015-03-15 20:31:41 -07:00
Max Brunsfeld
2e7ffb4d14 Tweak auto-format settings 
Prefer lines that exceed 80 characters by a small margin to
line breaks in argument lists
 2014-09-09 13:15:40 -07:00
Max Brunsfeld
1ff7cedf40 Unify ubiquitous tokens and lexical separators in API 2014-09-07 22:16:45 -07:00
Max Brunsfeld
545e575508 Revert "Remove the separator characters construct" 
This reverts commit 5cd07648fd.

The separators construct is useful as an optimization. It turns out that
constructing a node for every chunk of whitespace in a document causes a
significant performance regression.

Conflicts:
	src/compiler/build_tables/build_lex_table.cc
	src/compiler/grammar.cc
	src/runtime/parser.c
 2014-09-02 08:03:51 -07:00
Max Brunsfeld
5cd07648fd Remove the separator characters construct 
Now, grammars can handle whitespace by making it another ubiquitous
token, like comments.

For now, this has the side effect of whitespace being included in the
tree that precedes it. This was already an issue for other ubiquitous
tokens though, so it needs to be fixed anyway.
 2014-09-01 20:19:43 -07:00
Max Brunsfeld
18e918b704 Fix lint errors in public compiler header 2014-07-22 09:03:33 -07:00
Max Brunsfeld
98cc2f2264 Auto-format all source code with clang-format 2014-07-21 13:20:00 -07:00
Max Brunsfeld
83a1b9439e Fix handling of ubiquitous tokens used in grammar rules 2014-07-01 20:47:35 -07:00
Max Brunsfeld
7df35f9b8d Make separate types for syntax and lexical grammars 
This way, the separator characters can be added as a field to
lexical grammars only
 2014-06-25 13:27:16 -07:00
Max Brunsfeld
3cd031af38 Add keypattern rule helper 
This way, pattern rules (e.g. golang's comment) can be easily given the
same precedence as keyword rules.
 2014-06-11 12:40:49 -07:00
Max Brunsfeld
11acc7d087 Fix missing initializer warnings 2014-06-09 21:47:57 -07:00
Max Brunsfeld
54a555168d Add accessor methods on Grammar 2014-06-09 21:05:25 -07:00
Max Brunsfeld
649f200831 Expand regex/string rules as part of grammar preparation 
This makes it possible to report errors in regex parsing
 2014-05-19 20:54:59 -07:00
Max Brunsfeld
4700e33746 Introduce 'ubiquitous_tokens' concept, for parsing comments and such 2014-05-06 12:54:04 -07:00
Max Brunsfeld
3a50171249 Expose all grammar compilation errors 2014-05-01 23:28:40 -07:00
Max Brunsfeld
93620b3ed1 Add keyword helper for making higher-priority string tokens 2014-05-01 13:25:20 -07:00
Max Brunsfeld
6d40dcf881 Add token helper for building token rules 
Now you can specify the structure of tokens using
all of the rule functions, not just `str` and `pattern`
 2014-05-01 12:43:29 -07:00
Max Brunsfeld
a437d39773 Add rule precedence construct 
Still need to add some way of expressing left and right
associativity
 2014-04-15 08:40:46 -07:00
Max Brunsfeld
e1e0cc6278 Make sure conflicts returned by compile are unique 2014-04-10 08:38:14 -07:00
Max Brunsfeld
3982b73ed6 Start work on recording parse action conflicts 2014-04-08 08:19:55 -07:00
Max Brunsfeld
2df56f01c8 Remove public hidden symbol function 
Now, you can't a particular occurrence of a symbol
in a grammar. You can only hide a symbol globally
(right now, by beginning its name with an underscore).
 2014-03-25 08:16:26 -07:00
Max Brunsfeld
48baf056b4 Store grammars' rules in vectors, not maps 
This way, we can keep of the order in which the rules
were given, for resolving reduce/reduce conflicts.
 2014-03-24 13:17:38 -07:00
Max Brunsfeld
31a58bc7e4 Make include guards pass cpplint 2014-03-09 22:05:24 -07:00
Max Brunsfeld
39aa0ccc91 Add script to trim whitespace 2014-03-09 19:49:35 -07:00
Max Brunsfeld
63718d5bad Add basic support for error recovery 2014-02-26 19:03:43 -08:00
Max Brunsfeld
946088bccc Remove unused typedef from compiler.h 2014-02-21 08:23:20 -08:00
Max Brunsfeld
688fe993cf Remove rules that don't need to be public from compiler.h 2014-02-20 23:05:32 -08:00