Commit graph

14 commits

Author SHA1 Message Date
Max Brunsfeld
1ff7cedf40 Unify ubiquitous tokens and lexical separators in API 2014-09-07 22:16:45 -07:00
Max Brunsfeld
545e575508 Revert "Remove the separator characters construct"
This reverts commit 5cd07648fd.

The separators construct is useful as an optimization. It turns out that
constructing a node for every chunk of whitespace in a document causes a
significant performance regression.

Conflicts:
	src/compiler/build_tables/build_lex_table.cc
	src/compiler/grammar.cc
	src/runtime/parser.c
2014-09-02 08:03:51 -07:00
Max Brunsfeld
5cd07648fd Remove the separator characters construct
Now, grammars can handle whitespace by making it another ubiquitous
token, like comments.

For now, this has the side effect of whitespace being included in the
tree that precedes it. This was already an issue for other ubiquitous
tokens though, so it needs to be fixed anyway.
2014-09-01 20:19:43 -07:00
Max Brunsfeld
98cc2f2264 Auto-format all source code with clang-format 2014-07-21 13:20:00 -07:00
Max Brunsfeld
649f200831 Expand regex/string rules as part of grammar preparation
This makes it possible to report errors in regex parsing
2014-05-19 20:54:59 -07:00
Max Brunsfeld
b9393b5c1a Refactor grammar helper functions 2014-05-01 20:02:06 -07:00
Max Brunsfeld
b86203d205 Extract shared helper functions from example grammars 2014-04-27 21:45:05 -07:00
Max Brunsfeld
c2abfd2d03 Parse '.' in regexes 2014-04-24 13:21:46 -07:00
Max Brunsfeld
2cd595d81a Clean up helper functions in example grammars 2014-03-29 19:02:00 -07:00
Max Brunsfeld
13c4e6e648 Tweak format for example grammars 2014-03-28 13:51:32 -07:00
Max Brunsfeld
2df56f01c8 Remove public hidden symbol function
Now, you can't a particular occurrence of a symbol
in a grammar. You can only hide a symbol globally
(right now, by beginning its name with an underscore).
2014-03-25 08:16:26 -07:00
Max Brunsfeld
48baf056b4 Store grammars' rules in vectors, not maps
This way, we can keep of the order in which the rules
were given, for resolving reduce/reduce conflicts.
2014-03-24 13:17:38 -07:00
Max Brunsfeld
671f1a1ddc Start work on javascript grammar 2014-03-24 09:14:29 -07:00
Max Brunsfeld
06cd18b483 Put example grammars in tree_sitter::examples ns 2014-03-24 07:19:31 -07:00