traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
669 commits 16 branches 98 tags 37 MiB
Commit graph

183 commits

Author SHA1 Message Date
Max Brunsfeld
7872ddd21b Improve reduce/reduce conflict error message 2015-03-16 11:59:45 -07:00
Max Brunsfeld
9a198562e0 Treat parse conflicts as errors in grammar compilation 
For now, only reduce/reduce conflicts w/ no tie-breaking precedence
are treated as errors. The rest are dropped, because shift/reduce
conflicts are currently very common because we don't have a way
of specifying associativity along w/ precedence.
 2015-03-15 20:31:41 -07:00
Max Brunsfeld
3458fa6e50 Fix non-deterministic order in conflict description 2015-03-07 11:02:21 -08:00
Max Brunsfeld
2d436cf141 Identify fragile reductions at compile time 2015-02-21 15:11:03 -08:00
Max Brunsfeld
ccb0d0d043 Store parse items' list of consumed symbols 
Not just the consumed symbol count.
 2015-02-20 22:53:27 -08:00
Max Brunsfeld
109b5616d3 Remove unused arg to action_takes_precedence 2015-01-17 14:14:49 -08:00
Max Brunsfeld
160fca6579 Refactor avoidance of redundant repeat rules 2015-01-14 21:11:19 -08:00
Max Brunsfeld
a0d9da9d5c Rename static 'Build' methods to 'build' 2015-01-14 21:11:05 -08:00
Max Brunsfeld
34d96909d1 Move {Syntax,Lexical}Grammar into separate files 2015-01-14 21:10:41 -08:00
Max Brunsfeld
0d267e41aa Replace LexConflictManager class with action_takes_precedence function 2015-01-11 19:50:58 -08:00
Max Brunsfeld
5dc08ccce9 Include names of in-progress rules in shift/reduce conflicts 2014-11-05 18:39:50 -08:00
Max Brunsfeld
eaeb51a96f Don't open parser C file until code is generated 2014-10-21 08:49:42 -07:00
Max Brunsfeld
3bcb221379 Include non-terminal lookahead symbols for reduction actions 
This is necessary for re-using the right subtree after an edit
 2014-10-10 12:06:16 -07:00
Max Brunsfeld
070dc76050 Generate correct C literals for non-ascii characters 2014-09-28 18:40:15 -07:00
Max Brunsfeld
cb5ecbd491 Handle string and regex rules w/ non-ascii chars 2014-09-28 18:21:22 -07:00
Max Brunsfeld
a2b80098b2 Fix destination directory in compile_examples spec 2014-09-11 12:46:46 -07:00
Max Brunsfeld
cd8a683229 Improve error messages for invalid ubiquitous tokens 2014-09-10 13:02:16 -07:00
Max Brunsfeld
e9dad529f5 Make descriptions more consistent in compiler specs 2014-09-09 13:01:18 -07:00
Max Brunsfeld
1ff7cedf40 Unify ubiquitous tokens and lexical separators in API 2014-09-07 22:16:45 -07:00
Max Brunsfeld
a46f9d950c Handle '\s' correctly in regexps 2014-09-07 16:05:43 -07:00
Max Brunsfeld
ed11ef557a Fix expansion of repeat rules into recursive rules 
Previously, the way repeat rules were expanded, the auxiliary
rule always needed to be reduced, even if the repeating content
was empty. This caused problems in parse states where some items
contained the repeat rule and some did not. To make those cases
work, the repeat rule had to explicitly be marked as optional.
With this change, that is no longer necessary.
 2014-09-07 09:39:14 -07:00
Max Brunsfeld
d3204d3526 Include '_' in '\w' regex character class 2014-09-05 18:41:12 -07:00
Max Brunsfeld
545e575508 Revert "Remove the separator characters construct" 
This reverts commit 5cd07648fd.

The separators construct is useful as an optimization. It turns out that
constructing a node for every chunk of whitespace in a document causes a
significant performance regression.

Conflicts:
	src/compiler/build_tables/build_lex_table.cc
	src/compiler/grammar.cc
	src/runtime/parser.c
 2014-09-02 08:03:51 -07:00
Max Brunsfeld
5cd07648fd Remove the separator characters construct 
Now, grammars can handle whitespace by making it another ubiquitous
token, like comments.

For now, this has the side effect of whitespace being included in the
tree that precedes it. This was already an issue for other ubiquitous
tokens though, so it needs to be fixed anyway.
 2014-09-01 20:19:43 -07:00
Max Brunsfeld
226ffd6b5b Fix initializer list deduction warnings in specs 2014-08-27 22:23:45 -07:00
Max Brunsfeld
9338249075 Remove implicit CharacterRange constructors 
Also fix misc smaller lint errors
 2014-08-23 14:52:44 -07:00
Max Brunsfeld
0bb5663f0f Refactor - represent char sets in terms of inclusions and exclusions 2014-08-23 14:25:45 -07:00
Max Brunsfeld
6f374fddff Tweak format for pretty printing some classes 2014-08-23 13:52:00 -07:00
Max Brunsfeld
2963b08f79 Eliminate duplicates when constructing choice rules 2014-08-21 20:04:42 -07:00
Max Brunsfeld
b155994491 Fix indentation in specs 2014-08-07 08:11:21 -07:00
Max Brunsfeld
01571da30d Handle more escaped characters in regexps 2014-08-03 21:57:21 -07:00
Max Brunsfeld
6a8addb84f Fix segfault when grammar consists of a single token 2014-07-23 18:26:10 -07:00
Max Brunsfeld
02904085c2 Make separate helper scripts for testing compiler and runtime 2014-07-17 22:20:14 -07:00
Max Brunsfeld
6951acb13b Fix error when grammar contains to error productions 2014-07-13 21:26:21 -07:00
Max Brunsfeld
7dfaba2954 Fix conflict manager spec 2014-07-13 18:10:32 -07:00
Max Brunsfeld
b217cd38fb Handle built-in symbols correctly in conflict manager 2014-07-13 17:59:57 -07:00
Max Brunsfeld
77df7fe511 In lexer, always prefer the longest match 
Only use rules' precedence to decide between two tokens
that match the same string
 2014-07-03 08:57:35 -07:00
Max Brunsfeld
83a1b9439e Fix handling of ubiquitous tokens used in grammar rules 2014-07-01 20:47:35 -07:00
Max Brunsfeld
3be648593e merge_{sym,char}_transitions -> merge_{sym,char}_transition 2014-06-28 17:02:48 -07:00
Max Brunsfeld
9bad5dff3e Avoid unnecessary std::map construction when merging transition sets 2014-06-26 13:42:42 -07:00
Max Brunsfeld
9686c57e90 Allow ubiquitous tokens to also be used in grammar rules 2014-06-26 08:52:42 -07:00
Max Brunsfeld
a9dff20658 Make grammars' separator characters configurable 2014-06-26 07:31:08 -07:00
Max Brunsfeld
7df35f9b8d Make separate types for syntax and lexical grammars 
This way, the separator characters can be added as a field to
lexical grammars only
 2014-06-25 13:27:16 -07:00
Max Brunsfeld
d5674d33c4 Add test helper stream method for pairs 2014-06-24 21:54:38 -07:00
Max Brunsfeld
81880e000e Tweak header include paths in tests 2014-06-23 18:50:03 -07:00
Max Brunsfeld
2c382b7363 Trim trailing whitespace 2014-06-16 21:33:35 -07:00
Max Brunsfeld
1daaf4485f Refactor item set transition functions 2014-06-16 13:37:34 -07:00
Max Brunsfeld
39c1ab2d50 Refactor item_set_closure 
Inline unnecessary  function
 2014-06-16 13:20:39 -07:00
Max Brunsfeld
7a2c2c1c90 Store ParseItemSets as maps, w/ core items as keys 
ParseItem no longer has a lookahead_sym field; it now represents
the 'core' of a parse item. The lookahead context is stored separately,
as a set per core item. This makes iterating, copying and merging item
sets more efficient, because before, the core items were repeated for each
different lookahead symbol.

Also, the memoization in sym_transitions(ParseItemSet) has been removed.
Maybe I'll add it back later.
 2014-06-16 08:35:20 -07:00
Max Brunsfeld
174f306e2a Fix precedence of comments vs '/' operator 2014-06-11 12:27:58 -07:00