traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1835 commits 16 branches 98 tags 37 MiB
Commit graph

156 commits

Author SHA1 Message Date
Max Brunsfeld
379a2fd121 Incrementally build a tree of skipped tokens 
Rather than pushing them to the stack individually
 2018-04-09 12:29:22 -07:00
Max Brunsfeld
3c737d8295 Respect -D flag in fuzzing examples 2018-04-06 16:09:17 -07:00
Max Brunsfeld
09be0b6ef5 Store trees' children in TreeArrays, not w/ separate pointer and length 2018-04-06 13:26:18 -07:00
Max Brunsfeld
a6cf2e87e7 Fix halt_on_error tests 2018-04-06 13:26:18 -07:00
Max Brunsfeld
80f856cef5 Maintain a total node count on every tree 
This simplifies (and fixes bugs in) the parse stack's tracking of its
total node count since the last error, which is needed for error
recovery.
 2018-04-06 13:26:18 -07:00
Max Brunsfeld
e59558c83b Allow stack versions to be temporarily paused 
This way, when detecting an error, we can defer the decision about
whether to bail or recover until all stack versions are processed.
 2018-04-06 13:26:18 -07:00
Max Brunsfeld
1ca261c79b Fix some regex parsing bugs 
* Allow escape sequences to be used in ranges
* Don't give special meaning to dashes outside of character classes
 2018-04-06 12:46:06 -07:00
Axel Hecht
345e344377 Tests for issue 158 2018-04-05 14:39:25 +02:00
Max Brunsfeld
dbe77e7199 Simplify testing-only ts_stack_iterate function 2018-03-29 17:50:07 -07:00
Max Brunsfeld
5520983144 Clean up Stack API 
* Remove StackPopResult
* Rename top_state() -> state()
* Rename top_position() -> position()
* Improve docs
 2018-03-29 17:37:54 -07:00
Max Brunsfeld
ee995c3d6b Avoid redundant retains/releases by giving ts_stack_push move semantics 2018-03-29 17:18:43 -07:00
Max Brunsfeld
186f70649c Consolidate the unify for detecting conflicting tokens 2018-03-28 10:03:09 -07:00
Max Brunsfeld
a8bc67ac42 Allow LookaheadSet::for_each to terminate early 2018-03-28 10:03:09 -07:00
Max Brunsfeld
43e14332ed Avoid creating duplicate metadata rules 2018-03-28 10:03:09 -07:00
Max Brunsfeld
b7d0606fbd Be less conservative in merging parse states with external tokens 
Also, clean up the internal representation of external tokens
 2018-03-16 16:00:40 -07:00
Max Brunsfeld
fe29173d5f
Merge pull request #142 from tree-sitter/fuzz-halt-recover 
Add 'halt' and 'recover' modes to fuzzer
 2018-03-14 09:28:58 -07:00
Phil Turnbull
269b1a0864 Update repo for libFuzzer 
libFuzzer has now been broken out from LLVM and can be built separately
 2018-03-12 13:08:58 -07:00
Max Brunsfeld
7183f8d3e7 Fix unit reduction elimination bugs 
* Handle 'chains' of unit reductions starting in a single state
* Avoid eliminating rules which will later receive aliases
 2018-03-12 07:54:18 -07:00
Max Brunsfeld
df2430b94c Remove a C error recovery test temporarily 2018-03-08 14:18:12 -08:00
Max Brunsfeld
0810971f3e 🔥 symbol iterator API 
This idea was never fully baked.
 2018-03-08 14:16:37 -08:00
Max Brunsfeld
e927d02f43 Allow reusing leaf nodes unless the next leaf has changes 2018-03-07 17:44:54 -08:00
Max Brunsfeld
52087de4f0 Remove the concept of fragile reductions 
They were a vestige of when Tree-sitter did sentential form-based
incremental parsing (as opposed to simply state matching). This was
elegant but not compatible with GLR as far as I could tell.
 2018-03-02 14:51:54 -08:00
Max Brunsfeld
07fa3eb386 Fix capture of corpus descriptions in integration tests 2018-03-02 11:27:41 -08:00
Max Brunsfeld
a8d539023d Handle subdirectories existing in parsers' examples folders 2018-03-02 11:04:08 -08:00
Phil Turnbull
bc192d95ca Build fuzzer in 'halt' and 'recover' modes 
Build each language fuzzer in two modes (halt_on_error=true and
halt_on_error=false) and use different timeouts for each fuzzer.

Also merge the run-fuzzer and reproduce scripts so they use identical
values of ASAN_OPTIONS/UBSAN_OPTIONS/etc0
 2018-03-02 10:13:13 -08:00
Max Brunsfeld
d3ac345644 When parsing corpus, anchor header pattern to line start 2018-03-02 09:46:33 -08:00
Max Brunsfeld
82c7e170b3 Fix case where loop was created in the parse stack 
Fixes #133
 2018-03-02 09:05:20 -08:00
Max Brunsfeld
2daae48fe0 Handle conflicts in repeat rules after external tokens 
Signed-off-by: Rick Winfrey <rewinfrey@github.com>
 2018-02-14 11:24:51 -08:00
Max Brunsfeld
facafcd6e4 Pass row/column position to input seek method 2018-02-14 07:31:49 -08:00
Max Brunsfeld
299a146b66 Balance repetition trees after parsing 2018-02-12 11:41:56 -08:00
Max Brunsfeld
8c29841adf Represent repetitions with associative structure 2018-02-12 11:41:56 -08:00
Max Brunsfeld
46dcd53090 Do not insert missing tokens if halt_on_error option is passed 2018-01-24 14:04:55 -08:00
Max Brunsfeld
b520bdd2d5
Merge pull request #126 from tree-sitter/mb-fix-epsilon-rule-loophole 
Don't allow an epsilon start rule if it is used in other rules
 2018-01-23 17:19:55 -08:00
Max Brunsfeld
2e4f76c164 Don't allow an epsilon start rule if it is used in other rules 2018-01-23 17:05:28 -08:00
Max Brunsfeld
315dff3285 Add an API for getting a node's child index 2018-01-09 14:01:36 -08:00
Max Brunsfeld
f653f2b3bb Add ts_node_first_{child,named_child}_for_byte methods 2018-01-09 13:44:59 -08:00
Max Brunsfeld
13adfe4927 Update error corpus 2017-12-29 16:21:04 -08:00
Max Brunsfeld
6304a3bcd1 Make it easier to run tests with debug graphs 2017-12-28 12:41:23 -08:00
Max Brunsfeld
addeb6c4c1 Allocate and free trees using an object pool 2017-12-27 10:34:29 -08:00
Max Brunsfeld
0e69da37a5 Return a character count from the lexer's get_column method 2017-12-20 16:26:38 -08:00
Max Brunsfeld
e5851fd9b9 Don't use non-existent \a syntax in test grammars 2017-12-13 12:21:28 -08:00
Max Brunsfeld
f426b61e7c Fix expectation around preproc directive in C error test 2017-12-13 12:21:13 -08:00
Max Brunsfeld
fbcefe25f7 Avoid creating external tokens that start after they end 2017-12-07 11:50:27 -08:00
Max Brunsfeld
90629bd45a Add some assertions to the fixture grammar tests 2017-12-07 11:50:27 -08:00
Max Brunsfeld
493db39363 Never move the start rule of a grammar into the lexical grammar 
This preserves a useful invariant that the root node of the AST is never
a token.
 2017-12-07 11:50:27 -08:00
Max Brunsfeld
36c2b685b9 Always invalidate old chunk of text when parsing after an edit 2017-10-04 15:09:46 -07:00
Max Brunsfeld
c0073c5b72 Update error corpus to reflect C grammar changes 2017-10-04 15:06:12 -07:00
Max Brunsfeld
d342b61ede Re-enable JS fuzzing example test 2017-09-14 11:39:08 -07:00
Max Brunsfeld
9d67a98510 Merge pull request #103 from tree-sitter/python-assertion-failure 
Assertion failure in parser__advance
 2017-09-14 11:38:22 -07:00
Max Brunsfeld
d291af9a31 Refactor error comparisons 
* Deal with mergeability outside of error comparison function
* Make `better_version_exists` function pure (don't halt other versions
as a side effect).
* Tweak error comparison logic

Signed-off-by: Rick Winfrey <rewinfrey@github.com>
 2017-09-13 16:38:15 -07:00