traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1815 commits 16 branches 98 tags 37 MiB
Commit graph

1815 commits

This branch
This branch
All branches
Author SHA1 Message Date
Axel Hecht
fb5c79aa32 make default extras explicit in docs 2018-04-04 10:59:13 +02:00
Max Brunsfeld
d2019cd12c Ignore log file 2018-04-02 09:15:11 -07:00
Max Brunsfeld
89d0d1dd0e
Merge pull request #154 from 314eter/invalidtokencontents 
Add error message for TSCompileErrorTypeInvalidTokenContents
 2018-03-30 12:32:33 -07:00
Pieter Goetschalckx
3d0ca31cf1 Add error message for TSCompileErrorTypeInvalidTokenContents 2018-03-30 21:12:09 +02:00
Max Brunsfeld
dbe77e7199 Simplify testing-only ts_stack_iterate function 2018-03-29 17:50:07 -07:00
Max Brunsfeld
5520983144 Clean up Stack API 
* Remove StackPopResult
* Rename top_state() -> state()
* Rename top_position() -> position()
* Improve docs
 2018-03-29 17:37:54 -07:00
Max Brunsfeld
ee995c3d6b Avoid redundant retains/releases by giving ts_stack_push move semantics 2018-03-29 17:18:43 -07:00
Max Brunsfeld
64fd5d0b4b
Merge pull request #153 from tree-sitter/better-lexical-conflict-detection 
Make lexical conflict detection more conservative and reliable
 2018-03-28 11:05:07 -07:00
Max Brunsfeld
fb348c0f1e Fix signed/unsigned comparison warning 2018-03-28 11:04:49 -07:00
Max Brunsfeld
e917756ad1 Remove depends_on_lookahead field from parse table entries 
This simplifies the logic for determining whether a token is reusable
and makes it more conservative. It should fix some incremental parsing
bugs that are being caught by the randomized tests on CI.
 2018-03-28 10:58:33 -07:00
Max Brunsfeld
186f70649c Consolidate the unify for detecting conflicting tokens 2018-03-28 10:03:09 -07:00
Max Brunsfeld
a8bc67ac42 Allow LookaheadSet::for_each to terminate early 2018-03-28 10:03:09 -07:00
Max Brunsfeld
43e14332ed Avoid creating duplicate metadata rules 2018-03-28 10:03:09 -07:00
Max Brunsfeld
10c3a95679
Merge pull request #149 from yudai-nkt/suggest-devdependencies 
Suggest the use of devDependencies in a parser
 2018-03-19 09:37:58 -07:00
Yudai NAKATA
b9a1993bc5 Suggest the use of devDependencies in a parser 
Fix #146. [ci skip]
 2018-03-17 17:11:40 +09:00
Max Brunsfeld
997913aa82
Merge pull request #148 from tree-sitter/blank-external-tokens 
Be less conservative about merging parse states with external tokens
 2018-03-16 16:06:55 -07:00
Max Brunsfeld
b7d0606fbd Be less conservative in merging parse states with external tokens 
Also, clean up the internal representation of external tokens
 2018-03-16 16:00:40 -07:00
Max Brunsfeld
4d9d72747b
Merge pull request #145 from tree-sitter/corpus-tests 
small unit test protip
 2018-03-16 14:55:28 -07:00
Ayman Nadeem
94815a703b
small unit test protip 2018-03-14 15:07:09 -07:00
Max Brunsfeld
fe29173d5f
Merge pull request #142 from tree-sitter/fuzz-halt-recover 
Add 'halt' and 'recover' modes to fuzzer
 2018-03-14 09:28:58 -07:00
Phil Turnbull
269b1a0864 Update repo for libFuzzer 
libFuzzer has now been broken out from LLVM and can be built separately
 2018-03-12 13:08:58 -07:00
Phil Turnbull
547a71899c Allow extra libFuzzer arguments to script/reproduce 2018-03-12 12:54:06 -07:00
Max Brunsfeld
7183f8d3e7 Fix unit reduction elimination bugs 
* Handle 'chains' of unit reductions starting in a single state
* Avoid eliminating rules which will later receive aliases
 2018-03-12 07:54:18 -07:00
Max Brunsfeld
72849787b1 Fix logic for identifying keyword capture token 2018-03-12 07:52:57 -07:00
Max Brunsfeld
84b15d2c78
Merge pull request #140 from tree-sitter/eliminate-unit-reductions 
Eliminate non-user-visible unit reductions from parse tables
 2018-03-08 14:31:12 -08:00
Max Brunsfeld
df2430b94c Remove a C error recovery test temporarily 2018-03-08 14:18:12 -08:00
Max Brunsfeld
0810971f3e 🔥 symbol iterator API 
This idea was never fully baked.
 2018-03-08 14:16:37 -08:00
Max Brunsfeld
128edbebd6 Eliminate non-user-visible unit reductions from parse tables 2018-03-08 12:53:32 -08:00
Max Brunsfeld
f932d592d6
Merge pull request #137 from tree-sitter/keyword-lexing 
Create a separate lexer function for keyword tokens
 2018-03-07 17:55:25 -08:00
Max Brunsfeld
e927d02f43 Allow reusing leaf nodes unless the next leaf has changes 2018-03-07 17:44:54 -08:00
Max Brunsfeld
53cd89c614 Ensure keyword capture tokens aren't too loosely defined 2018-03-07 14:46:11 -08:00
Max Brunsfeld
c0cc35ff07 Create separate lexer function for keywords 2018-03-07 12:00:26 -08:00
Max Brunsfeld
16cdd2ffbe Bump language ABI version after removing fragile bit from actions 2018-03-05 17:13:11 -08:00
Max Brunsfeld
f96969738b Don't remove mergeable stack versions so aggressively during condense 2018-03-05 10:40:05 -08:00
Max Brunsfeld
daf2c01d8b Add docs section about hidden rules 2018-03-05 09:35:52 -08:00
Max Brunsfeld
60eacbc7ff Delete .dot file after successfully generating SVGs 2018-03-05 09:35:12 -08:00
Max Brunsfeld
dbc0c208f4 Add missing initialization of parser's in_ambiguity state 2018-03-02 15:25:39 -08:00
Max Brunsfeld
52087de4f0 Remove the concept of fragile reductions 
They were a vestige of when Tree-sitter did sentential form-based
incremental parsing (as opposed to simply state matching). This was
elegant but not compatible with GLR as far as I could tell.
 2018-03-02 14:51:54 -08:00
Max Brunsfeld
07fa3eb386 Fix capture of corpus descriptions in integration tests 2018-03-02 11:27:41 -08:00
Max Brunsfeld
fd77bf49a3 Add explanation of alias function in parsers doc 2018-03-02 11:09:53 -08:00
Max Brunsfeld
a8d539023d Handle subdirectories existing in parsers' examples folders 2018-03-02 11:04:08 -08:00
Phil Turnbull
bc192d95ca Build fuzzer in 'halt' and 'recover' modes 
Build each language fuzzer in two modes (halt_on_error=true and
halt_on_error=false) and use different timeouts for each fuzzer.

Also merge the run-fuzzer and reproduce scripts so they use identical
values of ASAN_OPTIONS/UBSAN_OPTIONS/etc0
 2018-03-02 10:13:13 -08:00
Max Brunsfeld
d3ac345644 When parsing corpus, anchor header pattern to line start 2018-03-02 09:46:33 -08:00
Max Brunsfeld
82c7e170b3 Fix case where loop was created in the parse stack 
Fixes #133
 2018-03-02 09:05:20 -08:00
Max Brunsfeld
32ef3e001a Account for epsilon external tokens when merging parse states 
Do not merge a token T into a parse state S if S contains
external tokens that can be *followed* by tokens that could
be shadowed by T.

At this point, the only automated test for this logic is via
the bash grammar, in which the `]` token should not be merged
into states in which `_concat` is valid, because `_concat`
can be followed by a `_special_characters` token, and `]`
would shadow `_special_characters`.
 2018-02-28 14:47:04 -08:00
Max Brunsfeld
dbecd9521f Add table of contents to docs pages 2018-02-26 17:42:42 -08:00
Max Brunsfeld
b357ecd9b8 Describe DSL, precedence in parsers doc 
[ci skip]
 2018-02-26 13:55:13 -08:00
Max Brunsfeld
70cd827f3c Talk about tree-sitter generate command in parsers doc 
[ci skip]
 2018-02-26 11:37:22 -08:00
Max Brunsfeld
a8d198f77c Expand the creating-parsers document 
[ci skip]
 2018-02-26 11:19:45 -08:00
Max Brunsfeld
a7d49b4d98 Fix wording in creating-parsers doc 2018-02-26 00:43:34 -08:00