Max Brunsfeld
64fd5d0b4b
Merge pull request #153 from tree-sitter/better-lexical-conflict-detection
...
Make lexical conflict detection more conservative and reliable
2018-03-28 11:05:07 -07:00
Max Brunsfeld
fb348c0f1e
Fix signed/unsigned comparison warning
2018-03-28 11:04:49 -07:00
Max Brunsfeld
e917756ad1
Remove depends_on_lookahead field from parse table entries
...
This simplifies the logic for determining whether a token is reusable
and makes it more conservative. It should fix some incremental parsing
bugs that are being caught by the randomized tests on CI.
2018-03-28 10:58:33 -07:00
Max Brunsfeld
186f70649c
Consolidate the unify for detecting conflicting tokens
2018-03-28 10:03:09 -07:00
Max Brunsfeld
a8bc67ac42
Allow LookaheadSet::for_each to terminate early
2018-03-28 10:03:09 -07:00
Max Brunsfeld
43e14332ed
Avoid creating duplicate metadata rules
2018-03-28 10:03:09 -07:00
Max Brunsfeld
10c3a95679
Merge pull request #149 from yudai-nkt/suggest-devdependencies
...
Suggest the use of devDependencies in a parser
2018-03-19 09:37:58 -07:00
Yudai NAKATA
b9a1993bc5
Suggest the use of devDependencies in a parser
...
Fix #146 . [ci skip]
2018-03-17 17:11:40 +09:00
Max Brunsfeld
997913aa82
Merge pull request #148 from tree-sitter/blank-external-tokens
...
Be less conservative about merging parse states with external tokens
2018-03-16 16:06:55 -07:00
Max Brunsfeld
b7d0606fbd
Be less conservative in merging parse states with external tokens
...
Also, clean up the internal representation of external tokens
2018-03-16 16:00:40 -07:00
Max Brunsfeld
4d9d72747b
Merge pull request #145 from tree-sitter/corpus-tests
...
small unit test protip
2018-03-16 14:55:28 -07:00
Ayman Nadeem
94815a703b
small unit test protip
2018-03-14 15:07:09 -07:00
Max Brunsfeld
fe29173d5f
Merge pull request #142 from tree-sitter/fuzz-halt-recover
...
Add 'halt' and 'recover' modes to fuzzer
2018-03-14 09:28:58 -07:00
Phil Turnbull
269b1a0864
Update repo for libFuzzer
...
libFuzzer has now been broken out from LLVM and can be built separately
2018-03-12 13:08:58 -07:00
Phil Turnbull
547a71899c
Allow extra libFuzzer arguments to script/reproduce
2018-03-12 12:54:06 -07:00
Max Brunsfeld
7183f8d3e7
Fix unit reduction elimination bugs
...
* Handle 'chains' of unit reductions starting in a single state
* Avoid eliminating rules which will later receive aliases
2018-03-12 07:54:18 -07:00
Max Brunsfeld
72849787b1
Fix logic for identifying keyword capture token
2018-03-12 07:52:57 -07:00
Max Brunsfeld
84b15d2c78
Merge pull request #140 from tree-sitter/eliminate-unit-reductions
...
Eliminate non-user-visible unit reductions from parse tables
2018-03-08 14:31:12 -08:00
Max Brunsfeld
df2430b94c
Remove a C error recovery test temporarily
2018-03-08 14:18:12 -08:00
Max Brunsfeld
0810971f3e
🔥 symbol iterator API
...
This idea was never fully baked.
2018-03-08 14:16:37 -08:00
Max Brunsfeld
128edbebd6
Eliminate non-user-visible unit reductions from parse tables
2018-03-08 12:53:32 -08:00
Max Brunsfeld
f932d592d6
Merge pull request #137 from tree-sitter/keyword-lexing
...
Create a separate lexer function for keyword tokens
2018-03-07 17:55:25 -08:00
Max Brunsfeld
e927d02f43
Allow reusing leaf nodes unless the next leaf has changes
2018-03-07 17:44:54 -08:00
Max Brunsfeld
53cd89c614
Ensure keyword capture tokens aren't too loosely defined
2018-03-07 14:46:11 -08:00
Max Brunsfeld
c0cc35ff07
Create separate lexer function for keywords
2018-03-07 12:00:26 -08:00
Max Brunsfeld
16cdd2ffbe
Bump language ABI version after removing fragile bit from actions
2018-03-05 17:13:11 -08:00
Max Brunsfeld
f96969738b
Don't remove mergeable stack versions so aggressively during condense
2018-03-05 10:40:05 -08:00
Max Brunsfeld
daf2c01d8b
Add docs section about hidden rules
2018-03-05 09:35:52 -08:00
Max Brunsfeld
60eacbc7ff
Delete .dot file after successfully generating SVGs
2018-03-05 09:35:12 -08:00
Max Brunsfeld
dbc0c208f4
Add missing initialization of parser's in_ambiguity state
2018-03-02 15:25:39 -08:00
Max Brunsfeld
52087de4f0
Remove the concept of fragile reductions
...
They were a vestige of when Tree-sitter did sentential form-based
incremental parsing (as opposed to simply state matching). This was
elegant but not compatible with GLR as far as I could tell.
2018-03-02 14:51:54 -08:00
Max Brunsfeld
07fa3eb386
Fix capture of corpus descriptions in integration tests
2018-03-02 11:27:41 -08:00
Max Brunsfeld
fd77bf49a3
Add explanation of alias function in parsers doc
2018-03-02 11:09:53 -08:00
Max Brunsfeld
a8d539023d
Handle subdirectories existing in parsers' examples folders
2018-03-02 11:04:08 -08:00
Phil Turnbull
bc192d95ca
Build fuzzer in 'halt' and 'recover' modes
...
Build each language fuzzer in two modes (halt_on_error=true and
halt_on_error=false) and use different timeouts for each fuzzer.
Also merge the run-fuzzer and reproduce scripts so they use identical
values of ASAN_OPTIONS/UBSAN_OPTIONS/etc0
2018-03-02 10:13:13 -08:00
Max Brunsfeld
d3ac345644
When parsing corpus, anchor header pattern to line start
2018-03-02 09:46:33 -08:00
Max Brunsfeld
82c7e170b3
Fix case where loop was created in the parse stack
...
Fixes #133
2018-03-02 09:05:20 -08:00
Max Brunsfeld
32ef3e001a
Account for epsilon external tokens when merging parse states
...
Do not merge a token T into a parse state S if S contains
external tokens that can be *followed* by tokens that could
be shadowed by T.
At this point, the only automated test for this logic is via
the bash grammar, in which the `]` token should not be merged
into states in which `_concat` is valid, because `_concat`
can be followed by a `_special_characters` token, and `]`
would shadow `_special_characters`.
2018-02-28 14:47:04 -08:00
Max Brunsfeld
dbecd9521f
Add table of contents to docs pages
2018-02-26 17:42:42 -08:00
Max Brunsfeld
b357ecd9b8
Describe DSL, precedence in parsers doc
...
[ci skip]
2018-02-26 13:55:13 -08:00
Max Brunsfeld
70cd827f3c
Talk about tree-sitter generate command in parsers doc
...
[ci skip]
2018-02-26 11:37:22 -08:00
Max Brunsfeld
a8d198f77c
Expand the creating-parsers document
...
[ci skip]
2018-02-26 11:19:45 -08:00
Max Brunsfeld
a7d49b4d98
Fix wording in creating-parsers doc
2018-02-26 00:43:34 -08:00
Max Brunsfeld
10a3cbd814
Move grammar schema to src folder
...
Now that there's a docs folder that contains actual docs.
2018-02-26 00:40:20 -08:00
Max Brunsfeld
e3e7c8ed9d
Start work on grammar writing doc
...
[ci skip]
2018-02-26 00:38:54 -08:00
Max Brunsfeld
a2aa64ec97
Start work on github pages site
...
[ci skip]
2018-02-24 21:39:33 -08:00
Max Brunsfeld
da20ddf70e
Tweak README
2018-02-19 14:16:11 -08:00
Max Brunsfeld
16a45d4aa4
Fix hole in logic for terminating tree balancing
...
It's important that the repetition nodes have a child count of 2,
because we assign to their second child. We could maybe generalize
this to allow balancing in the presence of 'extra' nodes like comments
and errors, but this might be complicated.
2018-02-16 12:44:30 -08:00
Max Brunsfeld
2daae48fe0
Handle conflicts in repeat rules after external tokens
...
Signed-off-by: Rick Winfrey <rewinfrey@github.com>
2018-02-14 11:24:51 -08:00
Max Brunsfeld
facafcd6e4
Pass row/column position to input seek method
2018-02-14 07:31:49 -08:00
