traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1532 commits 16 branches 98 tags 37 MiB
Commit graph

1532 commits

This branch
This branch
All branches
Author SHA1 Message Date
Max Brunsfeld
2b73a30fba Build benchmarks in release mode 2017-07-06 11:49:32 -07:00
Max Brunsfeld
a64db98218 Rename lib.sh -> scan-build.sh 2017-07-06 10:32:41 -07:00
Max Brunsfeld
78333b70c0 Build benchmarks with scan-build on CI 2017-07-06 10:22:14 -07:00
Max Brunsfeld
8c005cddc6 Print average and worst speeds in benchmark command 2017-07-06 10:12:22 -07:00
Max Brunsfeld
8f028ebf68 Avoid deep tree comparison when both trees have errors 2017-07-05 17:33:35 -07:00
Max Brunsfeld
c53f9bcbd9 Build benchmarks in Test mode for now 2017-07-05 17:27:50 -07:00
Max Brunsfeld
782bf48772 Don't do skip_preceding_subtrees recovery when there are lots of versions 2017-07-05 15:34:19 -07:00
Max Brunsfeld
17bc3dfaf7 Add a benchmark command 
This command measures the speed of parsing each grammar's examples.
It also uses each grammar to parse all of the *other* grammars' examples
in order to measure error recovery performance with fairly large files.
 2017-07-05 14:14:38 -07:00
Max Brunsfeld
298228d8de Clean up test_grammars file 2017-07-05 11:39:33 -07:00
Max Brunsfeld
69500c9dd7 Merge pull request #85 from tree-sitter/fuzzing-examples 
Fix more performance problems found by fuzzing
 2017-07-05 09:48:56 -07:00
Max Brunsfeld
d322f0b6a7 🎨 2017-07-04 21:59:54 -07:00
Max Brunsfeld
e7ccd9c17c Put back check for better existing versions during recover action 
When checking for better existing versions, only kill the current
version if there is a better version earlier in the rotation.
 2017-07-03 12:27:23 -07:00
Max Brunsfeld
f93f78ef2d Remove version-pruning criteria based on pushed node count 2017-07-02 23:42:23 -07:00
Max Brunsfeld
fcffd4b732 Add test for an example found during fuzzing 2017-06-30 21:55:50 -07:00
Max Brunsfeld
f722923493 Limit the search depth for skip_preceding_trees recovery 2017-06-30 17:49:09 -07:00
Max Brunsfeld
89e5037f01 Manually tail-call-optimize stack_node_release function 2017-06-30 17:49:09 -07:00
Max Brunsfeld
eccb3893eb Prune unneeded stack versions based on a depth criteria 2017-06-30 17:49:09 -07:00
Max Brunsfeld
d6579956f5 Enforce a hard version count limit during the recovery action 2017-06-30 17:49:09 -07:00
Max Brunsfeld
3e84b15204 Merge pull request #84 from tree-sitter/faster-error-recovery 
Avoid hangs during error recovery
 2017-06-29 16:53:52 -07:00
Max Brunsfeld
061fba6b92 🎨 Just call it 'inline' 2017-06-29 16:49:59 -07:00
Max Brunsfeld
a89322c5f1 Remove unneeded parameters from public interface of stack_iterate callback 2017-06-29 16:43:56 -07:00
Max Brunsfeld
009d6d1534 Improve heuristics for pruning parse versions based on errors 
* Rewrite the error cost comparison in terms of explicit, discrete
conditions.
* Allow merging versions have different error costs.
* Store the depth of each stack version since the last error. Use this
state to prevent incorrect merging.
* Sort the stack versions in order of preference and put a hard limit on
the version count.
 2017-06-29 15:00:20 -07:00
Max Brunsfeld
445be0736a Clean up ts_stack_push function 2017-06-29 15:00:20 -07:00
Max Brunsfeld
66be393b78 Stack - consider empty external token state identical to NULL 2017-06-29 15:00:20 -07:00
Max Brunsfeld
5e3818882b Merge pull request #83 from tree-sitter/ref-counts 
Increase size of ref_counts and add assertions
 2017-06-29 10:15:51 -07:00
Phil Turnbull
5ef9c4d6aa Increase size of ref_counts and add assertions 
This bumps the size of the reference counts from 16- to 32-bit counters to make
it less likely to overflow. Also assert in the retain function that the
reference count didn't overflow.

32-bits seems big enough for non-pathological examples but a more fool-proof
fix may be to bump it to 64-bits.
 2017-06-29 06:39:01 -07:00
Max Brunsfeld
839dc53bf2 Merge pull request #82 from tree-sitter/address-sanitizer 
Use address sanitizer when running tests on mac, fix newly discovered bugs
 2017-06-28 11:04:15 -07:00
Max Brunsfeld
dfd7b1f5f6 Consolidate memory management logic in Stack 2017-06-27 16:17:24 -07:00
Max Brunsfeld
ca4db76bb1 Override LINK to call clang++ w/ -fsanitize=address in test script 
Signed-off-by: Philip Turnbull <philipturnbull@github.com>
 2017-06-27 15:44:58 -07:00
Max Brunsfeld
0143bfdad4 Avoid use-after-free of external token states 
Previously, it was possible for references to external token states to
outlive the trees to which those states belonged.

Now, instead of storing references to external token states in the Stack
and in the Lexer, we store references to the external token trees
themselves, and we retain the trees to prevent use-after-free.
 2017-06-27 14:54:27 -07:00
Max Brunsfeld
f678018d3d Avoid use-after-free when copying stack iterators 2017-06-27 14:54:27 -07:00
Max Brunsfeld
0054272879 Record deallocations even when recording allocations is disabled 2017-06-27 14:54:27 -07:00
Max Brunsfeld
76e35fd037 Enable address sanitizer when running tests on mac 
Signed-off-by: Philip Turnbull <philipturnbull@github.com>
 2017-06-27 14:54:20 -07:00
Max Brunsfeld
076002a01e Merge pull request #78 from philipturnbull/update-utf8proc 
Out of bounds read in utf8proc
 2017-06-23 12:18:21 -07:00
Max Brunsfeld
f62ee5a0f3 Fix OOB reads at ends of chunks 
Signed-off-by: Philip Turnbull <philipturnbull@github.com>
 2017-06-23 12:09:16 -07:00
Max Brunsfeld
8ee3f96960 Fix formatting of non-ascii unexpected characters 
Signed-off-by: Philip Turnbull <philipturnbull@github.com>
 2017-06-23 12:08:50 -07:00
Max Brunsfeld
d6cd0ca4d0 Merge pull request #80 from tree-sitter/fix-another-parse-state-merging-problem 
Fix another parse state merging problem
 2017-06-22 16:12:30 -07:00
Max Brunsfeld
20982fdcb9 Mark tokens as non-reusable in states where shorter takes take precedence 
This fixes some randomized test failures in the C grammar, relating to Object-like macros.
The object-like macro rule relies on a whitespace token in order to distinguish object-like
macros whose values begin with a '(' from function-like macros. The presence of that
whitespace token means that other nodes should not be reusable in that state.
 2017-06-22 16:04:42 -07:00
Max Brunsfeld
8517313a45 🎨 2017-06-22 15:33:07 -07:00
Max Brunsfeld
8157b81b68 Improve logic for short-circuiting trivial lexing conflict detection 2017-06-22 15:33:01 -07:00
Max Brunsfeld
2c043803f1 Be more conservative about avoiding lexing conflicts when merging states 
This fixes a bug in the C++ grammar where the `>>` token was merged into
a state where it was previously not valid, but the `>` token *was*
valid. This caused nested templates like -

std::vector<std::pair<int, int>>

to not parse correctly.
 2017-06-22 15:32:13 -07:00
Max Brunsfeld
6db12ab44e Adjust C error recovery tests to reflect grammar changes 2017-06-21 16:58:40 -07:00
Max Brunsfeld
0cc1090001 Read grammar tests from corpus dir instead of grammar_test dir 2017-06-21 16:31:25 -07:00
Phil Turnbull
8a15da90fb Update utf8proc dependency to v2.1 
This includes JuliaLang/utf8proc#66 which is an out-of-bounds read when parsing
malformed utf8 characters.
 2017-06-21 09:55:13 -04:00
Max Brunsfeld
513edec7c1 Merge pull request #77 from philipturnbull/scan-build-fixes 
Fix errors found by scan-build
 2017-06-20 10:15:20 -07:00
Phil Turnbull
7bdb0917d3 Revert "Add standalone scan-build script" 
This reverts commit 9135d14b81.
 2017-06-19 10:24:40 -04:00
Phil Turnbull
1b9e78add2 Fix formatting and use '-n' 2017-06-19 10:24:08 -04:00
Phil Turnbull
e9bf794cd9 Remove unneeded build step 2017-06-19 10:23:54 -04:00
Phil Turnbull
ee3caafe7b Use -j2 on the CI boxes 
The travis-ci trusty container has two cores.
 2017-06-16 15:16:40 -04:00
Phil Turnbull
9135d14b81 Add standalone scan-build script 
For running scan-build outside of CI, e.g. `./script/scan-build -j4`
 2017-06-16 15:16:40 -04:00