Max Brunsfeld
e4f57d6fee
Test more cases in fixture grammar with inline rules
2017-07-12 10:12:42 -07:00
Max Brunsfeld
5c8f7c035e
Add stream operator for ParseItemSet
2017-07-12 09:42:56 -07:00
Max Brunsfeld
65bf1389e1
Add a way to automatically inline rules
2017-07-11 23:13:44 -07:00
Max Brunsfeld
26a25278cd
When comparing parse items, ignore consumed part of their productions
...
This speeds up parser generation by increasing the likelihood that we'll recognize
parse item sets as equivalent in advance, rather than having to merge their states
after the fact.
2017-07-11 17:30:32 -07:00
Max Brunsfeld
a199b217f3
Optimize ParseTableBuilder for non-terminals w/ many productions
2017-07-11 12:54:29 -07:00
Max Brunsfeld
68c3ba1b8b
🎨 merge_parse_state
2017-07-10 16:46:11 -07:00
Max Brunsfeld
43d347c225
Merge pull request #87 from tree-sitter/dynamic-precedence
...
Introduce rule for resolving runtime ambiguities
2017-07-10 16:43:58 -07:00
Max Brunsfeld
5bd5b4bb05
Replace <cctype> -> <cwctype>
2017-07-10 14:35:14 -07:00
Max Brunsfeld
bf4b8bf55b
Use my fork of crypto-algorithms
2017-07-10 14:29:14 -07:00
Max Brunsfeld
59236d2ed1
Avoid redundant character comparisons in generated lex function
2017-07-10 14:09:31 -07:00
Max Brunsfeld
2755b07222
Don't store unfinished item signature on ParseStates
2017-07-10 10:47:38 -07:00
Max Brunsfeld
1586d70cbe
Compute conflicting tokens more precisely
...
While generating the parse table, keep track of which tokens can follow one another.
Then use this information to evaluate token conflicts more precisely. This will
result in a smaller parse table than the previous, overly-conservative approach.
2017-07-07 17:54:24 -07:00
Max Brunsfeld
a98abde529
Provide all preceding symbols as context when reporting conflicts
2017-07-07 14:52:56 -07:00
Max Brunsfeld
c91ceaaa8d
🎨 build_parse_table
2017-07-07 14:52:45 -07:00
Max Brunsfeld
0de93b3bf2
Allow negative dynamic precedences
2017-07-06 22:21:59 -07:00
Max Brunsfeld
107feb7960
Bump the language version number after adding dynamic precedences
2017-07-06 15:58:29 -07:00
Max Brunsfeld
08bb365f6c
Allow PREC_DYNAMIC in JSON schema
2017-07-06 15:51:03 -07:00
Max Brunsfeld
d8e9d04fe7
Add PREC_DYNAMIC rule for resolving runtime ambiguities
2017-07-06 15:24:45 -07:00
Max Brunsfeld
cb652239f6
Add missing semicolons in flatten_grammar test
2017-07-06 12:48:50 -07:00
Max Brunsfeld
31f827945a
Merge pull request #86 from tree-sitter/benchmarks
...
Add a benchmark command
2017-07-06 12:38:00 -07:00
Max Brunsfeld
21bc50377e
Run make with the right target when building benchmarks on CI
2017-07-06 12:36:57 -07:00
Max Brunsfeld
96068bbacb
Represent all speeds as size_t in benchmarks
2017-07-06 12:24:41 -07:00
Max Brunsfeld
2b73a30fba
Build benchmarks in release mode
2017-07-06 11:49:32 -07:00
Max Brunsfeld
a64db98218
Rename lib.sh -> scan-build.sh
2017-07-06 10:32:41 -07:00
Max Brunsfeld
78333b70c0
Build benchmarks with scan-build on CI
2017-07-06 10:22:14 -07:00
Max Brunsfeld
8c005cddc6
Print average and worst speeds in benchmark command
2017-07-06 10:12:22 -07:00
Max Brunsfeld
8f028ebf68
Avoid deep tree comparison when both trees have errors
2017-07-05 17:33:35 -07:00
Max Brunsfeld
c53f9bcbd9
Build benchmarks in Test mode for now
2017-07-05 17:27:50 -07:00
Max Brunsfeld
782bf48772
Don't do skip_preceding_subtrees recovery when there are lots of versions
2017-07-05 15:34:19 -07:00
Max Brunsfeld
17bc3dfaf7
Add a benchmark command
...
This command measures the speed of parsing each grammar's examples.
It also uses each grammar to parse all of the *other* grammars' examples
in order to measure error recovery performance with fairly large files.
2017-07-05 14:14:38 -07:00
Max Brunsfeld
298228d8de
Clean up test_grammars file
2017-07-05 11:39:33 -07:00
Max Brunsfeld
69500c9dd7
Merge pull request #85 from tree-sitter/fuzzing-examples
...
Fix more performance problems found by fuzzing
2017-07-05 09:48:56 -07:00
Max Brunsfeld
d322f0b6a7
🎨
2017-07-04 21:59:54 -07:00
Max Brunsfeld
e7ccd9c17c
Put back check for better existing versions during recover action
...
When checking for better existing versions, only kill the current
version if there is a better version earlier in the rotation.
2017-07-03 12:27:23 -07:00
Max Brunsfeld
f93f78ef2d
Remove version-pruning criteria based on pushed node count
2017-07-02 23:42:23 -07:00
Max Brunsfeld
fcffd4b732
Add test for an example found during fuzzing
2017-06-30 21:55:50 -07:00
Max Brunsfeld
f722923493
Limit the search depth for skip_preceding_trees recovery
2017-06-30 17:49:09 -07:00
Max Brunsfeld
89e5037f01
Manually tail-call-optimize stack_node_release function
2017-06-30 17:49:09 -07:00
Max Brunsfeld
eccb3893eb
Prune unneeded stack versions based on a depth criteria
2017-06-30 17:49:09 -07:00
Max Brunsfeld
d6579956f5
Enforce a hard version count limit during the recovery action
2017-06-30 17:49:09 -07:00
Max Brunsfeld
3e84b15204
Merge pull request #84 from tree-sitter/faster-error-recovery
...
Avoid hangs during error recovery
2017-06-29 16:53:52 -07:00
Max Brunsfeld
061fba6b92
🎨 Just call it 'inline'
2017-06-29 16:49:59 -07:00
Max Brunsfeld
a89322c5f1
Remove unneeded parameters from public interface of stack_iterate callback
2017-06-29 16:43:56 -07:00
Max Brunsfeld
009d6d1534
Improve heuristics for pruning parse versions based on errors
...
* Rewrite the error cost comparison in terms of explicit, discrete
conditions.
* Allow merging versions have different error costs.
* Store the depth of each stack version since the last error. Use this
state to prevent incorrect merging.
* Sort the stack versions in order of preference and put a hard limit on
the version count.
2017-06-29 15:00:20 -07:00
Max Brunsfeld
445be0736a
Clean up ts_stack_push function
2017-06-29 15:00:20 -07:00
Max Brunsfeld
66be393b78
Stack - consider empty external token state identical to NULL
2017-06-29 15:00:20 -07:00
Max Brunsfeld
5e3818882b
Merge pull request #83 from tree-sitter/ref-counts
...
Increase size of ref_counts and add assertions
2017-06-29 10:15:51 -07:00
Phil Turnbull
5ef9c4d6aa
Increase size of ref_counts and add assertions
...
This bumps the size of the reference counts from 16- to 32-bit counters to make
it less likely to overflow. Also assert in the retain function that the
reference count didn't overflow.
32-bits seems big enough for non-pathological examples but a more fool-proof
fix may be to bump it to 64-bits.
2017-06-29 06:39:01 -07:00
Max Brunsfeld
839dc53bf2
Merge pull request #82 from tree-sitter/address-sanitizer
...
Use address sanitizer when running tests on mac, fix newly discovered bugs
2017-06-28 11:04:15 -07:00
Max Brunsfeld
dfd7b1f5f6
Consolidate memory management logic in Stack
2017-06-27 16:17:24 -07:00
