traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1853 commits 16 branches 98 tags 37 MiB
Commit graph

736 commits

Author SHA1 Message Date
Max Brunsfeld
ed11ddbd38 Explicitly compare regions between subtrees in get_changed_ranges 2017-08-06 17:16:29 -07:00
Max Brunsfeld
f5aa633011 Reorder iterator methods in get_changed_ranges.c 2017-08-05 20:33:38 -07:00
Max Brunsfeld
94dc703bfc Require that grammars' start rules be visible 2017-08-04 17:07:37 -07:00
Max Brunsfeld
1dca3a0b58 Simplify parse version reordering 2017-08-04 14:51:14 -07:00
Max Brunsfeld
85be0e7e75 Ensure that no backtracking occurs during tree-comparison 2017-08-04 14:50:50 -07:00
Max Brunsfeld
9260d8163c Refactor and fix bugs in tree comparison algorithm 2017-08-04 14:03:41 -07:00
Max Brunsfeld
46ec0804f8 Move tree comparison function to a separate translation unit 2017-08-04 11:03:14 -07:00
Max Brunsfeld
09f4796f6b Get tests passing w/ new alias API 2017-08-01 14:35:34 -07:00
Max Brunsfeld
cb5fe80348 Rename RENAME rule to ALIAS, allow it to create anonymous nodes 2017-07-31 16:41:11 -07:00
Max Brunsfeld
2437287e3e Remove rename symbol when reusing a previously-renamed tree 2017-07-21 16:24:33 -07:00
Max Brunsfeld
cf445da253 Allow renaming hidden rules 2017-07-21 15:58:01 -07:00
Max Brunsfeld
1df41a9107 Avoid anonymous struct to silence gcc's override-init warning (again) 2017-07-21 10:17:54 -07:00
Max Brunsfeld
f33421c53e Fix incorrect node renames in the presence of extra tokens 2017-07-18 21:24:34 -07:00
Max Brunsfeld
10d28d4b56 Merge pull request #92 from tree-sitter/utf16-oob 
Add test for UTF16 out-of-bound read
 2017-07-18 17:24:31 -07:00
Max Brunsfeld
afb499bf2e Handle rename symbols in ts_language APIs 2017-07-18 12:01:52 -07:00
Max Brunsfeld
9a04231ab1 Remove length restriction in external scanner serialization API 2017-07-17 17:12:36 -07:00
Max Brunsfeld
e355929a30 Handle renamed nodes properly in tree_path_get_changes 2017-07-17 17:07:56 -07:00
Phil Turnbull
e7662c2213 Handle out-of-bound read in utf16_iterate 
Also simplify the test so we call `utf16_iterate` directly. Calling
`utf16_iterate` via `SpyInput` and `ts_document_parse` doesn't seem to reliably
trigger the problem using valgrind.

valgrind also doesn't detect the problem if we use a string literal like:
  `utf16_iterate("", 1, &code_point);`
 2017-07-17 13:57:12 -07:00
Max Brunsfeld
99885788bc 🎨 2017-07-14 10:41:09 -07:00
Max Brunsfeld
4b40a1ed6c Support anonymous tokens inside of RENAME rules 2017-07-14 10:19:58 -07:00
Max Brunsfeld
b3a72954ff Introduce RENAME rule type 2017-07-13 17:17:22 -07:00
Max Brunsfeld
7293e6f0cc Fix compile warnings 2017-07-12 22:08:36 -07:00
Max Brunsfeld
d8e9d04fe7 Add PREC_DYNAMIC rule for resolving runtime ambiguities 2017-07-06 15:24:45 -07:00
Max Brunsfeld
8f028ebf68 Avoid deep tree comparison when both trees have errors 2017-07-05 17:33:35 -07:00
Max Brunsfeld
782bf48772 Don't do skip_preceding_subtrees recovery when there are lots of versions 2017-07-05 15:34:19 -07:00
Max Brunsfeld
17bc3dfaf7 Add a benchmark command 
This command measures the speed of parsing each grammar's examples.
It also uses each grammar to parse all of the *other* grammars' examples
in order to measure error recovery performance with fairly large files.
 2017-07-05 14:14:38 -07:00
Max Brunsfeld
d322f0b6a7 🎨 2017-07-04 21:59:54 -07:00
Max Brunsfeld
e7ccd9c17c Put back check for better existing versions during recover action 
When checking for better existing versions, only kill the current
version if there is a better version earlier in the rotation.
 2017-07-03 12:27:23 -07:00
Max Brunsfeld
f93f78ef2d Remove version-pruning criteria based on pushed node count 2017-07-02 23:42:23 -07:00
Max Brunsfeld
f722923493 Limit the search depth for skip_preceding_trees recovery 2017-06-30 17:49:09 -07:00
Max Brunsfeld
89e5037f01 Manually tail-call-optimize stack_node_release function 2017-06-30 17:49:09 -07:00
Max Brunsfeld
eccb3893eb Prune unneeded stack versions based on a depth criteria 2017-06-30 17:49:09 -07:00
Max Brunsfeld
d6579956f5 Enforce a hard version count limit during the recovery action 2017-06-30 17:49:09 -07:00
Max Brunsfeld
061fba6b92 🎨 Just call it 'inline' 2017-06-29 16:49:59 -07:00
Max Brunsfeld
a89322c5f1 Remove unneeded parameters from public interface of stack_iterate callback 2017-06-29 16:43:56 -07:00
Max Brunsfeld
009d6d1534 Improve heuristics for pruning parse versions based on errors 
* Rewrite the error cost comparison in terms of explicit, discrete
conditions.
* Allow merging versions have different error costs.
* Store the depth of each stack version since the last error. Use this
state to prevent incorrect merging.
* Sort the stack versions in order of preference and put a hard limit on
the version count.
 2017-06-29 15:00:20 -07:00
Max Brunsfeld
445be0736a Clean up ts_stack_push function 2017-06-29 15:00:20 -07:00
Max Brunsfeld
66be393b78 Stack - consider empty external token state identical to NULL 2017-06-29 15:00:20 -07:00
Phil Turnbull
5ef9c4d6aa Increase size of ref_counts and add assertions 
This bumps the size of the reference counts from 16- to 32-bit counters to make
it less likely to overflow. Also assert in the retain function that the
reference count didn't overflow.

32-bits seems big enough for non-pathological examples but a more fool-proof
fix may be to bump it to 64-bits.
 2017-06-29 06:39:01 -07:00
Max Brunsfeld
dfd7b1f5f6 Consolidate memory management logic in Stack 2017-06-27 16:17:24 -07:00
Max Brunsfeld
0143bfdad4 Avoid use-after-free of external token states 
Previously, it was possible for references to external token states to
outlive the trees to which those states belonged.

Now, instead of storing references to external token states in the Stack
and in the Lexer, we store references to the external token trees
themselves, and we retain the trees to prevent use-after-free.
 2017-06-27 14:54:27 -07:00
Max Brunsfeld
f678018d3d Avoid use-after-free when copying stack iterators 2017-06-27 14:54:27 -07:00
Max Brunsfeld
f62ee5a0f3 Fix OOB reads at ends of chunks 
Signed-off-by: Philip Turnbull <philipturnbull@github.com>
 2017-06-23 12:09:16 -07:00
Max Brunsfeld
8ee3f96960 Fix formatting of non-ascii unexpected characters 
Signed-off-by: Philip Turnbull <philipturnbull@github.com>
 2017-06-23 12:08:50 -07:00
Max Brunsfeld
513edec7c1 Merge pull request #77 from philipturnbull/scan-build-fixes 
Fix errors found by scan-build
 2017-06-20 10:15:20 -07:00
Max Brunsfeld
599367d36d Always recur into error nodes when reporting changed ranges 2017-06-15 17:06:48 -07:00
Max Brunsfeld
c66fddd3aa Add TSInput option to measure columns in bytes not characters 2017-06-15 16:35:34 -07:00
Phil Turnbull
cfca764d48 Root can never be NULL in this context 2017-06-15 07:47:16 -04:00
Max Brunsfeld
b862db766e Merge remote-tracking branch 'origin/master' into update-fixture-grammars 2017-06-14 17:11:44 -07:00
Phil Turnbull
d1b19e8196 Prevent NULL pointer dereference in parser__accept 
parser__select_tree can return true if 'left != NULL' and 'right == NULL' which
will later cause a NULL ptr deref:

src/runtime/parser.c:842:14: warning: Access to field 'ref_count' results in a dereference of a null pointer (loaded from variable 'root')
      assert(root->ref_count > 0);
             ^~~~~~~~~~~~~~~
 2017-06-14 11:12:06 -04:00