Max Brunsfeld
379a2fd121
Incrementally build a tree of skipped tokens
...
Rather than pushing them to the stack individually
2018-04-09 12:29:22 -07:00
Max Brunsfeld
a79a8c216f
Include each node count and error cost in stack dot graphs
2018-04-06 16:10:36 -07:00
Max Brunsfeld
87098760de
🎨
2018-04-06 13:51:31 -07:00
Max Brunsfeld
94ed1b6964
Make array_splice take an array, not a pointer and length
2018-04-06 13:29:43 -07:00
Max Brunsfeld
09be0b6ef5
Store trees' children in TreeArrays, not w/ separate pointer and length
2018-04-06 13:26:18 -07:00
Max Brunsfeld
a6cf2e87e7
Fix halt_on_error tests
2018-04-06 13:26:18 -07:00
Max Brunsfeld
34349f9cbb
Put back limit on error handling based on accepted tree count
2018-04-06 13:26:18 -07:00
Max Brunsfeld
0ec7e5ce42
Remove ts_stack_force_merge function
2018-04-06 13:26:18 -07:00
Max Brunsfeld
1d9d6f37ad
Introduce an error cost per error instance to favor fewer errors
2018-04-06 13:26:18 -07:00
Max Brunsfeld
80f856cef5
Maintain a total node count on every tree
...
This simplifies (and fixes bugs in) the parse stack's tracking of its
total node count since the last error, which is needed for error
recovery.
2018-04-06 13:26:18 -07:00
Max Brunsfeld
f3bbf045b7
Avoid unnecessary stack node retain and release on every push
2018-04-06 13:26:18 -07:00
Max Brunsfeld
e59558c83b
Allow stack versions to be temporarily paused
...
This way, when detecting an error, we can defer the decision about
whether to bail or recover until all stack versions are processed.
2018-04-06 13:26:18 -07:00
Max Brunsfeld
dbe77e7199
Simplify testing-only ts_stack_iterate function
2018-03-29 17:50:07 -07:00
Max Brunsfeld
5520983144
Clean up Stack API
...
* Remove StackPopResult
* Rename top_state() -> state()
* Rename top_position() -> position()
* Improve docs
2018-03-29 17:37:54 -07:00
Max Brunsfeld
ee995c3d6b
Avoid redundant retains/releases by giving ts_stack_push move semantics
2018-03-29 17:18:43 -07:00
Max Brunsfeld
e917756ad1
Remove depends_on_lookahead field from parse table entries
...
This simplifies the logic for determining whether a token is reusable
and makes it more conservative. It should fix some incremental parsing
bugs that are being caught by the randomized tests on CI.
2018-03-28 10:58:33 -07:00
Max Brunsfeld
0810971f3e
🔥 symbol iterator API
...
This idea was never fully baked.
2018-03-08 14:16:37 -08:00
Max Brunsfeld
e927d02f43
Allow reusing leaf nodes unless the next leaf has changes
2018-03-07 17:44:54 -08:00
Max Brunsfeld
c0cc35ff07
Create separate lexer function for keywords
2018-03-07 12:00:26 -08:00
Max Brunsfeld
f96969738b
Don't remove mergeable stack versions so aggressively during condense
2018-03-05 10:40:05 -08:00
Max Brunsfeld
dbc0c208f4
Add missing initialization of parser's in_ambiguity state
2018-03-02 15:25:39 -08:00
Max Brunsfeld
52087de4f0
Remove the concept of fragile reductions
...
They were a vestige of when Tree-sitter did sentential form-based
incremental parsing (as opposed to simply state matching). This was
elegant but not compatible with GLR as far as I could tell.
2018-03-02 14:51:54 -08:00
Max Brunsfeld
82c7e170b3
Fix case where loop was created in the parse stack
...
Fixes #133
2018-03-02 09:05:20 -08:00
Max Brunsfeld
16a45d4aa4
Fix hole in logic for terminating tree balancing
...
It's important that the repetition nodes have a child count of 2,
because we assign to their second child. We could maybe generalize
this to allow balancing in the presence of 'extra' nodes like comments
and errors, but this might be complicated.
2018-02-16 12:44:30 -08:00
Max Brunsfeld
facafcd6e4
Pass row/column position to input seek method
2018-02-14 07:31:49 -08:00
Max Brunsfeld
134c455b80
Simplify logic for terminating tree balancing
2018-02-12 12:13:21 -08:00
Max Brunsfeld
299a146b66
Balance repetition trees after parsing
2018-02-12 11:41:56 -08:00
Max Brunsfeld
8c29841adf
Represent repetitions with associative structure
2018-02-12 11:41:56 -08:00
Max Brunsfeld
f8704d28da
Log changed ranges when reparsing
2018-01-26 15:40:07 -08:00
Max Brunsfeld
46dcd53090
Do not insert missing tokens if halt_on_error option is passed
2018-01-24 14:04:55 -08:00
Max Brunsfeld
919c9d8715
Ensure root node has a null parent
2018-01-23 17:20:15 -08:00
Max Brunsfeld
dafa897021
Bail on error recovery if too many alternative parses have already completed
2018-01-09 17:08:36 -08:00
Max Brunsfeld
315dff3285
Add an API for getting a node's child index
2018-01-09 14:01:36 -08:00
Max Brunsfeld
f653f2b3bb
Add ts_node_first_{child,named_child}_for_byte methods
2018-01-09 13:44:59 -08:00
Max Brunsfeld
1e04489e50
Fix error in handling of padding in get_changed_ranges
2017-12-29 18:02:06 -08:00
Max Brunsfeld
f3c3fd3c9e
Make it easier to enable/disable logging in get_changed_ranges
2017-12-29 18:01:42 -08:00
Max Brunsfeld
21a88b1731
Don't count less-far-along versions in better_version_exists method
2017-12-29 16:10:43 -08:00
Max Brunsfeld
d3c85f288d
Start work on repairing errors by inserting missing tokens
2017-12-29 15:11:00 -08:00
Max Brunsfeld
f2dc620610
Extract parser__recover_to_state method
2017-12-29 15:10:59 -08:00
Max Brunsfeld
adf47e2b57
Fix invalid usage of 'extra' field on non-shift parse action
2017-12-29 11:46:41 -08:00
Max Brunsfeld
d9094e8146
Consolidate more logic into do_potential_reductions method
2017-12-28 15:49:48 -08:00
Max Brunsfeld
eee3db08d2
Avoid repeated calls to {start,end}_point in descendant_for_point_range
2017-12-27 11:55:52 -08:00
Max Brunsfeld
172cbb2d22
Fix infinite loop due to skipping empty tokens during error recovery
2017-12-27 11:18:06 -08:00
Max Brunsfeld
2625c3a96c
Remove dead code in string_input.c
2017-12-27 10:34:29 -08:00
Max Brunsfeld
addeb6c4c1
Allocate and free trees using an object pool
2017-12-27 10:34:29 -08:00
Max Brunsfeld
0e69da37a5
Return a character count from the lexer's get_column method
2017-12-20 16:26:38 -08:00
Max Brunsfeld
fcff16cb86
Add get_column method to lexer
2017-12-19 17:54:15 -08:00
Max Brunsfeld
fbcefe25f7
Avoid creating external tokens that start after they end
2017-12-07 11:50:27 -08:00
Max Brunsfeld
5d676de051
Remove unnecessary conditional in parser__accept
2017-12-07 11:50:27 -08:00
Max Brunsfeld
48681c3f0e
Initialize error start and end positions at their declarations
...
Fixes #113
Clang doesn't seem to be able to tell that these variables were guaranteed to
be initialized by the time they were read.
2017-10-31 10:06:44 -07:00
