Max Brunsfeld
5520983144
Clean up Stack API
...
* Remove StackPopResult
* Rename top_state() -> state()
* Rename top_position() -> position()
* Improve docs
2018-03-29 17:37:54 -07:00
Max Brunsfeld
ee995c3d6b
Avoid redundant retains/releases by giving ts_stack_push move semantics
2018-03-29 17:18:43 -07:00
Max Brunsfeld
fb348c0f1e
Fix signed/unsigned comparison warning
2018-03-28 11:04:49 -07:00
Max Brunsfeld
e917756ad1
Remove depends_on_lookahead field from parse table entries
...
This simplifies the logic for determining whether a token is reusable
and makes it more conservative. It should fix some incremental parsing
bugs that are being caught by the randomized tests on CI.
2018-03-28 10:58:33 -07:00
Max Brunsfeld
186f70649c
Consolidate the unify for detecting conflicting tokens
2018-03-28 10:03:09 -07:00
Max Brunsfeld
a8bc67ac42
Allow LookaheadSet::for_each to terminate early
2018-03-28 10:03:09 -07:00
Max Brunsfeld
43e14332ed
Avoid creating duplicate metadata rules
2018-03-28 10:03:09 -07:00
Max Brunsfeld
b7d0606fbd
Be less conservative in merging parse states with external tokens
...
Also, clean up the internal representation of external tokens
2018-03-16 16:00:40 -07:00
Max Brunsfeld
7183f8d3e7
Fix unit reduction elimination bugs
...
* Handle 'chains' of unit reductions starting in a single state
* Avoid eliminating rules which will later receive aliases
2018-03-12 07:54:18 -07:00
Max Brunsfeld
72849787b1
Fix logic for identifying keyword capture token
2018-03-12 07:52:57 -07:00
Max Brunsfeld
0810971f3e
🔥 symbol iterator API
...
This idea was never fully baked.
2018-03-08 14:16:37 -08:00
Max Brunsfeld
128edbebd6
Eliminate non-user-visible unit reductions from parse tables
2018-03-08 12:53:32 -08:00
Max Brunsfeld
e927d02f43
Allow reusing leaf nodes unless the next leaf has changes
2018-03-07 17:44:54 -08:00
Max Brunsfeld
53cd89c614
Ensure keyword capture tokens aren't too loosely defined
2018-03-07 14:46:11 -08:00
Max Brunsfeld
c0cc35ff07
Create separate lexer function for keywords
2018-03-07 12:00:26 -08:00
Max Brunsfeld
f96969738b
Don't remove mergeable stack versions so aggressively during condense
2018-03-05 10:40:05 -08:00
Max Brunsfeld
dbc0c208f4
Add missing initialization of parser's in_ambiguity state
2018-03-02 15:25:39 -08:00
Max Brunsfeld
52087de4f0
Remove the concept of fragile reductions
...
They were a vestige of when Tree-sitter did sentential form-based
incremental parsing (as opposed to simply state matching). This was
elegant but not compatible with GLR as far as I could tell.
2018-03-02 14:51:54 -08:00
Max Brunsfeld
82c7e170b3
Fix case where loop was created in the parse stack
...
Fixes #133
2018-03-02 09:05:20 -08:00
Max Brunsfeld
32ef3e001a
Account for epsilon external tokens when merging parse states
...
Do not merge a token T into a parse state S if S contains
external tokens that can be *followed* by tokens that could
be shadowed by T.
At this point, the only automated test for this logic is via
the bash grammar, in which the `]` token should not be merged
into states in which `_concat` is valid, because `_concat`
can be followed by a `_special_characters` token, and `]`
would shadow `_special_characters`.
2018-02-28 14:47:04 -08:00
Max Brunsfeld
10a3cbd814
Move grammar schema to src folder
...
Now that there's a docs folder that contains actual docs.
2018-02-26 00:40:20 -08:00
Max Brunsfeld
16a45d4aa4
Fix hole in logic for terminating tree balancing
...
It's important that the repetition nodes have a child count of 2,
because we assign to their second child. We could maybe generalize
this to allow balancing in the presence of 'extra' nodes like comments
and errors, but this might be complicated.
2018-02-16 12:44:30 -08:00
Max Brunsfeld
2daae48fe0
Handle conflicts in repeat rules after external tokens
...
Signed-off-by: Rick Winfrey <rewinfrey@github.com>
2018-02-14 11:24:51 -08:00
Max Brunsfeld
facafcd6e4
Pass row/column position to input seek method
2018-02-14 07:31:49 -08:00
Max Brunsfeld
134c455b80
Simplify logic for terminating tree balancing
2018-02-12 12:13:21 -08:00
Max Brunsfeld
299a146b66
Balance repetition trees after parsing
2018-02-12 11:41:56 -08:00
Max Brunsfeld
8c29841adf
Represent repetitions with associative structure
2018-02-12 11:41:56 -08:00
Max Brunsfeld
f8704d28da
Log changed ranges when reparsing
2018-01-26 15:40:07 -08:00
Max Brunsfeld
46dcd53090
Do not insert missing tokens if halt_on_error option is passed
2018-01-24 14:04:55 -08:00
Max Brunsfeld
919c9d8715
Ensure root node has a null parent
2018-01-23 17:20:15 -08:00
Max Brunsfeld
b520bdd2d5
Merge pull request #126 from tree-sitter/mb-fix-epsilon-rule-loophole
...
Don't allow an epsilon start rule if it is used in other rules
2018-01-23 17:19:55 -08:00
Max Brunsfeld
2e4f76c164
Don't allow an epsilon start rule if it is used in other rules
2018-01-23 17:05:28 -08:00
Max Brunsfeld
dafa897021
Bail on error recovery if too many alternative parses have already completed
2018-01-09 17:08:36 -08:00
Max Brunsfeld
315dff3285
Add an API for getting a node's child index
2018-01-09 14:01:36 -08:00
Max Brunsfeld
f653f2b3bb
Add ts_node_first_{child,named_child}_for_byte methods
2018-01-09 13:44:59 -08:00
Max Brunsfeld
1e04489e50
Fix error in handling of padding in get_changed_ranges
2017-12-29 18:02:06 -08:00
Max Brunsfeld
f3c3fd3c9e
Make it easier to enable/disable logging in get_changed_ranges
2017-12-29 18:01:42 -08:00
Max Brunsfeld
21a88b1731
Don't count less-far-along versions in better_version_exists method
2017-12-29 16:10:43 -08:00
Max Brunsfeld
d3c85f288d
Start work on repairing errors by inserting missing tokens
2017-12-29 15:11:00 -08:00
Max Brunsfeld
f2dc620610
Extract parser__recover_to_state method
2017-12-29 15:10:59 -08:00
Max Brunsfeld
adf47e2b57
Fix invalid usage of 'extra' field on non-shift parse action
2017-12-29 11:46:41 -08:00
Max Brunsfeld
d9094e8146
Consolidate more logic into do_potential_reductions method
2017-12-28 15:49:48 -08:00
Max Brunsfeld
eee3db08d2
Avoid repeated calls to {start,end}_point in descendant_for_point_range
2017-12-27 11:55:52 -08:00
Max Brunsfeld
172cbb2d22
Fix infinite loop due to skipping empty tokens during error recovery
2017-12-27 11:18:06 -08:00
Max Brunsfeld
2625c3a96c
Remove dead code in string_input.c
2017-12-27 10:34:29 -08:00
Max Brunsfeld
addeb6c4c1
Allocate and free trees using an object pool
2017-12-27 10:34:29 -08:00
Max Brunsfeld
0e69da37a5
Return a character count from the lexer's get_column method
2017-12-20 16:26:38 -08:00
Max Brunsfeld
fcff16cb86
Add get_column method to lexer
2017-12-19 17:54:15 -08:00
Max Brunsfeld
532bbeca0d
Remove wrong handling of \a in a regex
2017-12-12 16:50:53 -08:00
Max Brunsfeld
fbcefe25f7
Avoid creating external tokens that start after they end
2017-12-07 11:50:27 -08:00
