Max Brunsfeld
e7217f1bac
Clean up some methods in parser.c
2016-11-14 17:25:55 -08:00
Max Brunsfeld
535879a2bd
Represent byte, char and tree counts as 32 bit numbers
...
The parser spends the majority of its time allocating and freeing trees and stack nodes.
Also, the memory footprint of the AST is a significant concern when using tree-sitter
with large files. This library is already unlikely to work very well with source files
larger than 4GB, so representing rows, columns, byte lengths and child indices as
unsigned 32 bit integers seems like the right choice.
2016-11-14 12:19:13 -08:00
Max Brunsfeld
1118a9142a
Introduce Symbol::Index type alias
2016-11-14 10:25:26 -08:00
Max Brunsfeld
fad7294ba4
Store shift states for non-terminals directly in the main parse table
2016-11-14 08:36:06 -08:00
Max Brunsfeld
8d9c261e3a
Don't include reduce actions for nonterminal lookaheads
2016-11-10 11:33:37 -08:00
Max Brunsfeld
c9dcb29c6f
Remove the TS prefix from some internal type/function names
2016-11-09 20:59:05 -08:00
Max Brunsfeld
7bcae8f6a8
🎨 flatten_grammar
2016-11-09 20:29:21 -08:00
Max Brunsfeld
4106ecda43
Remove logic for recovering from OOM
2016-11-04 09:18:38 -07:00
Timothy Clem
693c6d40dd
Move setup of mergeable_symbols to constructor, use set throughout
2016-10-18 15:18:33 -07:00
Max Brunsfeld
eed54d95e1
Merge branch 'master' into changed-ranges
2016-10-16 21:10:25 -07:00
Max Brunsfeld
1412419434
Verify changed ranges for random deletions as well as random insertions
2016-10-16 14:55:51 -07:00
Max Brunsfeld
4bad58b407
Remove outdated test
2016-10-16 14:54:59 -07:00
Max Brunsfeld
b3140b2689
Implement ts_document_parse_and_get_changed_ranges
2016-10-15 22:31:21 -07:00
Max Brunsfeld
6bde4c54d9
Merge pull request #40 from tree-sitter/remove-runtime-dep-from-parser-header
...
Remove generated parsers' dependency on runtime.h
2016-10-05 14:13:53 -07:00
Max Brunsfeld
e149d94ff5
Remove generated parsers' dependency on runtime.h
2016-10-05 14:02:49 -07:00
Timothy Clem
0ffebc3d0c
s/TSDebugger/TSLogger in spec name
2016-10-05 08:48:12 -07:00
Max Brunsfeld
3014101104
Fix inconsistencies in nodes sizes after edits
2016-09-19 13:35:08 -07:00
Max Brunsfeld
744eb7b356
Assert consistency of trees' sizes in bytes
2016-09-19 12:40:45 -07:00
Max Brunsfeld
00528e50ce
Change edit API to be byte-based
2016-09-13 13:08:52 -07:00
Max Brunsfeld
cc62fe0375
Represent Lengths in terms of Points
2016-09-09 21:11:02 -07:00
Max Brunsfeld
eb5dda75c4
Start work on randomized changed-region testing
...
Signed-off-by: Nathan Sobo <nathan@github.com>
2016-09-09 09:20:04 -07:00
Max Brunsfeld
591fcc980c
Update trees' offsets when editing them
2016-09-08 17:54:51 -07:00
Max Brunsfeld
131bbee160
Rename parse_and_diff -> parse_and_get_changed_ranges
...
Signed-off-by: Nathan Sobo <nathan@github.com>
2016-09-08 17:51:34 -07:00
Max Brunsfeld
fce8d57152
Start work on document_parse_and_diff API
2016-09-08 17:51:20 -07:00
Max Brunsfeld
a6a08dde31
Rename ts_node_name -> ts_node_type
2016-09-06 21:43:59 -07:00
Max Brunsfeld
38241d466b
Rename .read_fn, .seek_fn -> .read, .seek
2016-09-06 21:39:10 -07:00
Max Brunsfeld
f6da44fdbb
Add ts_node_descendant_for_byte_range
2016-09-06 21:33:19 -07:00
Max Brunsfeld
70756034f1
Allow descendant queries by both 1D and 2D coordinates
2016-09-06 21:17:26 -07:00
Max Brunsfeld
096ac2d4b6
Rename ts_document_set_debugger -> ts_document_set_logger
2016-09-06 17:40:26 -07:00
Max Brunsfeld
64a6c9db0e
Rename ts_document_make -> ts_document_new
2016-09-06 17:26:18 -07:00
Max Brunsfeld
b76574e01c
Handle ambiguities between extra and non-extra tokens using normal GLR splitting
2016-09-06 10:22:16 -07:00
Max Brunsfeld
1c52c30111
Fix unexpected EOF errors getting lost
2016-09-03 22:46:14 -07:00
Max Brunsfeld
b6e2bed761
Fix specs after js grammar change
2016-09-01 17:45:35 -07:00
Max Brunsfeld
c1b6d9f5be
Improve error comparison criteria
...
Signed-off-by: Nathan Sobo <nathan@github.com>
2016-09-01 11:39:23 -07:00
Max Brunsfeld
7483da4184
Add push_count to stack, use it in error comparisons
2016-08-31 17:29:14 -07:00
Max Brunsfeld
0faae52132
Fix some inconsistencies in error cost calculation
...
Signed-off-by: Nathan Sobo <nathan@github.com>
2016-08-31 10:51:59 -07:00
Max Brunsfeld
1d617ab5e0
Allow reductions based on error token, skipping some preceding content
2016-08-29 17:34:51 -07:00
Max Brunsfeld
31d1160e21
Base error costs on top-level trees skipped and lines of text skipped
...
Rather than on the total number of tokens skipped
2016-08-29 17:06:23 -07:00
Max Brunsfeld
de9b9db7c5
Remove accidentally-committed reference stack files
2016-08-29 13:07:46 -07:00
Max Brunsfeld
f209c24cf6
Trim trailing newlines from corpus texts
2016-08-29 13:00:28 -07:00
Max Brunsfeld
e947d7e2ad
Adjust test assertions for subtly different recoveries
2016-08-29 11:23:52 -07:00
Max Brunsfeld
1b8843dd41
Perform all possible reductions recursively upon detecting an error
2016-08-29 11:23:35 -07:00
Max Brunsfeld
87ca3cb099
Reuse nodes based on state matching, not sentential form validity
...
I think that state matching is the only correct strategy for incremental
node reuse that is compatible with the new error recovery algorithm. It's
also simpler than the sentential-form algorithm. With the compressed parse
tables, state matching shouldn't be too conservative of a test.
2016-07-31 21:31:19 -07:00
Max Brunsfeld
285f2272fd
Move random string helpers into a separate file
2016-07-17 06:22:05 -07:00
Max Brunsfeld
c3a242740b
Allow lookahead to be broken down further after performing reductions
2016-07-04 12:20:23 -07:00
Max Brunsfeld
8c26d99353
Store error recovery actions in the normal parse table
2016-06-27 14:07:47 -07:00
Max Brunsfeld
08e47001f1
Silence mismatched delete warning in spec helper
2016-06-27 13:38:49 -07:00
Max Brunsfeld
9538b5b879
Don't count extra trees toward stack versions' error costs
2016-06-26 22:46:40 -07:00
Max Brunsfeld
6fd3edceae
Fix logic for inserting leading & trailing extras into root node on acceptance
2016-06-26 11:57:42 -07:00
Max Brunsfeld
9972709e43
Allow error recovery to skip non-terminal nodes after error detection
2016-06-24 10:28:05 -07:00
