Max Brunsfeld
ac9d260734
Clean up parser fields
2017-08-31 12:50:10 -07:00
Max Brunsfeld
4a0587061e
Consolidate logic for deciding on a lookahead node
2017-08-31 12:19:37 -07:00
Max Brunsfeld
a98d449d88
Add an option to immediately halt on syntax error
2017-05-01 13:50:49 -07:00
Max Brunsfeld
3a4daace26
Move reusable node functions to their own file
2017-01-05 10:07:27 -08:00
Max Brunsfeld
e6c82ead2c
Start work toward maintaining external scanner's state during incremental parses
2016-12-20 17:06:20 -08:00
Max Brunsfeld
0f8e130687
Call external scanner functions when lexing
2016-12-02 22:03:48 -08:00
Max Brunsfeld
535879a2bd
Represent byte, char and tree counts as 32 bit numbers
...
The parser spends the majority of its time allocating and freeing trees and stack nodes.
Also, the memory footprint of the AST is a significant concern when using tree-sitter
with large files. This library is already unlikely to work very well with source files
larger than 4GB, so representing rows, columns, byte lengths and child indices as
unsigned 32 bit integers seems like the right choice.
2016-11-14 12:19:13 -08:00
Max Brunsfeld
c9dcb29c6f
Remove the TS prefix from some internal type/function names
2016-11-09 20:59:05 -08:00
Max Brunsfeld
eed54d95e1
Merge branch 'master' into changed-ranges
2016-10-16 21:10:25 -07:00
Max Brunsfeld
b3140b2689
Implement ts_document_parse_and_get_changed_ranges
2016-10-15 22:31:21 -07:00
Max Brunsfeld
e149d94ff5
Remove generated parsers' dependency on runtime.h
2016-10-05 14:02:49 -07:00
Max Brunsfeld
fcf9293d35
Use explicit stack for assigning trees' parent pointers
2016-09-19 12:40:45 -07:00
Max Brunsfeld
00528e50ce
Change edit API to be byte-based
2016-09-13 13:08:52 -07:00
Max Brunsfeld
e7ed92f1df
Replace ts_parser -> parser
2016-08-29 12:08:58 -07:00
Max Brunsfeld
c3a242740b
Allow lookahead to be broken down further after performing reductions
2016-07-04 12:20:23 -07:00
Max Brunsfeld
9c37c06ec7
Remove unnecessary parser debugger accessors
2016-07-04 12:20:23 -07:00
Max Brunsfeld
22c550c9d6
Discard tokens after error detection to find the best repair
...
* Use GLR stack-splitting to try all numbers of tokens to
discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
rather than the statically-computed 'in-progress symbols' list
2016-05-11 13:49:43 -07:00
Max Brunsfeld
fd4c33209e
Select ambiguous alternatives by minimizing error size
2016-04-24 00:54:20 -07:00
Max Brunsfeld
956b3e68d7
Only store one reusable node, not one for each stack version
2016-04-01 21:17:23 -07:00
Max Brunsfeld
2e35587161
Use new stack_pop_until function for repairing errors
2016-03-07 20:06:46 -08:00
Max Brunsfeld
aef7582a2a
Start using the forward move to recover from errors
...
Some unit tests passing. Corpus tests still failing
2016-03-02 21:08:42 -08:00
Max Brunsfeld
abbc282950
Add a public function for printing debugging graphs
2016-02-23 11:16:50 -08:00
Max Brunsfeld
3d7df851b5
Rename Vector -> Array
2016-02-17 20:41:29 -08:00
Max Brunsfeld
6fa7eca966
Make vector struct type-safe
2016-02-17 15:30:47 -08:00
Max Brunsfeld
1543a6c7b0
Handle allocation failures when instantiating documents
2016-01-18 11:32:10 -08:00
Max Brunsfeld
13d8de3eb3
Remove stack head after it is accepted
2015-12-24 22:04:20 -08:00
Max Brunsfeld
6abce098c0
Track whether parse stack was split before consuming each lookahead
2015-12-10 13:00:27 -08:00
Max Brunsfeld
26e34ddcbb
Allow all parse stack heads to finish
2015-12-08 13:01:33 -08:00
Max Brunsfeld
32b1088823
Rename HeadState -> LookaheadState, remove parser.lookahead field
2015-11-20 12:55:01 -08:00
Max Brunsfeld
c88e9044d5
Make stack popping more robust
2015-11-20 00:04:21 -08:00
Max Brunsfeld
64874449e4
Allow different parse stack heads to lex differently
2015-11-19 20:55:18 -08:00
Max Brunsfeld
f37f73f92f
Add ability to edit multiple times between parses
2015-09-18 23:20:06 -07:00
Max Brunsfeld
6254f45c1b
Rename ParseStack -> Stack
2015-09-18 22:02:06 -07:00
Max Brunsfeld
b3d883e128
Store edits in trees, not by splitting stack
...
This allows for multiple edits per parse, though it is not exposed through
the API yet
2015-09-18 22:02:06 -07:00
Max Brunsfeld
252fa7b631
Add Document getter methods for input, language
2015-09-08 23:33:43 -07:00
Max Brunsfeld
54e40b8146
Rework AST access API: reduce heap allocation
2015-07-31 15:47:48 -07:00
Max Brunsfeld
442db56b92
Use graph-structured parse stack in parser
...
Not using the splitting feature yet.
2015-06-18 17:03:17 -07:00
Max Brunsfeld
f7e4445358
Handle goto actions after reductions in a more standard way
...
Rather than letting the reduced tree become the new lookahead symbol,
and re-adding it to the stack via a subsequent shift action, just
add it to the stack as part of the reduce action. This is more in
line with the way LR is described traditionally.
2015-05-27 10:53:02 -07:00
Max Brunsfeld
8cf800ef5d
Unify debugging API for parsing and lexing
2014-10-17 17:52:54 -07:00
Max Brunsfeld
c594208ab8
Allow callbacks to be specified for debug output
2014-10-13 01:02:18 -07:00
Max Brunsfeld
4dcc712a8c
Start work on re-using right side of parse tree
2014-10-09 19:58:15 -07:00
Max Brunsfeld
4327f3ed26
Refactor parser and stack
2014-08-09 01:03:55 -07:00
Max Brunsfeld
eecbcccee0
Remove generated parsers' dependency on the runtime library
...
Generated parsers no longer export a parser constructor function.
They now export an opaque Language object which can be set on
Documents directly. This way, the logic for constructing parsers
lives entirely in the runtime. The Languages are just structs which
have no load-time dependency on the runtime
2014-07-30 23:40:02 -07:00
