Max Brunsfeld
36608180d2
Store external token states in the parse stack
2017-01-08 22:06:05 -08:00
Max Brunsfeld
3a4daace26
Move reusable node functions to their own file
2017-01-05 10:07:27 -08:00
Max Brunsfeld
83c4c03a70
Remove unnecessary describes in Parser spec
2016-12-21 11:38:25 -08:00
Max Brunsfeld
75bcfed247
Rename doc variable -> document in Parser spec
2016-12-21 11:32:27 -08:00
Max Brunsfeld
4fd7b0e55d
Remove nested before_each's in Parser spec
2016-12-21 11:31:28 -08:00
Max Brunsfeld
e6c82ead2c
Start work toward maintaining external scanner's state during incremental parses
2016-12-20 17:06:20 -08:00
Max Brunsfeld
727727623a
Start work on unit test that edits python code
...
Signed-off-by: Nathan Sobo <nathan@github.com>
2016-12-20 13:10:18 -08:00
Max Brunsfeld
4106ecda43
Remove logic for recovering from OOM
2016-11-04 09:18:38 -07:00
Max Brunsfeld
eed54d95e1
Merge branch 'master' into changed-ranges
2016-10-16 21:10:25 -07:00
Max Brunsfeld
4bad58b407
Remove outdated test
2016-10-16 14:54:59 -07:00
Max Brunsfeld
e149d94ff5
Remove generated parsers' dependency on runtime.h
2016-10-05 14:02:49 -07:00
Max Brunsfeld
3014101104
Fix inconsistencies in nodes sizes after edits
2016-09-19 13:35:08 -07:00
Max Brunsfeld
a6a08dde31
Rename ts_node_name -> ts_node_type
2016-09-06 21:43:59 -07:00
Max Brunsfeld
70756034f1
Allow descendant queries by both 1D and 2D coordinates
2016-09-06 21:17:26 -07:00
Max Brunsfeld
096ac2d4b6
Rename ts_document_set_debugger -> ts_document_set_logger
2016-09-06 17:40:26 -07:00
Max Brunsfeld
64a6c9db0e
Rename ts_document_make -> ts_document_new
2016-09-06 17:26:18 -07:00
Max Brunsfeld
1c52c30111
Fix unexpected EOF errors getting lost
2016-09-03 22:46:14 -07:00
Max Brunsfeld
b6e2bed761
Fix specs after js grammar change
2016-09-01 17:45:35 -07:00
Max Brunsfeld
1d617ab5e0
Allow reductions based on error token, skipping some preceding content
2016-08-29 17:34:51 -07:00
Max Brunsfeld
e947d7e2ad
Adjust test assertions for subtly different recoveries
2016-08-29 11:23:52 -07:00
Max Brunsfeld
87ca3cb099
Reuse nodes based on state matching, not sentential form validity
...
I think that state matching is the only correct strategy for incremental
node reuse that is compatible with the new error recovery algorithm. It's
also simpler than the sentential-form algorithm. With the compressed parse
tables, state matching shouldn't be too conservative of a test.
2016-07-31 21:31:19 -07:00
Max Brunsfeld
c3a242740b
Allow lookahead to be broken down further after performing reductions
2016-07-04 12:20:23 -07:00
Max Brunsfeld
773e50f26b
Update error recovery specs to reflect slightly different recoveries
2016-06-18 20:46:16 -07:00
Max Brunsfeld
1e353381ff
Don't create error node in lexer unless token is completely invalid
...
Before, any syntax error would cause the lexer to create an error
leaf node. This could happen even with a valid input, if the parse
stack had split and one particular version of the parse stack
failed to parse.
Now, an error leaf node is only created when the lexer cannot understand
part of the input stream at all. When a normal syntax error occurs,
the lexer just returns a token that is outside of the expected token
set, and the parser handles the unexpected token.
2016-05-26 14:15:10 -07:00
Max Brunsfeld
22c550c9d6
Discard tokens after error detection to find the best repair
...
* Use GLR stack-splitting to try all numbers of tokens to
discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
rather than the statically-computed 'in-progress symbols' list
2016-05-11 13:49:43 -07:00
Max Brunsfeld
0d19f157ed
Adjust some spec assertions to reflect finer-grained error recoveries
2016-04-22 10:19:44 -07:00
Max Brunsfeld
240355b04c
Make test for allocation failure handling fail more gracefully
2016-03-10 11:36:26 -08:00
Max Brunsfeld
2e35587161
Use new stack_pop_until function for repairing errors
2016-03-07 20:06:46 -08:00
Max Brunsfeld
bc8df9f5c5
Avoid recompiling test languages when possible
2016-03-03 12:05:04 -08:00
Max Brunsfeld
aef7582a2a
Start using the forward move to recover from errors
...
Some unit tests passing. Corpus tests still failing
2016-03-02 21:08:42 -08:00
Max Brunsfeld
e7abfdd373
Prevent string assertion failures from creating later memory leak errors
2016-03-02 20:58:39 -08:00
Max Brunsfeld
c96c4a08e6
Use an object pool for stack nodes, to reduce allocations
...
Also, fix some leaks in the case where memory allocation failed during parsing
2016-02-04 11:19:42 -08:00
Max Brunsfeld
4a7312e514
Fix memory leaks when editing
2016-02-02 12:22:29 -08:00
Max Brunsfeld
39417e55df
Test successful parsing after handling allocation failures
2016-01-19 19:52:07 -08:00
Max Brunsfeld
3dde0a6f39
Handle allocation failures during parsing
2016-01-19 18:08:01 -08:00
Max Brunsfeld
1543a6c7b0
Handle allocation failures when instantiating documents
2016-01-18 11:32:10 -08:00
Max Brunsfeld
0f7dbea9a3
Unify test targets, use externally defined languages as fixtures
2016-01-15 11:19:24 -08:00
Max Brunsfeld
1c6ad5f7e4
Rename ubiquitous_tokens -> extra_tokens in compiler API
...
They were already called this in the runtime code.
'Extra' is just easier to say.
2015-12-17 15:50:50 -08:00
Max Brunsfeld
b3a6de6dad
Replace node pos/size functions with start/end char/byte functions
2015-12-03 22:59:27 -08:00
Max Brunsfeld
863cabc827
Don't include trailing ubiquitous tokens as children when reducing
2015-12-02 15:31:15 -08:00
Max Brunsfeld
010a606b76
Add undo method to SpyInput
2015-09-19 14:46:14 -07:00
Max Brunsfeld
f37f73f92f
Add ability to edit multiple times between parses
2015-09-18 23:20:06 -07:00
Max Brunsfeld
b3d883e128
Store edits in trees, not by splitting stack
...
This allows for multiple edits per parse, though it is not exposed through
the API yet
2015-09-18 22:02:06 -07:00
Max Brunsfeld
66ad271c82
Add Document parse_count() method, for tracking if trees are still valid
2015-09-10 14:23:42 -07:00
Max Brunsfeld
7ee5eaa16a
Rename node accessor methods
...
Instead of child() vs concrete_child(), next_sibling() vs next_concrete_sibling(), etc,
the default is switched: child() refers to the concrete syntax tree, and named_child()
refers to the AST. Because the AST is abstract through exclusion of some nodes, the
names are clearer if the qualifier goes on the AST operations
2015-09-08 23:16:24 -07:00
Max Brunsfeld
53926c467e
Don't automatically hide singleton nodes
2015-09-02 16:36:29 -07:00
Max Brunsfeld
acf9280eda
Make expression and statement rules hidden in javascript grammar
2015-09-02 13:05:54 -07:00
Max Brunsfeld
21258e6a9e
Remove 'document' wrapper node
2015-08-22 10:48:34 -07:00
Max Brunsfeld
e89ec7c85e
Whitespace
2015-07-31 16:29:01 -07:00
Max Brunsfeld
54e40b8146
Rework AST access API: reduce heap allocation
2015-07-31 15:47:48 -07:00
