traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1253 commits 16 branches 98 tags 37 MiB
Commit graph

86 commits

Author SHA1 Message Date
Max Brunsfeld
e149d94ff5 Remove generated parsers' dependency on runtime.h 2016-10-05 14:02:49 -07:00
Max Brunsfeld
a6a08dde31 Rename ts_node_name -> ts_node_type 2016-09-06 21:43:59 -07:00
Max Brunsfeld
70756034f1 Allow descendant queries by both 1D and 2D coordinates 2016-09-06 21:17:26 -07:00
Max Brunsfeld
096ac2d4b6 Rename ts_document_set_debugger -> ts_document_set_logger 2016-09-06 17:40:26 -07:00
Max Brunsfeld
64a6c9db0e Rename ts_document_make -> ts_document_new 2016-09-06 17:26:18 -07:00
Max Brunsfeld
1c52c30111 Fix unexpected EOF errors getting lost 2016-09-03 22:46:14 -07:00
Max Brunsfeld
b6e2bed761 Fix specs after js grammar change 2016-09-01 17:45:35 -07:00
Max Brunsfeld
1d617ab5e0 Allow reductions based on error token, skipping some preceding content 2016-08-29 17:34:51 -07:00
Max Brunsfeld
e947d7e2ad Adjust test assertions for subtly different recoveries 2016-08-29 11:23:52 -07:00
Max Brunsfeld
87ca3cb099 Reuse nodes based on state matching, not sentential form validity 
I think that state matching is the only correct strategy for incremental
node reuse that is compatible with the new error recovery algorithm. It's
also simpler than the sentential-form algorithm. With the compressed parse
tables, state matching shouldn't be too conservative of a test.
 2016-07-31 21:31:19 -07:00
Max Brunsfeld
c3a242740b Allow lookahead to be broken down further after performing reductions 2016-07-04 12:20:23 -07:00
Max Brunsfeld
773e50f26b Update error recovery specs to reflect slightly different recoveries 2016-06-18 20:46:16 -07:00
Max Brunsfeld
1e353381ff Don't create error node in lexer unless token is completely invalid 
Before, any syntax error would cause the lexer to create an error
leaf node. This could happen even with a valid input, if the parse
stack had split and one particular version of the parse stack
failed to parse.

Now, an error leaf node is only created when the lexer cannot understand
part of the input stream at all. When a normal syntax error occurs,
the lexer just returns a token that is outside of the expected token
set, and the parser handles the unexpected token.
 2016-05-26 14:15:10 -07:00
Max Brunsfeld
22c550c9d6 Discard tokens after error detection to find the best repair 
* Use GLR stack-splitting to try all numbers of tokens to
  discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
  rather than the statically-computed 'in-progress symbols' list
 2016-05-11 13:49:43 -07:00
Max Brunsfeld
0d19f157ed Adjust some spec assertions to reflect finer-grained error recoveries 2016-04-22 10:19:44 -07:00
Max Brunsfeld
240355b04c Make test for allocation failure handling fail more gracefully 2016-03-10 11:36:26 -08:00
Max Brunsfeld
2e35587161 Use new stack_pop_until function for repairing errors 2016-03-07 20:06:46 -08:00
Max Brunsfeld
bc8df9f5c5 Avoid recompiling test languages when possible 2016-03-03 12:05:04 -08:00
Max Brunsfeld
aef7582a2a Start using the forward move to recover from errors 
Some unit tests passing. Corpus tests still failing
 2016-03-02 21:08:42 -08:00
Max Brunsfeld
e7abfdd373 Prevent string assertion failures from creating later memory leak errors 2016-03-02 20:58:39 -08:00
Max Brunsfeld
c96c4a08e6 Use an object pool for stack nodes, to reduce allocations 
Also, fix some leaks in the case where memory allocation failed during parsing
 2016-02-04 11:19:42 -08:00
Max Brunsfeld
4a7312e514 Fix memory leaks when editing 2016-02-02 12:22:29 -08:00
Max Brunsfeld
39417e55df Test successful parsing after handling allocation failures 2016-01-19 19:52:07 -08:00
Max Brunsfeld
3dde0a6f39 Handle allocation failures during parsing 2016-01-19 18:08:01 -08:00
Max Brunsfeld
1543a6c7b0 Handle allocation failures when instantiating documents 2016-01-18 11:32:10 -08:00
Max Brunsfeld
0f7dbea9a3 Unify test targets, use externally defined languages as fixtures 2016-01-15 11:19:24 -08:00
Max Brunsfeld
1c6ad5f7e4 Rename ubiquitous_tokens -> extra_tokens in compiler API 
They were already called this in the runtime code.
'Extra' is just easier to say.
 2015-12-17 15:50:50 -08:00
Max Brunsfeld
b3a6de6dad Replace node pos/size functions with start/end char/byte functions 2015-12-03 22:59:27 -08:00
Max Brunsfeld
863cabc827 Don't include trailing ubiquitous tokens as children when reducing 2015-12-02 15:31:15 -08:00
Max Brunsfeld
010a606b76 Add undo method to SpyInput 2015-09-19 14:46:14 -07:00
Max Brunsfeld
f37f73f92f Add ability to edit multiple times between parses 2015-09-18 23:20:06 -07:00
Max Brunsfeld
b3d883e128 Store edits in trees, not by splitting stack 
This allows for multiple edits per parse, though it is not exposed through
the API yet
 2015-09-18 22:02:06 -07:00
Max Brunsfeld
66ad271c82 Add Document parse_count() method, for tracking if trees are still valid 2015-09-10 14:23:42 -07:00
Max Brunsfeld
7ee5eaa16a Rename node accessor methods 
Instead of child() vs concrete_child(), next_sibling() vs next_concrete_sibling(), etc,
the default is switched: child() refers to the concrete syntax tree, and named_child()
refers to the AST. Because the AST is abstract through exclusion of some nodes, the
names are clearer if the qualifier goes on the AST operations
 2015-09-08 23:16:24 -07:00
Max Brunsfeld
53926c467e Don't automatically hide singleton nodes 2015-09-02 16:36:29 -07:00
Max Brunsfeld
acf9280eda Make expression and statement rules hidden in javascript grammar 2015-09-02 13:05:54 -07:00
Max Brunsfeld
21258e6a9e Remove 'document' wrapper node 2015-08-22 10:48:34 -07:00
Max Brunsfeld
e89ec7c85e Whitespace 2015-07-31 16:29:01 -07:00
Max Brunsfeld
54e40b8146 Rework AST access API: reduce heap allocation 2015-07-31 15:47:48 -07:00
Max Brunsfeld
32e79700bf Rename SpyReader -> SpyInput 2015-07-16 17:32:19 -07:00
Max Brunsfeld
3d0890eecf Preserve tokens within errors 2015-06-15 15:26:06 -07:00
Max Brunsfeld
2d436cf141 Identify fragile reductions at compile time 2015-02-21 15:11:03 -08:00
Max Brunsfeld
647c3e2010 Add tests for editing all language examples 
This drove out several fixes to the incremental parsing algorithm.
There are a few examples which still don't work and which have
been explicitly excluded, for now.
 2014-10-22 20:10:08 -07:00
Max Brunsfeld
91cf35b72c Rework javascript fixture grammar 2014-10-21 08:49:16 -07:00
Max Brunsfeld
c26cee5d18 Remove debugger in parser spec 2014-10-17 23:47:20 -07:00
Max Brunsfeld
0962b21dd3 Fix parse errors with tree-reuse 
- Don't reuse error nodes
- When re-using nodes, update lexer's token_end_position
 2014-10-17 23:20:36 -07:00
Max Brunsfeld
22ee68e1a9 Make node for each var assignment in JS grammar 2014-10-15 15:04:57 -07:00
Max Brunsfeld
d3137c6ac6 Organize parser spec 2014-10-14 23:23:12 -07:00
Max Brunsfeld
fb38140317 Discard portion of right subtree that is within the edited region 2014-10-12 11:51:12 -07:00
Max Brunsfeld
f460b921e2 Fix off-by-one error in storing reusable right-subtree 2014-10-10 12:10:23 -07:00