traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1147 commits 16 branches 98 tags 37 MiB
Commit graph

73 commits

Author SHA1 Message Date
Max Brunsfeld
22c550c9d6 Discard tokens after error detection to find the best repair 
* Use GLR stack-splitting to try all numbers of tokens to
  discard until a repair is found.
* Check the validity of repairs by looking at the child trees,
  rather than the statically-computed 'in-progress symbols' list
 2016-05-11 13:49:43 -07:00
Max Brunsfeld
0d19f157ed Adjust some spec assertions to reflect finer-grained error recoveries 2016-04-22 10:19:44 -07:00
Max Brunsfeld
240355b04c Make test for allocation failure handling fail more gracefully 2016-03-10 11:36:26 -08:00
Max Brunsfeld
2e35587161 Use new stack_pop_until function for repairing errors 2016-03-07 20:06:46 -08:00
Max Brunsfeld
bc8df9f5c5 Avoid recompiling test languages when possible 2016-03-03 12:05:04 -08:00
Max Brunsfeld
aef7582a2a Start using the forward move to recover from errors 
Some unit tests passing. Corpus tests still failing
 2016-03-02 21:08:42 -08:00
Max Brunsfeld
e7abfdd373 Prevent string assertion failures from creating later memory leak errors 2016-03-02 20:58:39 -08:00
Max Brunsfeld
c96c4a08e6 Use an object pool for stack nodes, to reduce allocations 
Also, fix some leaks in the case where memory allocation failed during parsing
 2016-02-04 11:19:42 -08:00
Max Brunsfeld
4a7312e514 Fix memory leaks when editing 2016-02-02 12:22:29 -08:00
Max Brunsfeld
39417e55df Test successful parsing after handling allocation failures 2016-01-19 19:52:07 -08:00
Max Brunsfeld
3dde0a6f39 Handle allocation failures during parsing 2016-01-19 18:08:01 -08:00
Max Brunsfeld
1543a6c7b0 Handle allocation failures when instantiating documents 2016-01-18 11:32:10 -08:00
Max Brunsfeld
0f7dbea9a3 Unify test targets, use externally defined languages as fixtures 2016-01-15 11:19:24 -08:00
Max Brunsfeld
1c6ad5f7e4 Rename ubiquitous_tokens -> extra_tokens in compiler API 
They were already called this in the runtime code.
'Extra' is just easier to say.
 2015-12-17 15:50:50 -08:00
Max Brunsfeld
b3a6de6dad Replace node pos/size functions with start/end char/byte functions 2015-12-03 22:59:27 -08:00
Max Brunsfeld
863cabc827 Don't include trailing ubiquitous tokens as children when reducing 2015-12-02 15:31:15 -08:00
Max Brunsfeld
010a606b76 Add undo method to SpyInput 2015-09-19 14:46:14 -07:00
Max Brunsfeld
f37f73f92f Add ability to edit multiple times between parses 2015-09-18 23:20:06 -07:00
Max Brunsfeld
b3d883e128 Store edits in trees, not by splitting stack 
This allows for multiple edits per parse, though it is not exposed through
the API yet
 2015-09-18 22:02:06 -07:00
Max Brunsfeld
66ad271c82 Add Document parse_count() method, for tracking if trees are still valid 2015-09-10 14:23:42 -07:00
Max Brunsfeld
7ee5eaa16a Rename node accessor methods 
Instead of child() vs concrete_child(), next_sibling() vs next_concrete_sibling(), etc,
the default is switched: child() refers to the concrete syntax tree, and named_child()
refers to the AST. Because the AST is abstract through exclusion of some nodes, the
names are clearer if the qualifier goes on the AST operations
 2015-09-08 23:16:24 -07:00
Max Brunsfeld
53926c467e Don't automatically hide singleton nodes 2015-09-02 16:36:29 -07:00
Max Brunsfeld
acf9280eda Make expression and statement rules hidden in javascript grammar 2015-09-02 13:05:54 -07:00
Max Brunsfeld
21258e6a9e Remove 'document' wrapper node 2015-08-22 10:48:34 -07:00
Max Brunsfeld
e89ec7c85e Whitespace 2015-07-31 16:29:01 -07:00
Max Brunsfeld
54e40b8146 Rework AST access API: reduce heap allocation 2015-07-31 15:47:48 -07:00
Max Brunsfeld
32e79700bf Rename SpyReader -> SpyInput 2015-07-16 17:32:19 -07:00
Max Brunsfeld
3d0890eecf Preserve tokens within errors 2015-06-15 15:26:06 -07:00
Max Brunsfeld
2d436cf141 Identify fragile reductions at compile time 2015-02-21 15:11:03 -08:00
Max Brunsfeld
647c3e2010 Add tests for editing all language examples 
This drove out several fixes to the incremental parsing algorithm.
There are a few examples which still don't work and which have
been explicitly excluded, for now.
 2014-10-22 20:10:08 -07:00
Max Brunsfeld
91cf35b72c Rework javascript fixture grammar 2014-10-21 08:49:16 -07:00
Max Brunsfeld
c26cee5d18 Remove debugger in parser spec 2014-10-17 23:47:20 -07:00
Max Brunsfeld
0962b21dd3 Fix parse errors with tree-reuse 
- Don't reuse error nodes
- When re-using nodes, update lexer's token_end_position
 2014-10-17 23:20:36 -07:00
Max Brunsfeld
22ee68e1a9 Make node for each var assignment in JS grammar 2014-10-15 15:04:57 -07:00
Max Brunsfeld
d3137c6ac6 Organize parser spec 2014-10-14 23:23:12 -07:00
Max Brunsfeld
fb38140317 Discard portion of right subtree that is within the edited region 2014-10-12 11:51:12 -07:00
Max Brunsfeld
f460b921e2 Fix off-by-one error in storing reusable right-subtree 2014-10-10 12:10:23 -07:00
Max Brunsfeld
4dcc712a8c Start work on re-using right side of parse tree 2014-10-09 19:58:15 -07:00
Max Brunsfeld
af7f57a80e Fix sizing of error nodes after edits 2014-10-05 16:56:50 -07:00
Max Brunsfeld
808b003f1a Read unicode characters correctly in Lexer advance 2014-10-03 15:44:49 -07:00
Max Brunsfeld
a69dfa08f3 Add spec for inserting text w/ unicode characters 2014-10-02 11:54:00 -07:00
Max Brunsfeld
8bee9d8fb9 Fix typo in parser spec descriptions 2014-10-02 11:52:58 -07:00
Max Brunsfeld
5f313896c3 Make ::input a method on SpyReader, not a field 2014-09-30 14:57:57 -07:00
Max Brunsfeld
8d7d9af661 Remove unnecessary helper function in parser spec 2014-09-29 10:51:12 -07:00
Max Brunsfeld
700919951e Reorganize parser spec about handling edits 2014-09-29 10:48:35 -07:00
Max Brunsfeld
7988829c08 Add spec for recognition of UTF8 characters 2014-09-27 16:00:48 -07:00
Max Brunsfeld
c1565c1aae Track AST nodes' sizes in characters as well as bytes 
The `pos` and `size` functions for Nodes now return TSLength structs,
which contain lengths in both characters and bytes. This is important
for knowing the number of unicode characters in a Node.
 2014-09-26 16:15:07 -07:00
Max Brunsfeld
68d6e242ee Fix parsing of wildcard patterns at the ends of documents 
- Remove special EOF handling from lexer
- Explicitly exclude the EOF character from all-inclusive character sets.
 2014-09-11 13:10:23 -07:00
Max Brunsfeld
f05762b4a0 Move parser tests into their own file 2014-09-10 18:49:53 -07:00
Max Brunsfeld
9302080aa6 Remove unit test on parser 
It wasn't adding any value
 2014-08-09 00:00:20 -07:00