Max Brunsfeld
647c3e2010
Add tests for editing all language examples
...
This drove out several fixes to the incremental parsing algorithm.
There are a few examples which still don't work and which have
been explicitly excluded, for now.
2014-10-22 20:10:08 -07:00
Max Brunsfeld
de9a48d11f
Tweak debugging in parser and lexer
2014-10-22 20:10:08 -07:00
Max Brunsfeld
d8099826f9
Workaround duplicate symbol problem in golang grammar
...
Currently, a grammar can't have a string rule and a keyword rule with
the same string.
2014-10-21 13:03:50 -07:00
Max Brunsfeld
8a03bd8978
Comment out some parts of JS grammar to speed up tests
2014-10-21 09:01:56 -07:00
Max Brunsfeld
eaeb51a96f
Don't open parser C file until code is generated
2014-10-21 08:49:42 -07:00
Max Brunsfeld
91cf35b72c
Rework javascript fixture grammar
2014-10-21 08:49:16 -07:00
Max Brunsfeld
8134b64d00
Remove repetition from language spec descriptions
2014-10-19 12:36:43 -07:00
Max Brunsfeld
c26cee5d18
Remove debugger in parser spec
2014-10-17 23:47:20 -07:00
Max Brunsfeld
0962b21dd3
Fix parse errors with tree-reuse
...
- Don't reuse error nodes
- When re-using nodes, update lexer's token_end_position
2014-10-17 23:20:36 -07:00
Max Brunsfeld
ae0a7fc97d
Add logging debugger for debugging failing tests
2014-10-17 23:05:08 -07:00
Max Brunsfeld
41a067fef9
Fix build warnings in document spec
2014-10-17 21:27:49 -07:00
Max Brunsfeld
8cf800ef5d
Unify debugging API for parsing and lexing
2014-10-17 17:52:54 -07:00
Max Brunsfeld
22ee68e1a9
Make node for each var assignment in JS grammar
2014-10-15 15:04:57 -07:00
Max Brunsfeld
d3137c6ac6
Organize parser spec
2014-10-14 23:23:12 -07:00
Max Brunsfeld
d33b074c30
Don't call input::seek_fn unnecessarily
2014-10-14 22:56:42 -07:00
Max Brunsfeld
b5d022a70c
Fix missing field warnings for debugger structs
2014-10-14 22:50:24 -07:00
Max Brunsfeld
c594208ab8
Allow callbacks to be specified for debug output
2014-10-13 01:02:18 -07:00
Max Brunsfeld
fb38140317
Discard portion of right subtree that is within the edited region
2014-10-12 11:51:12 -07:00
Max Brunsfeld
8c3ea6ef30
In JS grammar, don't include enclosing parens in formal_parameters node
2014-10-10 17:11:14 -07:00
Max Brunsfeld
f460b921e2
Fix off-by-one error in storing reusable right-subtree
2014-10-10 12:10:23 -07:00
Max Brunsfeld
3bcb221379
Include non-terminal lookahead symbols for reduction actions
...
This is necessary for re-using the right subtree after an edit
2014-10-10 12:06:16 -07:00
Max Brunsfeld
4dcc712a8c
Start work on re-using right side of parse tree
2014-10-09 19:58:15 -07:00
Max Brunsfeld
af7f57a80e
Fix sizing of error nodes after edits
2014-10-05 16:56:50 -07:00
Max Brunsfeld
e5ea4efb0b
Use stdbool.h
2014-10-03 16:06:08 -07:00
Max Brunsfeld
808b003f1a
Read unicode characters correctly in Lexer advance
2014-10-03 15:44:49 -07:00
Max Brunsfeld
1fa3bf0f07
In SpyReader::read, always return complete unicode characters
2014-10-03 14:30:19 -07:00
Max Brunsfeld
17f43e5e0c
Clean up SpyReader
2014-10-03 14:21:39 -07:00
Max Brunsfeld
a69dfa08f3
Add spec for inserting text w/ unicode characters
2014-10-02 11:54:00 -07:00
Max Brunsfeld
8bee9d8fb9
Fix typo in parser spec descriptions
2014-10-02 11:52:58 -07:00
Max Brunsfeld
aab449635f
Allow greek letters as variables in arithmetic fixture grammar
2014-10-02 11:50:16 -07:00
Max Brunsfeld
0f524121f1
Add SpyReader methods for inserting/removing by char index
2014-10-02 11:43:22 -07:00
Max Brunsfeld
5f313896c3
Make ::input a method on SpyReader, not a field
2014-09-30 14:57:57 -07:00
Max Brunsfeld
8d7d9af661
Remove unnecessary helper function in parser spec
2014-09-29 10:51:12 -07:00
Max Brunsfeld
700919951e
Reorganize parser spec about handling edits
2014-09-29 10:48:35 -07:00
Max Brunsfeld
070dc76050
Generate correct C literals for non-ascii characters
2014-09-28 18:40:15 -07:00
Max Brunsfeld
cb5ecbd491
Handle string and regex rules w/ non-ascii chars
2014-09-28 18:21:22 -07:00
Max Brunsfeld
13ba35e26f
Fix indentation in fixture grammars
2014-09-27 16:10:30 -07:00
Max Brunsfeld
7988829c08
Add spec for recognition of UTF8 characters
2014-09-27 16:00:48 -07:00
Max Brunsfeld
26ac5788b6
Don't use struct literal syntax for TSLength
2014-09-26 16:31:36 -07:00
Max Brunsfeld
04dc721241
Add missing import for string.h
2014-09-26 16:21:09 -07:00
Max Brunsfeld
c1565c1aae
Track AST nodes' sizes in characters as well as bytes
...
The `pos` and `size` functions for Nodes now return TSLength structs,
which contain lengths in both characters and bytes. This is important
for knowing the number of unicode characters in a Node.
2014-09-26 16:15:07 -07:00
Max Brunsfeld
c576d7d4a0
In SpyReader, don't return pointers into main content string
...
This improves test coverage of the lexer. Before, a SpyReader's read function
would return pointers into a single string that contained the entire text. This
could have masked bugs where out-of-bounds characters were being read.
Now the chunks returned by the reader are copied into a separate buffer.
2014-09-26 16:12:52 -07:00
Max Brunsfeld
68d6e242ee
Fix parsing of wildcard patterns at the ends of documents
...
- Remove special EOF handling from lexer
- Explicitly exclude the EOF character from all-inclusive character sets.
2014-09-11 13:10:23 -07:00
Max Brunsfeld
a2b80098b2
Fix destination directory in compile_examples spec
2014-09-11 12:46:46 -07:00
Max Brunsfeld
f05762b4a0
Move parser tests into their own file
2014-09-10 18:49:53 -07:00
Max Brunsfeld
9682ef6c79
Rename examples directory to spec/fixtures
...
- I want to move away from having complete grammars for real languages
(e.g. javascript, golang) in this repo. These languages take a long
time to compile, and they now exist in their own repos
(node-tree-sitter-javascript etc).
- I want to start testing more compiler edge cases through integration
tests, so I want to put more small, weird grammars in here. That makes
me not want to call the directory `examples`.
2014-09-10 13:31:06 -07:00
Max Brunsfeld
cd8a683229
Improve error messages for invalid ubiquitous tokens
2014-09-10 13:02:16 -07:00
Max Brunsfeld
e9dad529f5
Make descriptions more consistent in compiler specs
2014-09-09 13:01:18 -07:00
Max Brunsfeld
1ff7cedf40
Unify ubiquitous tokens and lexical separators in API
2014-09-07 22:16:45 -07:00
Max Brunsfeld
a46f9d950c
Handle '\s' correctly in regexps
2014-09-07 16:05:43 -07:00
