Max Brunsfeld
e479773aac
Rename files to avoid duplicate basenames
2014-03-02 15:07:43 -08:00
Max Brunsfeld
72a0c0e09c
Keep track of AST node sizes and positions
2014-03-01 15:44:25 -08:00
Max Brunsfeld
5045d5cccc
Clean up tree spec
2014-03-01 15:09:41 -08:00
Max Brunsfeld
ded54a3a1a
Fix some memory management bugs with trees
2014-03-01 00:25:05 -08:00
Max Brunsfeld
21c0f51b84
Rename ts_document_{set_text,set_input_string}
...
Because next I'll add a more general 'set_input' method
2014-02-27 13:30:11 -08:00
Max Brunsfeld
c43ec90dad
Fix missing auxiliary tokens in lex error state
2014-02-27 00:38:08 -08:00
Max Brunsfeld
8b7ed13c8b
Add more error recovery tests for JSON
2014-02-26 22:58:45 -08:00
Max Brunsfeld
8deeff392b
Fix memory leak in tree
2014-02-26 19:54:22 -08:00
Max Brunsfeld
63718d5bad
Add basic support for error recovery
2014-02-26 19:03:43 -08:00
Max Brunsfeld
df05c75525
Backfill unit tests for follow sets
...
Also, add some comment docs for some compiler functions
2014-02-26 00:39:31 -08:00
Max Brunsfeld
df223d566e
Fix bug in parser error handling
2014-02-25 18:33:24 -08:00
Max Brunsfeld
e58a6d8ba7
Start work on error recovery
...
- In runtime, make parse errors part of the parse tree
- Add error state to lexers in which they can accept any token
2014-02-24 18:42:54 -08:00
Max Brunsfeld
713b3899c5
Simplify logic for extracting tokens from grammar
2014-02-23 10:00:49 -08:00
Max Brunsfeld
688fe993cf
Remove rules that don't need to be public from compiler.h
2014-02-20 23:05:32 -08:00
Max Brunsfeld
ceee7eefd0
Don't store text on documents
2014-02-20 18:38:31 -08:00
Max Brunsfeld
6ed6aa05cc
Update runtime code naming
...
Also, add macros so that code generator doesn't
need to know about any runtime variables names
or types
2014-02-20 13:30:43 -08:00
Max Brunsfeld
e87380a8b7
Make separate PreparedGrammar subclass of Grammar for internal use
2014-02-19 13:36:38 -08:00
Max Brunsfeld
acad97cfd2
Separate concepts of hidden and auxiliary symbols
...
This way, we can hide auxiliary symbols from library users,
but still allow them to use hidden symbols
2014-02-19 13:05:54 -08:00
Max Brunsfeld
49ad910474
Skip whitespace by default in lexer
2014-02-19 12:48:38 -08:00
Max Brunsfeld
2c56612650
Get makefile working
2014-02-18 09:07:00 -08:00
Max Brunsfeld
fd757c20d0
Move example grammars from fixtures -> examples dir
2014-02-17 13:18:09 -08:00
Max Brunsfeld
585f9f0054
Fix misplaced file
2014-02-17 12:59:24 -08:00
Max Brunsfeld
d09fa910ef
Put public headers in tree_sitter directory
2014-02-17 12:53:57 -08:00
Max Brunsfeld
84b8addb63
Organize spec helper files
2014-02-17 12:45:53 -08:00
Max Brunsfeld
9e2dc14182
Extract public compiler API into its own header file
2014-02-17 12:40:46 -08:00
Max Brunsfeld
0b4e1c8d0d
Remove unused import in generated parsers
2014-02-15 22:26:41 -08:00
Max Brunsfeld
dbbb446082
Reorganize runtime header files
2014-02-15 17:00:33 -08:00
Max Brunsfeld
5c1a0982df
Change repeat to mean zero-or-more
2014-02-15 15:43:32 -08:00
Max Brunsfeld
bf07522026
Fix bug in FIRST set function
2014-02-13 18:52:17 -08:00
Max Brunsfeld
2ee9455174
Give tokens descriptive names in json grammar
2014-02-13 13:22:47 -08:00
Max Brunsfeld
a77ca1ee08
Allow users to put their own auxiliary rules in grammars
2014-02-13 13:09:00 -08:00
Max Brunsfeld
7490a7ac94
Change repeat to mean zero-or-more in lexical rules
...
Still need to make it work this way in syntactic rules
2014-02-12 18:35:29 -08:00
Max Brunsfeld
70173512f1
Don't extract isolated blanks from grammars into tokens
2014-02-12 18:31:57 -08:00
Max Brunsfeld
9a737e2316
Use map, not unordered_map in Grammar
2014-02-12 08:30:50 -08:00
Max Brunsfeld
27f305d556
Put rule_can_be_blank function in its own file
2014-02-11 13:21:45 -08:00
Max Brunsfeld
1962c17f45
Remove transition_map class
2014-02-11 13:15:44 -08:00
Max Brunsfeld
66f7dcf28a
Remove some unused imports
2014-02-10 21:09:43 -08:00
Max Brunsfeld
ab431c24dc
Update json spec
2014-02-10 21:04:19 -08:00
Max Brunsfeld
3cb65c9c81
Make the EOF be an auxiliary symbol
...
This way, it couldn’t conflict if a user had a rule called “__END__”
2014-02-10 18:53:01 -08:00
Max Brunsfeld
15c9e2d398
Make ordering of cases deterministic in generated parsers
2014-02-10 18:38:01 -08:00
Max Brunsfeld
8baa1396fd
Properly merge transitions on overlapping character sets!
2014-02-10 13:20:43 -08:00
Max Brunsfeld
905a408998
Make separate functions for character-level and syntax-level rule transitions
2014-02-09 14:31:27 -08:00
Max Brunsfeld
df3397f02c
Implement character set difference
2014-02-07 12:57:35 -08:00
Max Brunsfeld
e8337a3c70
Fix errors in computing character set unions
2014-02-06 09:12:03 -08:00
Max Brunsfeld
d3d25f2683
Represent character sets as sets of character ranges
2014-02-05 18:56:04 -08:00
Max Brunsfeld
8cce11a52a
Rename Character -> CharacterSet, CharacterMatch -> CharacterRange
2014-02-03 13:05:51 -08:00
Max Brunsfeld
716a4a4259
Store character rules’ matches in a set, not a vector
2014-01-31 18:56:43 -08:00
Max Brunsfeld
5ed5ae7514
In LR(1) items, only store consumed symbols as booleans
...
the booleans represent the symbols point to auxiliary tokens
or not. This is all we need to know for the purpose of building
parse tables. Any other information just leads to redundant
parse states.
2014-01-31 00:13:05 -08:00
Max Brunsfeld
0d3a941848
Add support for character ranges in regex patterns
2014-01-30 18:54:39 -08:00
Max Brunsfeld
60e2d00b4d
Parse simple character sets in pattern rules
2014-01-30 13:04:31 -08:00
