Max Brunsfeld
5320cad065
Trim trailing whitespace
2014-04-04 13:10:55 -07:00
Max Brunsfeld
1cc7e32e2d
Fix handling of tokens consisting of separator characters
...
The parser is no longer hard-coded to skip whitespace. Tokens
such as newlines, whose characters overlap with the separator
characters, can now be correctly recognized.
2014-04-03 19:10:09 -07:00
Max Brunsfeld
13c4e6e648
Tweak format for example grammars
2014-03-28 13:51:32 -07:00
Max Brunsfeld
4454925b5a
Clean up parser macros more
2014-03-26 13:03:12 -07:00
Max Brunsfeld
f601322956
Clean up macros in parser.h
2014-03-25 19:51:34 -07:00
Max Brunsfeld
80b19cbb83
Construct entire parse table statically
...
This removes the need for the 'init_parse_table' function,
which was not really thread safe
2014-03-25 19:34:17 -07:00
Max Brunsfeld
25861b7f03
Remove reduction-specific collapse flags in favor of globally hidden symbols
2014-03-25 09:05:55 -07:00
Max Brunsfeld
3a7c4bb5b1
Store AST nodes' non-hidden children
2014-03-24 01:03:32 -07:00
Max Brunsfeld
95188d84b6
Make tree struct private
2014-03-24 00:34:13 -07:00
Max Brunsfeld
c40411b4d1
Handle unexpected tokens properly
...
Also, add null and boolean values to json grammar
2014-03-22 15:46:58 -07:00
Karen Wang and Max Brunsfeld
f248ece3aa
Use table-driven tests for specific parsers
2014-03-22 15:19:13 -07:00
Max Brunsfeld
5869c1ea18
Clean up stack breakdown function
2014-03-21 13:02:25 -07:00
Max Brunsfeld
bdbe69258a
Fix bug in stack breakdown procedure
2014-03-21 12:46:23 -07:00
Max Brunsfeld
cbc66b359e
Cleanup
2014-03-19 23:10:28 -07:00
Max Brunsfeld
fbe8b0a905
Fix incremental parsing
...
Stop collapsing hidden symbols upon reducing them.
Sadly, this messes up the ability to re-use parse
trees. Instead, for now, hide these nodes when
stringifying parse trees
2014-03-19 19:27:31 -07:00
Max Brunsfeld
7e94a4f1b2
Start work on reading input incrementally
2014-03-18 13:23:21 -07:00
Max Brunsfeld
8e7f59db1c
Allow room in parse table for error and EOF entries
2014-03-18 08:04:17 -07:00
Max Brunsfeld
67b33a615b
Refactor generated parsers to used explicit table
...
This is slightly slower than encoding the parse table in
flow control, but allows the parser to inspect the parse
table more flexibly. This is needed for incremental parsing.
2014-03-17 18:43:17 -07:00
Max Brunsfeld
0d6435e24a
Pass edit information into parser function
2014-03-15 16:55:35 -07:00
Max Brunsfeld
464a0e8dad
Clean up parser header further
2014-03-15 15:15:59 -07:00
Max Brunsfeld
0dc3a95d0c
Refactor parser header
...
Make separate lexer, stack and parser structs.
2014-03-15 14:43:50 -07:00
Max Brunsfeld
ff0c8a98b8
Consolidate reading of input chunks in parser
2014-03-11 12:43:13 -07:00
Max Brunsfeld
42e9a264f3
In parser, read chunked input correctly
2014-03-10 13:25:31 -07:00
Max Brunsfeld
31a58bc7e4
Make include guards pass cpplint
2014-03-09 22:05:24 -07:00
Max Brunsfeld
9c6c47e02f
Make parser input reads indicate the # of bytes read
2014-03-08 16:51:08 -08:00
Max Brunsfeld
b167ee84fa
Fix integer sign warning in parser.h
2014-03-08 15:04:23 -08:00
Max Brunsfeld
0f4b6332cb
Refactor document to allow other input data structures
2014-03-01 22:43:25 -08:00
Max Brunsfeld
72a0c0e09c
Keep track of AST node sizes and positions
2014-03-01 15:44:25 -08:00
Max Brunsfeld
ded54a3a1a
Fix some memory management bugs with trees
2014-03-01 00:25:05 -08:00
Max Brunsfeld
926b4ef242
Clean up parser.h
2014-02-27 00:06:22 -08:00
Max Brunsfeld
0d4687bdad
Make start and EOF symbols into builtin symbols
2014-02-26 23:18:49 -08:00
Max Brunsfeld
63718d5bad
Add basic support for error recovery
2014-02-26 19:03:43 -08:00
Max Brunsfeld
df223d566e
Fix bug in parser error handling
2014-02-25 18:33:24 -08:00
Max Brunsfeld
4541332c2b
Put definition of EOF symbol back in parse table builder (not in parser.h)
2014-02-25 13:25:58 -08:00
Max Brunsfeld
e58a6d8ba7
Start work on error recovery
...
- In runtime, make parse errors part of the parse tree
- Add error state to lexers in which they can accept any token
2014-02-24 18:42:54 -08:00
Max Brunsfeld
a34da59889
Update naming style in parser.h
2014-02-20 18:43:35 -08:00
Max Brunsfeld
ceee7eefd0
Don't store text on documents
2014-02-20 18:38:31 -08:00
Max Brunsfeld
6ed6aa05cc
Update runtime code naming
...
Also, add macros so that code generator doesn't
need to know about any runtime variables names
or types
2014-02-20 13:30:43 -08:00
Max Brunsfeld
49ad910474
Skip whitespace by default in lexer
2014-02-19 12:48:38 -08:00
Max Brunsfeld
d09fa910ef
Put public headers in tree_sitter directory
2014-02-17 12:53:57 -08:00