Max Brunsfeld
b833942bb8
Clean up Document spec
2016-12-21 11:42:32 -08:00
Max Brunsfeld
83c4c03a70
Remove unnecessary describes in Parser spec
2016-12-21 11:38:25 -08:00
Max Brunsfeld
75bcfed247
Rename doc variable -> document in Parser spec
2016-12-21 11:32:27 -08:00
Max Brunsfeld
4fd7b0e55d
Remove nested before_each's in Parser spec
2016-12-21 11:31:28 -08:00
Max Brunsfeld
42c41c158c
Refactor logic for handling shared internal/external tokens
2016-12-21 10:49:55 -08:00
Max Brunsfeld
006e9fe4a6
Specify c language explicitly when compiling test parsers
2016-12-20 17:28:28 -08:00
Max Brunsfeld
e6c82ead2c
Start work toward maintaining external scanner's state during incremental parses
2016-12-20 17:06:20 -08:00
Max Brunsfeld
2b3da512a4
Add serialize, deserialize and reset callbacks to external scanners
...
Signed-off-by: Nathan Sobo <nathan@github.com>
2016-12-20 13:12:01 -08:00
Max Brunsfeld
727727623a
Start work on unit test that edits python code
...
Signed-off-by: Nathan Sobo <nathan@github.com>
2016-12-20 13:10:18 -08:00
Max Brunsfeld
80b7affb74
Compile & link fixture grammars' external scanners in test suite
2016-12-20 13:07:45 -08:00
Max Brunsfeld
a1770ce844
Allow external tokens to be used as extras
2016-12-12 22:06:01 -08:00
Max Brunsfeld
0e595346be
Make lexer log output easier to read
2016-12-09 13:33:37 -08:00
Max Brunsfeld
10b51a05a1
Allow external scanners to refer to (and return) internally-defined tokens
...
Tokens that are defined in the grammar's rules may now be included in the
externals list also, so that external scanners can check if they are valid
lookaheads or not, and if so, can return them to the parser if needed.
2016-12-09 13:32:58 -08:00
Max Brunsfeld
83514293b5
Allow external tokens to be either visible or hidden
2016-12-05 17:26:11 -08:00
Max Brunsfeld
a09409900f
Silence missing intializer warnings in compiler unit tests
2016-12-05 16:37:06 -08:00
Max Brunsfeld
6073d9c0e8
Restore all languages in corpus specs
2016-12-05 16:36:50 -08:00
Max Brunsfeld
c4fe8ded95
Remove state argument to Lexer advance method
2016-12-05 16:36:34 -08:00
Max Brunsfeld
49d25bd0f8
Remove EXTERNAL_TOKEN grammar rule type
2016-12-04 15:02:32 -08:00
Max Brunsfeld
d72b49316b
Handle external tokens in apply_transitive_closure
2016-12-04 10:40:32 -08:00
Max Brunsfeld
0f8e130687
Call external scanner functions when lexing
2016-12-02 22:03:48 -08:00
Max Brunsfeld
c966af0412
Start work on external tokens
2016-12-02 16:24:19 -08:00
Max Brunsfeld
46854cc274
Compile and link test grammars in one step
2016-12-02 16:23:42 -08:00
Max Brunsfeld
be9e79db1b
Avoid incorrect application of precedence
2016-12-01 10:24:06 -08:00
Max Brunsfeld
996ca91e70
Disallow syntax rules that match the empty string (for now)
2016-11-30 23:19:54 -08:00
Max Brunsfeld
d627042fa6
Fix javascript error test
...
A single line with two function declarations now parses
successfully, so to create the desired error recovery
scenario, wrap the two functions in an assignment
2016-11-30 23:19:34 -08:00
Max Brunsfeld
06215607d1
Precompute transitive closure contributions by grammar symbol
2016-11-20 11:49:55 -08:00
Max Brunsfeld
6cf4ccb840
Represent rule metadata as a struct, not a map
2016-11-19 13:59:34 -08:00
Max Brunsfeld
cab1bd3ac5
Make conflict messages explicit about precedence combinations
2016-11-18 17:05:16 -08:00
Max Brunsfeld
42ba70a5a0
🎨
2016-11-18 16:16:51 -08:00
Max Brunsfeld
a39b5109c9
Ensure out/tmp directory exists in specs
2016-11-18 14:49:56 -08:00
Max Brunsfeld
c0fb0c12a8
Don't use boost regex lib on travis
2016-11-18 14:46:53 -08:00
Max Brunsfeld
60593d13f4
Use a newer gcc on travis
2016-11-18 14:41:02 -08:00
Max Brunsfeld
32387400c6
Rework LR conflict resolution
...
* Unify precedence/associativity-based resolution with the
search for a whitelisted conflict
* Improve conflict error messages
2016-11-18 13:50:55 -08:00
Max Brunsfeld
e7217f1bac
Clean up some methods in parser.c
2016-11-14 17:25:55 -08:00
Max Brunsfeld
535879a2bd
Represent byte, char and tree counts as 32 bit numbers
...
The parser spends the majority of its time allocating and freeing trees and stack nodes.
Also, the memory footprint of the AST is a significant concern when using tree-sitter
with large files. This library is already unlikely to work very well with source files
larger than 4GB, so representing rows, columns, byte lengths and child indices as
unsigned 32 bit integers seems like the right choice.
2016-11-14 12:19:13 -08:00
Max Brunsfeld
1118a9142a
Introduce Symbol::Index type alias
2016-11-14 10:25:26 -08:00
Max Brunsfeld
fad7294ba4
Store shift states for non-terminals directly in the main parse table
2016-11-14 08:36:06 -08:00
Max Brunsfeld
8d9c261e3a
Don't include reduce actions for nonterminal lookaheads
2016-11-10 11:33:37 -08:00
Max Brunsfeld
c9dcb29c6f
Remove the TS prefix from some internal type/function names
2016-11-09 20:59:05 -08:00
Max Brunsfeld
7bcae8f6a8
🎨 flatten_grammar
2016-11-09 20:29:21 -08:00
Max Brunsfeld
4106ecda43
Remove logic for recovering from OOM
2016-11-04 09:18:38 -07:00
Timothy Clem
693c6d40dd
Move setup of mergeable_symbols to constructor, use set throughout
2016-10-18 15:18:33 -07:00
Max Brunsfeld
eed54d95e1
Merge branch 'master' into changed-ranges
2016-10-16 21:10:25 -07:00
Max Brunsfeld
1412419434
Verify changed ranges for random deletions as well as random insertions
2016-10-16 14:55:51 -07:00
Max Brunsfeld
4bad58b407
Remove outdated test
2016-10-16 14:54:59 -07:00
Max Brunsfeld
b3140b2689
Implement ts_document_parse_and_get_changed_ranges
2016-10-15 22:31:21 -07:00
Max Brunsfeld
6bde4c54d9
Merge pull request #40 from tree-sitter/remove-runtime-dep-from-parser-header
...
Remove generated parsers' dependency on runtime.h
2016-10-05 14:13:53 -07:00
Max Brunsfeld
e149d94ff5
Remove generated parsers' dependency on runtime.h
2016-10-05 14:02:49 -07:00
Timothy Clem
0ffebc3d0c
s/TSDebugger/TSLogger in spec name
2016-10-05 08:48:12 -07:00
Max Brunsfeld
3014101104
Fix inconsistencies in nodes sizes after edits
2016-09-19 13:35:08 -07:00
