traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1358 commits 16 branches 98 tags 37 MiB
Commit graph

52 commits

Author SHA1 Message Date
Max Brunsfeld
42c41c158c Refactor logic for handling shared internal/external tokens 2016-12-21 10:49:55 -08:00
Max Brunsfeld
006e9fe4a6 Specify c language explicitly when compiling test parsers 2016-12-20 17:28:28 -08:00
Max Brunsfeld
e6c82ead2c Start work toward maintaining external scanner's state during incremental parses 2016-12-20 17:06:20 -08:00
Max Brunsfeld
727727623a Start work on unit test that edits python code 
Signed-off-by: Nathan Sobo <nathan@github.com>
 2016-12-20 13:10:18 -08:00
Max Brunsfeld
80b7affb74 Compile & link fixture grammars' external scanners in test suite 2016-12-20 13:07:45 -08:00
Max Brunsfeld
c966af0412 Start work on external tokens 2016-12-02 16:24:19 -08:00
Max Brunsfeld
46854cc274 Compile and link test grammars in one step 2016-12-02 16:23:42 -08:00
Max Brunsfeld
06215607d1 Precompute transitive closure contributions by grammar symbol 2016-11-20 11:49:55 -08:00
Max Brunsfeld
6cf4ccb840 Represent rule metadata as a struct, not a map 2016-11-19 13:59:34 -08:00
Max Brunsfeld
a39b5109c9 Ensure out/tmp directory exists in specs 2016-11-18 14:49:56 -08:00
Max Brunsfeld
c0fb0c12a8 Don't use boost regex lib on travis 2016-11-18 14:46:53 -08:00
Max Brunsfeld
32387400c6 Rework LR conflict resolution 
* Unify precedence/associativity-based resolution with the
  search for a whitelisted conflict
* Improve conflict error messages
 2016-11-18 13:50:55 -08:00
Max Brunsfeld
535879a2bd Represent byte, char and tree counts as 32 bit numbers 
The parser spends the majority of its time allocating and freeing trees and stack nodes.
Also, the memory footprint of the AST is a significant concern when using tree-sitter
with large files. This library is already unlikely to work very well with source files
larger than 4GB, so representing rows, columns, byte lengths and child indices as
unsigned 32 bit integers seems like the right choice.
 2016-11-14 12:19:13 -08:00
Max Brunsfeld
fad7294ba4 Store shift states for non-terminals directly in the main parse table 2016-11-14 08:36:06 -08:00
Max Brunsfeld
c9dcb29c6f Remove the TS prefix from some internal type/function names 2016-11-09 20:59:05 -08:00
Max Brunsfeld
4106ecda43 Remove logic for recovering from OOM 2016-11-04 09:18:38 -07:00
Max Brunsfeld
eed54d95e1 Merge branch 'master' into changed-ranges 2016-10-16 21:10:25 -07:00
Max Brunsfeld
b3140b2689 Implement ts_document_parse_and_get_changed_ranges 2016-10-15 22:31:21 -07:00
Max Brunsfeld
e149d94ff5 Remove generated parsers' dependency on runtime.h 2016-10-05 14:02:49 -07:00
Max Brunsfeld
00528e50ce Change edit API to be byte-based 2016-09-13 13:08:52 -07:00
Max Brunsfeld
cc62fe0375 Represent Lengths in terms of Points 2016-09-09 21:11:02 -07:00
Max Brunsfeld
eb5dda75c4 Start work on randomized changed-region testing 
Signed-off-by: Nathan Sobo <nathan@github.com>
 2016-09-09 09:20:04 -07:00
Max Brunsfeld
fce8d57152 Start work on document_parse_and_diff API 2016-09-08 17:51:20 -07:00
Max Brunsfeld
38241d466b Rename .read_fn, .seek_fn -> .read, .seek 2016-09-06 21:39:10 -07:00
Max Brunsfeld
096ac2d4b6 Rename ts_document_set_debugger -> ts_document_set_logger 2016-09-06 17:40:26 -07:00
Max Brunsfeld
de9b9db7c5 Remove accidentally-committed reference stack files 2016-08-29 13:07:46 -07:00
Max Brunsfeld
f209c24cf6 Trim trailing newlines from corpus texts 2016-08-29 13:00:28 -07:00
Max Brunsfeld
285f2272fd Move random string helpers into a separate file 2016-07-17 06:22:05 -07:00
Max Brunsfeld
08e47001f1 Silence mismatched delete warning in spec helper 2016-06-27 13:38:49 -07:00
Max Brunsfeld
38c144b4a3 Refine logic for deciding when tokens need to be re-lexed 
* While generating the lex table, note which tokens can match the
  same string. A token needs to be relexed when it has possible
  homonyms in the current state.
* Also note which tokens can match substrings of each other tokens.
  A token needs to be relexed when there are viable tokens that
  could match longer strings in the current state and the next
  token has been edited.
* Remove the logic for marking tokens as fragile on creation.
* Store the reusability/non-reusability of symbols off of individual
  actions and onto the entire entry for the state & symbol.
 2016-06-21 07:28:04 -07:00
Max Brunsfeld
70d3cde775 Remove extra leading newline from corpus spec texts 2016-06-15 10:31:34 -07:00
Max Brunsfeld
e1a3a1daeb Import error corpus entries from grammar repos 
Now that error recovery requires no input for the grammar author, it shouldn't
be tested in the individual grammar repos.
 2016-05-28 20:12:02 -07:00
Max Brunsfeld
59712ec492 Clean up lex table generation 2016-05-19 13:25:46 -07:00
Max Brunsfeld
88053cf723 In tests, don’t record allocations while printing debug graphs 2016-05-16 10:44:19 -07:00
Max Brunsfeld
fd4c33209e Select ambiguous alternatives by minimizing error size 2016-04-24 00:54:20 -07:00
Max Brunsfeld
655d374d0c Recompile test languages if parser.h changes 2016-04-18 11:17:06 -07:00
Max Brunsfeld
4f726da881 Fix logic for whether to regenerate parsers in specs 2016-03-10 11:35:59 -08:00
Max Brunsfeld
4348eb89d4 Expose lower stack nodes via pop_until() function 
This callback-based API allows the parser to easily visit each interior node
of the stack when searching for an error repair. It also is a better abstraction
over the stack's DAG implementation than having the public functions for
accessing entries and their successor entries.
 2016-03-07 16:09:34 -08:00
Max Brunsfeld
bc8df9f5c5 Avoid recompiling test languages when possible 2016-03-03 12:05:04 -08:00
Max Brunsfeld
b99c257e9e Make test symbol_names array accessible in specs 2016-02-23 00:08:20 -08:00
Max Brunsfeld
f444a715fd Clean up tree array assertions in stack spec 2016-02-22 09:23:25 -08:00
Max Brunsfeld
df1d9b2416 Move sexp-generation code to node.c 2016-02-19 15:41:09 -08:00
Max Brunsfeld
b80a330a74 Fix assorted memory leaks in test code 2016-02-05 12:23:54 -08:00
Max Brunsfeld
e3d65d5cbd Change name of generated parser files 
The previous name, parser.c made it harder to set breakpoints inside of
src/runtime/parser.c in LLDB
 2016-01-29 17:33:15 -08:00
Max Brunsfeld
6401a065ae Use different types for advance and accept-token actions 
Unlike with parse actions, lexical actions of different types never appear
in the same places in the table
 2016-01-22 22:24:11 -07:00
Max Brunsfeld
ff97a09343 Compile test grammars w/ debug symbols 2016-01-19 15:33:46 -08:00
Max Brunsfeld
1543a6c7b0 Handle allocation failures when instantiating documents 2016-01-18 11:32:10 -08:00
Max Brunsfeld
90cbd61280 Adjust static library path on linux 2016-01-15 11:40:46 -08:00
Max Brunsfeld
d6ed783c8c Improve error handling when dynamically loading languages in specs 2016-01-15 11:35:22 -08:00
Max Brunsfeld
0f7dbea9a3 Unify test targets, use externally defined languages as fixtures 2016-01-15 11:19:24 -08:00