traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
969 commits 16 branches 98 tags 37 MiB
Commit graph

303 commits

Author SHA1 Message Date
Max Brunsfeld
c495076adb Record in parse table which actions can hide splits 
Suppose a parse state S has multiple actions for a terminal lookahead symbol A.
Then during incremental parsing, while in state S, the parser should not
reuse a non-terminal lookahead B where FIRST(B) contains A, because reusing B
might prematurely discard one of the possible actions that a batch parser
would have attempted in state S, upon seeing A as a lookahead.
 2015-12-17 13:11:56 -08:00
Max Brunsfeld
7fbb628c78 Remove TreeSelectionCallback struct 
Just make a typedef for the function type
 2015-12-17 12:09:06 -08:00
Max Brunsfeld
e6f933a21f Add missing symbol metadata field initializers in specs 2015-12-17 12:06:43 -08:00
Max Brunsfeld
da2dfd703a {s,}rand -> {s,}random 
🎩 @robrix
 2015-12-10 21:23:23 -08:00
Max Brunsfeld
51998ac5bb In specs, seed random number generator with clock time by default 2015-12-10 21:01:27 -08:00
Max Brunsfeld
a3fdd7f20e Improve random word generation in random corpus specs 2015-12-10 16:47:55 -08:00
Max Brunsfeld
2c2c567a29 Improve randomized edits in corpus specs 2015-12-09 13:37:03 -08:00
Max Brunsfeld
7d066ff632 Change the random seed for the corpus specs 2015-12-08 17:32:16 -08:00
Max Brunsfeld
10286f307f Pass reference to parser in stack's tree selection callback 2015-12-08 12:21:27 -08:00
Max Brunsfeld
d69fef11f9 Break down top of parse stack when needed 
Also, start randomizing the edits in the corpus specs
 2015-12-04 20:57:33 -08:00
Max Brunsfeld
d2bf88d5fe Include rows and columns in TSLength 
This way, we don't have to have separate 1D and 2D versions for so many values
 2015-12-04 20:20:29 -08:00
Max Brunsfeld
8e217f758c Use individual args instead of TSLength in input seek function 2015-12-03 23:06:01 -08:00
Max Brunsfeld
b3a6de6dad Replace node pos/size functions with start/end char/byte functions 2015-12-03 22:59:27 -08:00
Max Brunsfeld
fe5286f863 Merge pull request #11 from maxbrunsfeld/move-ubiquitous-tokens-outward 
Put ubiquitous tokens as high in the tree as possible
 2015-12-03 10:06:50 -08:00
Max Brunsfeld
8a146a9bef Reset lexer correctly when old input was blank 2015-12-03 10:00:39 -08:00
Max Brunsfeld
863cabc827 Don't include trailing ubiquitous tokens as children when reducing 2015-12-02 15:31:15 -08:00
Max Brunsfeld
f08554e958 Replace NodeType enum with SymbolMetadata bitfield 
This will allow storing other metadata about symbols, like if they
only appear as ubiquitous tokens
 2015-12-02 15:10:24 -08:00
joshvera
883bb87e29 add asserts for points in language_specs 2015-12-02 17:29:10 -05:00
joshvera
2df2b58d3e Move point helpers elsewhere 2015-12-02 17:28:52 -05:00
joshvera
14bc05b8ac Fix some names and add more tests 2015-12-02 17:12:00 -05:00
joshvera
da8e48bf09 fix test names 2015-12-02 16:45:08 -05:00
joshvera
94a46d9ae1 Add more start_point and end_point tests 2015-12-02 16:40:56 -05:00
joshvera
cc4bd82e8a Fix failing tests using find 2015-12-02 16:06:55 -05:00
joshvera
9da4aeaeff columns start at 0 for sanity's sake 2015-11-30 17:22:47 -05:00
joshvera
f5fc247c8b Merge remote-tracking branch 'origin/master' into line-numbers 2015-11-30 12:36:11 -05:00
joshvera
3d9a44d880 Calculate the column and offset separately in TSNode 2015-11-25 13:36:19 -05:00
Max Brunsfeld
c88e9044d5 Make stack popping more robust 2015-11-20 00:04:21 -08:00
joshvera
b0f6bac3ab replace start and end with padding and size 2015-11-18 16:34:50 -08:00
Max Brunsfeld
ab34cfecd9 Replace TreeVector with a more generic Vector struct 2015-11-15 09:55:36 -08:00
joshvera
8058500c5b Add source info to TSTree 2015-11-12 15:32:53 -05:00
Max Brunsfeld
483577f00d Fix error when merging nodes at the base of the parse stack 2015-11-11 17:31:40 -08:00
Max Brunsfeld
f5d861a019 Fix bug where ts_stack_pop results were backwards for some stack configurations 2015-10-28 12:10:45 -07:00
Max Brunsfeld
36eae5d5e9 Add option to tree_string to include anonymous tokens 2015-10-28 12:09:28 -07:00
Max Brunsfeld
1829b34c6f Fix child count error when documents starts w/ ubiquitous tokens 2015-10-24 13:45:42 -07:00
Max Brunsfeld
3d0253f9b8 Start work on c++ fixture grammar 2015-10-13 11:28:53 -07:00
Max Brunsfeld
ebc52f109d Merge branch 'flatten-rules-into-productions' 
This branch had diverged considerably, so merging it required changing a lot
of code.

Conflicts:
	project.gyp
	spec/compiler/build_tables/action_takes_precedence_spec.cc
	spec/compiler/build_tables/build_conflict_spec.cc
	spec/compiler/build_tables/build_parse_table_spec.cc
	spec/compiler/build_tables/first_symbols_spec.cc
	spec/compiler/build_tables/item_set_closure_spec.cc
	spec/compiler/build_tables/item_set_transitions_spec.cc
	spec/compiler/build_tables/rule_can_be_blank_spec.cc
	spec/compiler/helpers/containers.h
	spec/compiler/prepare_grammar/expand_repeats_spec.cc
	spec/compiler/prepare_grammar/extract_tokens_spec.cc
	src/compiler/build_tables/action_takes_precedence.h
	src/compiler/build_tables/build_parse_table.cc
	src/compiler/build_tables/first_symbols.cc
	src/compiler/build_tables/first_symbols.h
	src/compiler/build_tables/item_set_closure.cc
	src/compiler/build_tables/item_set_transitions.cc
	src/compiler/build_tables/parse_item.cc
	src/compiler/build_tables/parse_item.h
	src/compiler/build_tables/rule_can_be_blank.cc
	src/compiler/build_tables/rule_can_be_blank.h
	src/compiler/prepare_grammar/expand_repeats.cc
	src/compiler/prepare_grammar/extract_tokens.cc
	src/compiler/prepare_grammar/extract_tokens.h
	src/compiler/prepare_grammar/prepare_grammar.cc
	src/compiler/rules/built_in_symbols.cc
	src/compiler/rules/built_in_symbols.h
	src/compiler/syntax_grammar.cc
	src/compiler/syntax_grammar.h
 2015-10-02 23:46:39 -07:00
Max Brunsfeld
90862dd18f Add some edit sequences to corpus tests 2015-09-22 21:19:19 -07:00
Max Brunsfeld
aba8af9e5b Cleanup debug logging in parser 2015-09-22 19:35:13 -07:00
Max Brunsfeld
b7b082d9b1 extract helper fn for testing edit sequences against the corpus 2015-09-20 13:38:46 -07:00
Max Brunsfeld
010a606b76 Add undo method to SpyInput 2015-09-19 14:46:14 -07:00
Max Brunsfeld
511e52ab55 Clean up SpyInput test helper 2015-09-19 13:29:30 -07:00
Max Brunsfeld
cdadee4610 Clean up language specs file 2015-09-18 23:42:20 -07:00
Max Brunsfeld
f37f73f92f Add ability to edit multiple times between parses 2015-09-18 23:20:06 -07:00
Max Brunsfeld
6254f45c1b Rename ParseStack -> Stack 2015-09-18 22:02:06 -07:00
Max Brunsfeld
b3d883e128 Store edits in trees, not by splitting stack 
This allows for multiple edits per parse, though it is not exposed through
the API yet
 2015-09-18 22:02:06 -07:00
Max Brunsfeld
0467d190fe Add ts_tree_edit function 2015-09-18 22:02:06 -07:00
Max Brunsfeld
66ad271c82 Add Document parse_count() method, for tracking if trees are still valid 2015-09-10 14:23:42 -07:00
Max Brunsfeld
a2d817f0f0 Remove unnecessary include from runtime spec-helper 2015-09-09 12:59:27 -07:00
Max Brunsfeld
252fa7b631 Add Document getter methods for input, language 2015-09-08 23:33:43 -07:00
Max Brunsfeld
ebd60213d9 Drop release functions from callback structs 
The caller can just as easily take care of the cleanup explicitly
 2015-09-08 23:24:33 -07:00