traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
6118 commits 16 branches 98 tags 37 MiB
Commit graph

6118 commits

This branch
This branch
All branches
Author SHA1 Message Date
Phil Turnbull
fdd8792ebc Correctly set is_first 
From scan-build: Value stored to 'is_first' is never read
 2017-06-14 11:12:06 -04:00
Max Brunsfeld
040e225a54 Merge pull request #76 from philipturnbull/remove-duplicates-oob 
OOB-read in remove_duplicate_parse_states
 2017-06-14 07:20:09 -07:00
Phil Turnbull
c58f6401d0 Non-terminal entries always have valid state-ids 2017-06-14 08:49:38 -04:00
Phil Turnbull
577e43f653 shift-extra actions do not have valid state_ids 2017-06-09 16:26:01 -04:00
Phil Turnbull
18ba6ebbd7 Move state_id check into each_referenced_state 2017-06-09 16:25:59 -04:00
Phil Turnbull
6897530c47 Check for invalid state indexes 
Some ParseActions have a state-id of -1 which can cause an out-of-bounds read
when removing duplicate parse states. This was found by AddressSanitizer:

==90699==ERROR: AddressSanitizer: heap-buffer-overflow on address 0x6320000187f8 at pc 0x0001071220a9 bp 0x7fff595fd440 sp 0x7fff595fd438
READ of size 8 at 0x6320000187f8 thread T0
    #0 0x1071220a8 in tree_sitter::build_tables::ParseTableBuilder::remove_duplicate_parse_states()::'lambda0'(unsigned long*)::operator()(unsigned long*) const build_parse_table.cc:398
    #1 0x107121fa5 in void std::__1::__invoke_void_return_wrapper<void>::__call<tree_sitter::build_tables::ParseTableBuilder::remove_duplicate_parse_states()::'lambda0'(unsigned long*)&, unsigned long*>(tree_sitter::build_tables::ParseTableBuilder::remove_duplicate_parse_states()::'lambda0'(unsigned long*)&&&, unsigned long*&&) __functional_base:416
...
0x6320000187f8 is located 8 bytes to the left of 88264-byte region [0x632000018800,0x63200002e0c8)
allocated by thread T0 here:
    #0 0x107b1576b in wrap__Znwm (libclang_rt.asan_osx_dynamic.dylib:x86_64h+0x6076b)
    #1 0x10711da2c in std::__1::vector<unsigned long, std::__1::allocator<unsigned long> >::allocate(unsigned long) new:169
    #2 0x10711d8fb in std::__1::vector<unsigned long, std::__1::allocator<unsigned long> >::vector(unsigned long) vector:1074
    #3 0x107112f5c in std::__1::vector<unsigned long, std::__1::allocator<unsigned long> >::vector(unsigned long) vector:1068
    #4 0x1070af381 in tree_sitter::build_tables::ParseTableBuilder::remove_duplicate_parse_states() build_parse_table.cc:378
    #5 0x10709d827 in tree_sitter::build_tables::ParseTableBuilder::build() build_parse_table.cc:85
...
SUMMARY: AddressSanitizer: heap-buffer-overflow build_parse_table.cc:398 in tree_sitter::build_tables::ParseTableBuilder::remove_duplicate_parse_states()::'lambda0'(unsigned long*)::operator()(unsigned long*) const
Shadow bytes around the buggy address:
  0x1c64000030a0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c64000030b0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c64000030c0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c64000030d0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
  0x1c64000030e0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa
=>0x1c64000030f0: fa fa fa fa fa fa fa fa fa fa fa fa fa fa fa[fa]
  0x1c6400003100: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c6400003110: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c6400003120: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c6400003130: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
  0x1c6400003140: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
 2017-06-07 17:23:44 -04:00
Phil Turnbull
dee86f908a Correctly check type is ParseActionTypeRecover 2017-06-07 17:05:39 -04:00
Max Brunsfeld
f829e81f30 Add MIT license 2017-05-03 10:32:12 -07:00
Max Brunsfeld
7b401de5a6 Don't use pointer equality to compare external token states 2017-05-03 09:57:09 -07:00
Max Brunsfeld
34b47536e0 Merge pull request #75 from tree-sitter/halt-on-error-option 
Add an option to immediately halt on syntax error
 2017-05-03 09:49:40 -07:00
Max Brunsfeld
e8a9bb7a51 🎨 Extract parser__halt_parse function 2017-05-01 14:41:55 -07:00
Max Brunsfeld
74f5ceddf7 Fix parsing of valid code with halt_on_error flag set 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-05-01 14:25:25 -07:00
Max Brunsfeld
a98d449d88 Add an option to immediately halt on syntax error 2017-05-01 13:50:49 -07:00
Timothy Clem
f594ed2519 Merge pull request #74 from tree-sitter/check-utf8proc_iterate-return 
Check utf8proc_iterate return
 2017-05-01 10:46:24 -07:00
Max Brunsfeld
82cb1c9806 Handle invalid UTF8 in encoding test helpers 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-04-27 14:49:44 -07:00
Max Brunsfeld
704c2d5907 Fix lookahead_char type in ts_tree_make_error function 2017-04-27 14:49:04 -07:00
Max Brunsfeld
03a555a86e Finish test for invalid UTF8 handling 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-04-27 14:48:16 -07:00
Timothy Clem
37f2a4745f Test demonstrating non-UT8 input failure 2017-04-27 14:46:36 -07:00
Timothy Clem
91558f0a0e utf8proc_iterate can set codepoint_ref to -1 and returns negative error 2017-04-27 14:46:36 -07:00
Rob Rix
09d3b5ef3a Merge pull request #73 from tree-sitter/symbol-types 
Symbol types
 2017-04-12 12:42:57 -04:00
Rob Rix
3a888b1623 Define a function providing the type of a given symbol. 2017-04-12 09:47:51 -04:00
Rob Rix
4b1f69142d Define a symbol type enum. 2017-04-12 09:46:01 -04:00
Max Brunsfeld
3298c2ce88 Merge pull request #72 from tree-sitter/unique-external-scanner-map 
Make external scanner symbol map unique
 2017-03-24 16:02:47 -07:00
joshvera
f76935cc7e just make it static 2017-03-24 18:38:21 -04:00
joshvera
6938b288a5 Make external scanner symbol map unique 2017-03-24 14:51:37 -04:00
Max Brunsfeld
1f908324dc Prevent infinite loop in skip_preceding_trees error recovery strategy 2017-03-21 12:14:44 -07:00
Max Brunsfeld
a15e974150 Make clearer assertions about SpyInput's read strings 2017-03-21 12:14:04 -07:00
Max Brunsfeld
ca943f09a4 Update expected trees in error recovery test 2017-03-21 11:41:01 -07:00
Max Brunsfeld
7e0ae4505a Handle invalid UTF8 in encoding test helpers 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-03-21 11:12:08 -07:00
Max Brunsfeld
7e13eac296 Fix lookahead_char type in ts_tree_make_error function 2017-03-21 11:05:48 -07:00
Max Brunsfeld
f032da198e Finish test for invalid UTF8 handling 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-03-21 11:05:32 -07:00
Max Brunsfeld
63fb041961 Merge remote-tracking branch 'origin/check-utf8proc_iterate-return' into update-fixture-grammars 2017-03-21 09:59:35 -07:00
Timothy Clem
7092d4522a Test demonstrating non-UT8 input failure 2017-03-21 09:58:35 -07:00
Timothy Clem
f394a48c0b utf8proc_iterate can set codepoint_ref to -1 and returns negative error 2017-03-20 16:54:19 -07:00
Max Brunsfeld
af553420bf Update JS error corpus 2017-03-19 22:21:12 -07:00
Max Brunsfeld
ed31e82ee6 Skip empty tokens when recovering from errors 2017-03-19 22:20:59 -07:00
Max Brunsfeld
20b8983749 Handle external scanner w/ .c extension in test helper 2017-03-19 22:20:16 -07:00
Max Brunsfeld
97bcc86a3a Use master version of all fixture grammars 2017-03-19 22:19:35 -07:00
Max Brunsfeld
3f0640aa8d Merge pull request #70 from tree-sitter/anonymous-external-tokens 
Allow anonymous tokens to be used in grammars' external token lists
 2017-03-17 17:10:13 -07:00
Max Brunsfeld
24878277e9 Use new version of python grammar in tests 2017-03-17 17:05:02 -07:00
Max Brunsfeld
6d8f9ebaba In tests, regenerate parser if its timestamp matches grammar.json 
After running the fetch-fixtures script, their timestamps may be equal,
but in this situation we *do* want to re-generate the parsers.
 2017-03-17 17:04:04 -07:00
Max Brunsfeld
17876b6826 Update grammar JSON schema 2017-03-17 16:41:30 -07:00
Max Brunsfeld
ed8fbff175 Allow anonymous tokens to be used in grammars' external token lists 2017-03-17 16:31:29 -07:00
Max Brunsfeld
e2baf0930b Use simple CharacterSet constructor in LexItem test 2017-03-17 14:47:11 -07:00
Max Brunsfeld
4d39f13eaf Merge pull request #69 from tree-sitter/rules-variant 
Implement Rule as a union type rather than an abstract base class
 2017-03-17 14:35:42 -07:00
Max Brunsfeld
b3edd8f749 Remove use of shared_ptr in choice, repeat, and seq factories 2017-03-17 14:28:13 -07:00
Max Brunsfeld
d9fb863bea Fix build errors w/ gcc 2017-03-17 14:03:49 -07:00
Max Brunsfeld
416cbb9def Add missing cassert includes 2017-03-17 13:54:40 -07:00
Max Brunsfeld
90d21adf3b Format make_visitor helper consistently w/ project 2017-03-17 13:37:26 -07:00
Max Brunsfeld
79ffc20be1 Remove RTTI flag in gyp file 2017-03-17 13:31:35 -07:00