traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions
1504 commits 16 branches 98 tags 37 MiB
Commit graph

36 commits

Author SHA1 Message Date
Max Brunsfeld
0143bfdad4 Avoid use-after-free of external token states 
Previously, it was possible for references to external token states to
outlive the trees to which those states belonged.

Now, instead of storing references to external token states in the Stack
and in the Lexer, we store references to the external token trees
themselves, and we retain the trees to prevent use-after-free.
 2017-06-27 14:54:27 -07:00
Max Brunsfeld
0054272879 Record deallocations even when recording allocations is disabled 2017-06-27 14:54:27 -07:00
Max Brunsfeld
076002a01e Merge pull request #78 from philipturnbull/update-utf8proc 
Out of bounds read in utf8proc
 2017-06-23 12:18:21 -07:00
Max Brunsfeld
f62ee5a0f3 Fix OOB reads at ends of chunks 
Signed-off-by: Philip Turnbull <philipturnbull@github.com>
 2017-06-23 12:09:16 -07:00
Max Brunsfeld
2c043803f1 Be more conservative about avoiding lexing conflicts when merging states 
This fixes a bug in the C++ grammar where the `>>` token was merged into
a state where it was previously not valid, but the `>` token *was*
valid. This caused nested templates like -

std::vector<std::pair<int, int>>

to not parse correctly.
 2017-06-22 15:32:13 -07:00
Max Brunsfeld
6db12ab44e Adjust C error recovery tests to reflect grammar changes 2017-06-21 16:58:40 -07:00
Max Brunsfeld
0cc1090001 Read grammar tests from corpus dir instead of grammar_test dir 2017-06-21 16:31:25 -07:00
Max Brunsfeld
513edec7c1 Merge pull request #77 from philipturnbull/scan-build-fixes 
Fix errors found by scan-build
 2017-06-20 10:15:20 -07:00
Max Brunsfeld
e19393eff5 🎨 2017-06-15 17:25:21 -07:00
Max Brunsfeld
932feb2498 Fix gcc warning in test helper file 2017-06-15 17:13:01 -07:00
Max Brunsfeld
c66fddd3aa Add TSInput option to measure columns in bytes not characters 2017-06-15 16:35:34 -07:00
Max Brunsfeld
b862db766e Merge remote-tracking branch 'origin/master' into update-fixture-grammars 2017-06-14 17:11:44 -07:00
Phil Turnbull
18f261ad51 Initialise all fields of TSParseOptions in tests 
This should prevent any confusing failures in the unit tests:

test/runtime/document_test.cc:381:7: warning: Passed-by-value struct argument contains uninitialized data (e.g., field: 'changed_range_count')
      ts_document_parse_with_options(document, options);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
test/runtime/document_test.cc:408:7: warning: Passed-by-value struct argument contains uninitialized data (e.g., field: 'changed_range_count')
      ts_document_parse_with_options(document, options);
      ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 2017-06-14 11:12:06 -04:00
Phil Turnbull
aa6e93820c Silence false-positive warning in ts_record_free 
This is safe but I think it is technically undefined behaviour to use a pointer
after it has been freed:

test/helpers/record_alloc.cc:75:3: warning: Use of memory after it is freed
  record_deallocation(pointer);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
 2017-06-14 11:12:06 -04:00
Max Brunsfeld
74f5ceddf7 Fix parsing of valid code with halt_on_error flag set 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-05-01 14:25:25 -07:00
Max Brunsfeld
a98d449d88 Add an option to immediately halt on syntax error 2017-05-01 13:50:49 -07:00
Max Brunsfeld
82cb1c9806 Handle invalid UTF8 in encoding test helpers 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-04-27 14:49:44 -07:00
Max Brunsfeld
03a555a86e Finish test for invalid UTF8 handling 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-04-27 14:48:16 -07:00
Timothy Clem
37f2a4745f Test demonstrating non-UT8 input failure 2017-04-27 14:46:36 -07:00
Max Brunsfeld
a15e974150 Make clearer assertions about SpyInput's read strings 2017-03-21 12:14:04 -07:00
Max Brunsfeld
ca943f09a4 Update expected trees in error recovery test 2017-03-21 11:41:01 -07:00
Max Brunsfeld
7e0ae4505a Handle invalid UTF8 in encoding test helpers 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-03-21 11:12:08 -07:00
Max Brunsfeld
f032da198e Finish test for invalid UTF8 handling 
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
 2017-03-21 11:05:32 -07:00
Max Brunsfeld
63fb041961 Merge remote-tracking branch 'origin/check-utf8proc_iterate-return' into update-fixture-grammars 2017-03-21 09:59:35 -07:00
Timothy Clem
7092d4522a Test demonstrating non-UT8 input failure 2017-03-21 09:58:35 -07:00
Max Brunsfeld
af553420bf Update JS error corpus 2017-03-19 22:21:12 -07:00
Max Brunsfeld
20b8983749 Handle external scanner w/ .c extension in test helper 2017-03-19 22:20:16 -07:00
Max Brunsfeld
6d8f9ebaba In tests, regenerate parser if its timestamp matches grammar.json 
After running the fetch-fixtures script, their timestamps may be equal,
but in this situation we *do* want to re-generate the parsers.
 2017-03-17 17:04:04 -07:00
Max Brunsfeld
ed8fbff175 Allow anonymous tokens to be used in grammars' external token lists 2017-03-17 16:31:29 -07:00
Max Brunsfeld
e2baf0930b Use simple CharacterSet constructor in LexItem test 2017-03-17 14:47:11 -07:00
Max Brunsfeld
b3edd8f749 Remove use of shared_ptr in choice, repeat, and seq factories 2017-03-17 14:28:13 -07:00
Max Brunsfeld
db4b9ebc7c Implement Rule as a union rather than an abstract base class 2017-03-17 13:29:31 -07:00
Max Brunsfeld
42b05b4b5e Add simple unit test for invalidating trees preceding an edit due to lookahead 2017-03-13 17:34:31 -07:00
Max Brunsfeld
d222dbb9fd Allow lexer to accept tokens that ended at previous positions 
* Track lookahead in each tree
* Add 'mark_end' API that external scanners can use
 2017-03-13 17:06:52 -07:00
Max Brunsfeld
f04d7c5860 Handle unused tokens 2017-03-09 21:16:37 -08:00
Max Brunsfeld
6dc0ff359d Rename spec -> test 
'Test' is a lot more straightforward of a name.
 2017-03-09 20:40:01 -08:00