Commit graph

57 commits

Author SHA1 Message Date
Max Brunsfeld
6fca8f2f4d Make ts_compile_grammar take an optional log file, start logging to it 2018-05-24 16:01:14 -07:00
Max Brunsfeld
074c051094 Change the TSInputEdit struct to work with old/new start and end positions 2018-05-17 11:14:51 -07:00
Max Brunsfeld
6fc8d9871c Hide the details of TSNode's fields in the public API 2018-05-16 15:44:04 -07:00
Max Brunsfeld
32c06b9b59 Make multi-threaded test work on windows
Co-Authored-By: Rick Winfrey <rewinfrey@github.com>
2018-05-11 17:10:05 -07:00
Max Brunsfeld
043a2fc0d9 Assert absence of memory leaks in randomized multi-threaded tree test 2018-05-11 16:53:47 -07:00
Max Brunsfeld
a3e08e7c31 Add randomized multi-threaded tests on parse trees
Co-Authored-By: Rick Winfrey <rewinfrey@github.com>
2018-05-11 16:10:36 -07:00
Max Brunsfeld
fe53506175 Declare subtrees as const wherever possible
Co-Authored-By: Rick Winfrey <rewinfrey@github.com>
2018-05-11 15:06:13 -07:00
Max Brunsfeld
e75ecd1bb1 Rework API completely 2018-05-11 10:46:13 -07:00
Max Brunsfeld
35510a612d Rename Tree -> Subtree 2018-05-10 15:11:14 -07:00
Max Brunsfeld
a53d0b43a1 Remove unnecessary include 2018-05-09 22:55:54 -07:00
Max Brunsfeld
92255bbfdd Remove document parameter from ts_node_type, ts_node_string
Co-Authored-By: Rick Winfrey <rewinfrey@github.com>
2018-05-09 15:28:28 -07:00
Max Brunsfeld
973e4a44f0 Start work on removing parent pointers
Co-Authored-By: Rick Winfrey <rewinfrey@github.com>
2018-05-09 12:22:19 -07:00
Max Brunsfeld
09be0b6ef5 Store trees' children in TreeArrays, not w/ separate pointer and length 2018-04-06 13:26:18 -07:00
Max Brunsfeld
a8bc67ac42 Allow LookaheadSet::for_each to terminate early 2018-03-28 10:03:09 -07:00
Max Brunsfeld
43e14332ed Avoid creating duplicate metadata rules 2018-03-28 10:03:09 -07:00
Max Brunsfeld
07fa3eb386 Fix capture of corpus descriptions in integration tests 2018-03-02 11:27:41 -08:00
Max Brunsfeld
a8d539023d Handle subdirectories existing in parsers' examples folders 2018-03-02 11:04:08 -08:00
Max Brunsfeld
d3ac345644 When parsing corpus, anchor header pattern to line start 2018-03-02 09:46:33 -08:00
Max Brunsfeld
facafcd6e4 Pass row/column position to input seek method 2018-02-14 07:31:49 -08:00
Max Brunsfeld
0e69da37a5 Return a character count from the lexer's get_column method 2017-12-20 16:26:38 -08:00
Max Brunsfeld
90629bd45a Add some assertions to the fixture grammar tests 2017-12-07 11:50:27 -08:00
Max Brunsfeld
36c2b685b9 Always invalidate old chunk of text when parsing after an edit 2017-10-04 15:09:46 -07:00
Max Brunsfeld
8b3941764f Make outstanding_allocation_indices return a vector, not a set 2017-09-07 17:48:44 -07:00
Max Brunsfeld
4daf22ba0c Read files in binary mode in tests 2017-08-09 10:07:03 -07:00
Max Brunsfeld
90eef13aeb Fix windows directory listing function 2017-08-08 22:21:39 -07:00
Max Brunsfeld
d4b2c58dc8 Fix another hard-coded / as a path separator 2017-08-08 22:08:27 -07:00
Max Brunsfeld
d0dc164013 Disable the default behavior of printing a logo, Microsoft. 2017-08-08 17:35:50 -07:00
Max Brunsfeld
17b58f41e1 Disable optimization when compiling grammars during tests 2017-08-08 17:09:34 -07:00
Max Brunsfeld
17310769a4 Fix usage of .so instead of .dll on windows 2017-08-08 15:20:38 -07:00
Max Brunsfeld
bb7889fac5 Don't rely on PWD env var on windows 2017-08-08 15:04:27 -07:00
Max Brunsfeld
34b5340d71 Fix paths to corpus files on windows 2017-08-08 14:06:11 -07:00
Max Brunsfeld
b43ae2826b Use C++ stdlib for random number generation 2017-08-08 12:42:49 -07:00
Max Brunsfeld
fc0f49e4ee Add windows implementations of some IO-related test helpers 2017-08-08 12:21:17 -07:00
Max Brunsfeld
230f89d0ff Fix build warnings in tests 2017-08-07 12:19:10 -07:00
Max Brunsfeld
09f4796f6b Get tests passing w/ new alias API 2017-08-01 14:35:34 -07:00
Phil Turnbull
52cec9ed39 Rework SpyInput buffer handling
SpyInput uses a fixed-size buffer and explicitly zeros memory which is good for
catching logic errors but defeats valgrind's memory tracking. Use a separate
buffer of exactly the correct size for each request. This correctly catches the
problem under valgrind:

```
==8694== Invalid read of size 2
==8694==    at 0x54EFFB: utf16_iterate (utf16.c:10)
==8694==    by 0x551126: ts_lexer__get_lookahead (lexer.c:54)
==8694==    by 0x5515CD: ts_lexer_start (lexer.c:154)
==8694==    by 0x54699F: parser(long,...)(long long) (parser.c:297)
==8694==    by 0x54788A: parser__get_lookahead (parser.c:439)
==8694==    by 0x54B2D3: parser__advance (parser.c:1150)
==8694==    by 0x54C2AA: parser_parse (parser.c:1348)
==8694==    by 0x53F063: ts_document_parse_with_options (document.c:136)
==8694==    by 0x53EF43: ts_document_parse (document.c:107)
==8694==    by 0x4AED11: {lambda()#1}::operator()() const::{lambda()#1}::operator()() const::{lambda()#4}::operator()() const::{lambda()#4}::operator()() const (document_test.cc:82)
==8694==    by 0x4B56B6: std::_Function_handler<void (), {lambda()#1}::operator()() const::{lambda()#1}::operator()() const::{lambda()#4}::operator()() const::{lambda()#4}>::_M_invoke(std::_Any_data const&) (functional:1871)
==8694==    by 0x40F8C5: std::function<void ()>::operator()() const (functional:2267)
==8694==  Address 0x5d08be0 is 0 bytes inside a block of size 1 alloc'd
==8694==    at 0x4C2E80F: operator new[](unsigned long) (in /usr/lib/valgrind/vgpreload_memcheck-amd64-linux.so)
==8694==    by 0x507C3E: SpyInput::read(void*, unsigned int*) (spy_input.cc:66)
==8694==    by 0x55103D: ts_lexer__get_chunk (lexer.c:29)
==8694==    by 0x5515B6: ts_lexer_start (lexer.c:152)
==8694==    by 0x54699F: parser(long,...)(long long) (parser.c:297)
==8694==    by 0x54788A: parser__get_lookahead (parser.c:439)
==8694==    by 0x54B2D3: parser__advance (parser.c:1150)
==8694==    by 0x54C2AA: parser_parse (parser.c:1348)
==8694==    by 0x53F063: ts_document_parse_with_options (document.c:136)
==8694==    by 0x53EF43: ts_document_parse (document.c:107)
==8694==    by 0x4AED11: {lambda()#1}::operator()() const::{lambda()#1}::operator()() const::{lambda()#4}::operator()() const::{lambda()#4}::operator()() const (document_test.cc:82)
==8694==    by 0x4B56B6: std::_Function_handler<void (), {lambda()#1}::operator()() const::{lambda()#1}::operator()() const::{lambda()#4}::operator()() const::{lambda()#4}>::_M_invoke(std::_Any_data const&) (functional:1871)
```
2017-07-18 12:16:37 -07:00
Max Brunsfeld
4b40a1ed6c Support anonymous tokens inside of RENAME rules 2017-07-14 10:19:58 -07:00
Max Brunsfeld
a3006bc2b5 Represent LookaheadSet using vectors of bool 2017-07-12 16:02:01 -07:00
Max Brunsfeld
5c8f7c035e Add stream operator for ParseItemSet 2017-07-12 09:42:56 -07:00
Max Brunsfeld
d8e9d04fe7 Add PREC_DYNAMIC rule for resolving runtime ambiguities 2017-07-06 15:24:45 -07:00
Max Brunsfeld
c53f9bcbd9 Build benchmarks in Test mode for now 2017-07-05 17:27:50 -07:00
Max Brunsfeld
17bc3dfaf7 Add a benchmark command
This command measures the speed of parsing each grammar's examples.
It also uses each grammar to parse all of the *other* grammars' examples
in order to measure error recovery performance with fairly large files.
2017-07-05 14:14:38 -07:00
Max Brunsfeld
fcffd4b732 Add test for an example found during fuzzing 2017-06-30 21:55:50 -07:00
Max Brunsfeld
0054272879 Record deallocations even when recording allocations is disabled 2017-06-27 14:54:27 -07:00
Max Brunsfeld
0cc1090001 Read grammar tests from corpus dir instead of grammar_test dir 2017-06-21 16:31:25 -07:00
Max Brunsfeld
513edec7c1 Merge pull request #77 from philipturnbull/scan-build-fixes
Fix errors found by scan-build
2017-06-20 10:15:20 -07:00
Max Brunsfeld
932feb2498 Fix gcc warning in test helper file 2017-06-15 17:13:01 -07:00
Max Brunsfeld
c66fddd3aa Add TSInput option to measure columns in bytes not characters 2017-06-15 16:35:34 -07:00
Phil Turnbull
aa6e93820c Silence false-positive warning in ts_record_free
This is safe but I think it is technically undefined behaviour to use a pointer
after it has been freed:

test/helpers/record_alloc.cc:75:3: warning: Use of memory after it is freed
  record_deallocation(pointer);
  ^~~~~~~~~~~~~~~~~~~~~~~~~~~~
2017-06-14 11:12:06 -04:00
Max Brunsfeld
82cb1c9806 Handle invalid UTF8 in encoding test helpers
Signed-off-by: Tim Clem <timothy.clem@gmail.com>
2017-04-27 14:49:44 -07:00