* Rename corpus test functions to allow easy filtering by language
* Use usize for seed argument
* Avoid retaining useless stack versions when reductions merge
We found this problem when debugging an infinite loop that happened
during error recovery when using the Zig grammar. The large number of
unnecessary paused stack versions were preventing the correct recovery
strategy from being tried.
* Fix leaked lookahead token when reduction results in a merged stack
* Enable running PHP tests in CI
* Fix possible infinite loop during error recovery at EOF
* Account for external scanner state changes when detecting changed ranges in subtrees
On macOS, this was done by default regardless of what the user wants.
This was also not done on Windows or Linux. Instead, we now provide a
`--open-log` flag to open the log file in the default browser, and it
works on all platforms.
The new dumped edits format is `edit.<SEED>.<TEST_INDEX>.<SEED_INDEX>.<TEST_NAME>`
To cleanup edits:
- Instead of the standard:
> rm target/scratch/edit.*
- Use the following shell command:
> find target/scratch/ -name 'edit.*' -delete
* Allow iterations to be specified via an env var
* Randomly decide the edit count, with a maximum
specified via an env var.
* Instead of separate env vars for starting seed + trial, just accept a seed
* Remove some noisy output
These tests are easier to write and maintain if the grammars are just JS,
like grammars normally are. It doesn't slow the tests down significantly
to shell out to `node` for each of these grammars.
This patch updates the CLI to use anyhow and thiserror for error
management. The main feature that our custom `Error` type was providing
was a _list_ of messages, which would allow us to annotate "lower-level"
errors with more contextual information. This is exactly what's
provided by anyhow's `Context` trait.
(This is setup work for a future PR that will pull the `config` and
`loader` modules out into separate crates; by using `anyhow` we wouldn't
have to deal with a circular dependency between with the new crates.)
We have several test cases defined in the `cli` crate that depend on the
`lib` crate's `allocation-tracking` feature. The implementation of the
actual allocation tracker used to live in the `cli` crate, close to the
test cases that use it. The `allocation-tracking` feature in the `lib`
crate was just used to tell the tree-sitter implementation to expect
that the allocation tracker exists, and to use it.
That pattern meant that we had a circular dependency: `cli` depends on
`lib`, but `lib` required some code that was implemented in `cli`.
That, in turn, caused linker errors — but only when compiling in certain
configurations! [1]
This patch moves all of the allocation tracking implementation into the
`lib` crate, gated on the existing `allocation-tracking` feature, which
fixes the circular dependency.
Note that this patch does **not** fix the fact that feature unification
causes the `lib` crate to be built with the `allocation-tracking`
feature enabled, even though it's not a default. Fixing that depends on
the forthcoming version 2 feature resolver [2], or using the `dev_dep`
workaround [3] in the meantime.
[1] https://github.com/tree-sitter/tree-sitter/issues/919
[2] https://doc.rust-lang.org/nightly/cargo/reference/features.html#feature-resolver-version-2
[3] https://github.com/tree-sitter/tree-sitter/issues/919#issuecomment-777107086
* Requery parse table after breaking down parse stack due to invalid lookahead
* Include Ruby parser in randomized test suite
Ruby and PHP are our only two languages that use non-terminal extras.
Adding Ruby uncovered some bugs.
* Print edited source code when running parse --edit w/ debug flag
* Recompute lookahead when breaking down stack on invalid lookahead
* Fix stack summary leak when there are two discontinuities on a stack version
