Commit graph

3965 commits

Author SHA1 Message Date
Max Brunsfeld
efe009f47d
Merge pull request #1759 from rhysd/doc-alias
Add C API names as document aliases in Rust docs
2022-05-31 14:45:18 -07:00
Max Brunsfeld
1c9d2c5bb0
Merge pull request #1763 from rhysd/fix-doc-warning
cli: fix warnings from `cargo doc`
2022-05-31 14:44:29 -07:00
rhysd
21c6cd8c22 cli: fix warnings from cargo doc 2022-05-31 10:59:30 +09:00
rhysd
08899428f3 Add C APIs as document aliases 2022-05-30 21:36:11 +09:00
Max Brunsfeld
b729029a40
Merge pull request #1747 from Cycatz/master
Add link to Org language parser
2022-05-16 09:51:42 -07:00
Cycatz
369c88713c
Add link to Org language parser 2022-05-16 15:58:33 +00:00
Max Brunsfeld
6192d29610
Merge pull request #1738 from rhysd/patch-2
Add link to JSON parser in document
2022-05-03 10:30:11 -07:00
Linda_pp
d9365a8fd6
Add link to JSON parser in document 2022-05-04 00:01:02 +09:00
Max Brunsfeld
cfbafabe6f
Merge pull request #1643 from fengT-T/master
Fix: Set https proxy for npm cli install script
2022-04-26 09:54:04 -07:00
Max Brunsfeld
465ceead0f
Merge pull request #1677 from siegel/master
Fixed warning/error when compiling with `clang -Os`.
2022-04-03 15:30:06 -07:00
Max Brunsfeld
bd3d84162f
Merge pull request #1692 from alebastr/alt-arch-fixes
Fix build and tests on alternative architectures
2022-04-03 15:29:12 -07:00
Aleksei Bavshin
fe33599f46
lib: fix incorrect int ptr cast on big-endian architectures
`*usize` -> `*u32` conversion on 64-bit big-endian machine takes high
halfword of the value. As a consequence, any result returned via
`count` is unexpectedly shifted left:

    u32   = 00 00 00 01             // 1
    usize = 00 00 00 01 00 00 00 00 // 4294967296

Fixes following test failure:
```
$ cargo test -- tests::corpus_test
<...>
running 13 tests
memory allocation of 206158430208 bytes failed
error: test failed, to rerun pass '--lib'
```
2022-03-23 00:47:01 -07:00
Aleksei Bavshin
cdf2ecd176
tags: fix incorrect uses of i8 instead of c_char
Fixes build on aarch64, ppc64le and other platforms that have c_char
defined as u8.
2022-03-23 00:01:51 -07:00
Aleksei Bavshin
b27b4665ac
test: fix incorrect uses of i8 instead of c_char
Fixes build on aarch64, ppc64le and other platforms that have c_char
defined as u8.
2022-03-23 00:00:26 -07:00
Rich Siegel
150eb2966b Fixed warning/error when compiling with clang -Os.
DISCUSSION:

When compiling with `-Os` for "smallest, fastest", an error is reported in `parser.c`:

```
/Users/siegel/git/tree-sitter/lib/src/./parser.c:1368:10: error: unused variable 'did_merge' [-Werror,-Wunused-variable]
    bool did_merge = ts_stack_merge(self->stack, version, previous_version_count);
         ^
1 error generated.
```

This is because with `NDEBUG` set,  `assert(e)` collapses to `(void)0`,
which in turn means that `did_merge` does not actually get consumed.
This seems to get caught when compiling with `-Os`, but not otherwise.

Compiler version:
```
Apple clang version 13.0.0 (clang-1300.0.29.30)
Target: arm64-apple-darwin21.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
```
2022-03-04 18:00:16 -05:00
Max Brunsfeld
ccd6bf554d 0.20.6 2022-03-02 20:50:52 -08:00
Max Brunsfeld
1b2e90f647 libs: 0.20.6 2022-03-02 20:50:29 -08:00
Max Brunsfeld
8decec3774 Properly incorporate lookahead bytes when recovering via missing token 2022-03-02 17:12:25 -08:00
Max Brunsfeld
7170ec7c96 Improve randomized testing setup
* Allow iterations to be specified via an env var
* Randomly decide the edit count, with a maximum
  specified via an env var.
* Instead of separate env vars for starting seed + trial, just accept a seed
* Remove some noisy output
2022-03-02 17:12:25 -08:00
Max Brunsfeld
4bf5149a18 Run cargo test directly on CI 2022-03-02 17:12:25 -08:00
Max Brunsfeld
a6a102bec9 Set CLI as default workspace member 2022-03-02 16:36:14 -08:00
Max Brunsfeld
4b93326898 Don't generate primary states array if it will be unused due to abi version setting 2022-03-02 14:57:59 -08:00
Max Brunsfeld
621eb04cb1 tags: 0.20.2 2022-03-02 14:48:58 -08:00
Max Brunsfeld
09d08e8f8b 0.20.5 2022-03-02 14:43:53 -08:00
Max Brunsfeld
fcbef45899 libs: 0.20.5 2022-03-02 14:43:16 -08:00
Max Brunsfeld
38d2cd15bd
Merge pull request #1667 from tree-sitter/dependabot/bundler/docs/nokogiri-1.13.3
Bump nokogiri from 1.11.4 to 1.13.3 in /docs
2022-02-26 16:56:14 -08:00
Max Brunsfeld
c42882b149
Merge pull request #1666 from the-mikedavis/md-tags-testing-docs
add unit testing documentation for tags queries
2022-02-26 16:13:02 -08:00
Max Brunsfeld
a494d6aa28
Merge pull request #1668 from tree-sitter/remember-lookahead-bytes-on-error-detection
Remember lookahead bytes on error detection
2022-02-26 14:59:40 -08:00
Max Brunsfeld
c697ebfb27 Add explicit unit test for error detection lookahead bug 2022-02-26 14:33:09 -08:00
dependabot[bot]
fa3e87479f
Bump nokogiri from 1.11.4 to 1.13.3 in /docs
Bumps [nokogiri](https://github.com/sparklemotion/nokogiri) from 1.11.4 to 1.13.3.
- [Release notes](https://github.com/sparklemotion/nokogiri/releases)
- [Changelog](https://github.com/sparklemotion/nokogiri/blob/main/CHANGELOG.md)
- [Commits](https://github.com/sparklemotion/nokogiri/compare/v1.11.4...v1.13.3)

---
updated-dependencies:
- dependency-name: nokogiri
  dependency-type: indirect
...

Signed-off-by: dependabot[bot] <support@github.com>
2022-02-26 03:43:21 +00:00
Michael Davis
699629ecdd
add unit testing documentation for tags queries 2022-02-25 05:46:30 -06:00
Max Brunsfeld
9866674cf8
Merge pull request #1660 from alex-pinkus/expanded-regex-support
Expand regex support to include emojis and binary ops
2022-02-24 17:14:23 -08:00
Max Brunsfeld
5eb0a3090f
Merge pull request #1547 from the-mikedavis/md-test-tags
test tags queries in 'tree-sitter test'
2022-02-24 15:22:15 -08:00
Patrick Thomson
3bd6fae4cc
Merge pull request #1649 from tree-sitter/tag-name-conventions
Describe tagging and associated naming conventions for syntax captures.
2022-02-24 09:26:00 -05:00
Max Brunsfeld
0fb864c1a0 Retain information about the lexer's lookahead for the token where an error was detected 2022-02-22 09:45:26 -08:00
Max Brunsfeld
0bdd9b640c Store the lookahead subtree of paused stack versions, not just the lookahead symbol 2022-02-22 09:45:26 -08:00
Max Brunsfeld
af00782dfd Add files needed for using clangd 2022-02-22 09:44:50 -08:00
Max Brunsfeld
d08f1af15c 🎨 2022-02-22 09:43:57 -08:00
Max Brunsfeld
be71c6e3e9
Merge pull request #1567 from jamessan/config-min-serde-json-ver
config: Bump minimum serde_json version to 1.0.45
2022-02-21 19:47:03 -08:00
Alex Pinkus
8fadf18655 Expand regex support to include emojis and binary ops
The `Emoji` property alias is already present, but the actual property
is not available since it lives in a new file. This adds that file to
the `generate-unicode-categories-json`.

The `emoji-data` file follows the same format as the ones we already
consume in `generate-unicode-categories-json`, so adding emoji support
is fairly easy. his, grammars would need to hard-code a set of
unicode ranges in their own regex. The Javascript library `emoji-regex`
cannot be used because of #451.

For unclear reasons, the characters #, *, and 0-9 are marked as
`Emoji=Yes` by `emoji-data.txt`. Because of this, a grammar that wishes
to use emojis is likely to want to exclude those characters. For that
reason, this change also adds support for binary operations in regexes,
e.g. `[\p{Emoji}&&[^#*0-9]]`.

Lastly (and perhaps controversially), this change introduces new
variables available at grammar compile time, for the major, minor, and
patch versions of the tree-sitter CLI used to compile the grammar. This
will allow grammars to conditionally adopt these new regex features
while remaining backward compatible with older versions of the CLI.
Without this part of the change, grammar authors who do not precompile
and check-in their `grammar.json` would need to wait for downstream
systems to adopt a newer tree-sitter CLI version before they could begin
to use these features.
2022-02-19 11:41:36 -08:00
Patrick Thomson
764c8c88ca last tweaks 2022-02-18 09:24:04 -05:00
Patrick Thomson
27019d1172 demonstrate that select-adjacent works 2022-02-17 18:28:09 -05:00
Patrick Thomson
65da86f16f Missing plural here. 2022-02-17 18:11:01 -05:00
Patrick Thomson
48748ee332 Typo. 2022-02-17 18:05:50 -05:00
Patrick Thomson
e1ac2e2648 Better nomenclature. 2022-02-17 18:05:19 -05:00
Patrick Thomson
4c60217345 Flesh out output. 2022-02-17 17:43:14 -05:00
Patrick Thomson
69a5f77eab Describe how to use tree-sitter tags as well. 2022-02-17 17:34:15 -05:00
Patrick Thomson
1fbace136d Add examples. 2022-02-17 17:20:21 -05:00
Patrick Thomson
70077b8205 Incorporate @dcreager's excellent suggestions. 2022-02-17 14:00:34 -05:00
Patrick Thomson
f41e13f5da Spacing and word choice. 2022-02-11 15:41:53 -05:00