Start work on contributing guide
This commit is contained in:
parent
d274e81d0d
commit
2f06abf984
2 changed files with 143 additions and 3 deletions
|
|
@ -3,7 +3,7 @@ title: Using Parsers
|
|||
permalink: using-parsers
|
||||
---
|
||||
|
||||
# Using Parsers
|
||||
# Ucsing Parsers
|
||||
|
||||
All of Tree-sitter's parsing functionality is exposed through C APIs. Applications written in higher-level languages can use Tree-sitter via binding libraries like [node-tree-sitter](https://github.com/tree-sitter/node-tree-sitter) or [rust-tree-sitter](https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding_rust), which have their own documentation.
|
||||
|
||||
|
|
@ -36,9 +36,9 @@ Alternatively, you can use the library in a larger project by adding one source
|
|||
## The Objects
|
||||
|
||||
There are four main types of objects involved when using Tree-sitter: languages, parsers, syntax trees, and syntax nodes. In C, these are called `TSLanguage`, `TSParser`, `TSTree`, and `TSNode`.
|
||||
* A `TSLanguage` is an opaque object that defines how to parse a particular programming language. The code for each `TSLanguage` is generated by Tree-sitter. Many languages are already available in separate git repositories within the the [Tree-sitter GitHub organization](https://github.com/tree-sitter). See [the next section](./creating-parsers) for how to create new languages.
|
||||
* An `TSLanguage` is an opaque object that defines how to parse a particular programming language. The code for each `TSLanguage` is generated by Tree-sitter. Many languages are already available in separate git repositories within the the [Tree-sitter GitHub organization](https://github.com/tree-sitter). See [the next section](./creating-parsers) for how to create new languages.
|
||||
* A `TSParser` is a stateful object that can be assigned a `TSLanguage` and used to produce a `TSTree` based on some source code.
|
||||
* A `TSTree` represents the syntax tree of an entire source code file. Its contains `TSNode` instances that indicate the structure of the source code. It can also be edited and used to produce a new `TSTree` in the event that the source code changes.
|
||||
* A `TSTree` represents the syntax tree of an entire source code file. It contains `TSNode` instances that indicate the structure of the source code. It can also be edited and used to produce a new `TSTree` in the event that the source code changes.
|
||||
* A `TSNode` represents a single node in the syntax tree. It tracks its start and end positions in the source code, as well as its relation to other nodes like its parent, siblings and children.
|
||||
|
||||
## An Example Program
|
||||
|
|
|
|||
140
docs/section-6-contributing.md
Normal file
140
docs/section-6-contributing.md
Normal file
|
|
@ -0,0 +1,140 @@
|
|||
---
|
||||
title: Contributing
|
||||
permalink: contributing
|
||||
---
|
||||
|
||||
# Contributing
|
||||
|
||||
## Developing Tree-sitter
|
||||
|
||||
### Prerequisites
|
||||
|
||||
To make changes to Tree-sitter, you should have:
|
||||
|
||||
1. A C compiler, for compiling the core library and the generated parsers.
|
||||
2. A [Rust toolchain](https://rustup.rs/), for compiling the Rust bindings, the highlighting library, and the CLI.
|
||||
3. Node.js and NPM, for generating parsers from `grammar.js` files.
|
||||
4. Either [Docker](https://www.docker.com/) or [Emscripten](https://emscripten.org/), for compiling the library to WASM.
|
||||
|
||||
### Building
|
||||
|
||||
Clone the repository:
|
||||
|
||||
```sh
|
||||
git clone https://github.com/tree-sitter/tree-sitter
|
||||
cd tree-sitter
|
||||
```
|
||||
|
||||
Clone the [`utf8proc`](https://juliastrings.github.io/utf8proc/) submodule:
|
||||
|
||||
```
|
||||
git submodule update --init
|
||||
```
|
||||
|
||||
Build the WASM library. We do this first because it gets embedded in the CLI to enable the `web-ui` command. If you have emscripten installed, this will use your `emcc` compiler. Otherwise, it will use Docker:
|
||||
|
||||
```sh
|
||||
./script/build-wasm
|
||||
```
|
||||
|
||||
Build the Rust libraries and the CLI:
|
||||
|
||||
```sh
|
||||
cargo build --release
|
||||
```
|
||||
|
||||
This will create the `tree-sitter` CLI executable in the `target/release` folder.
|
||||
|
||||
### Testing
|
||||
|
||||
Before you can run the tests, you need to clone some grammars that are used for testing:
|
||||
|
||||
```sh
|
||||
script/fetch-fixtures
|
||||
```
|
||||
|
||||
To test any changes you've made to the CLI, you can regenerate these parsers using your current CLI code:
|
||||
|
||||
```sh
|
||||
script/generate-fixtures
|
||||
```
|
||||
|
||||
Then you can run the tests:
|
||||
|
||||
```sh
|
||||
script/test
|
||||
```
|
||||
|
||||
### Debugging
|
||||
|
||||
The test script has a number of useful flags. You can list them all by running `script/test -h`. Here are some of the main flags:
|
||||
|
||||
If you want to run a specific unit test, pass its name (or part of its name) as an argument:
|
||||
|
||||
```sh
|
||||
script/test test_does_something
|
||||
```
|
||||
|
||||
You can run the tests under the debugger (either `lldb` or `gdb`) using the `-g` flag:
|
||||
|
||||
```sh
|
||||
script/test test_does_something -g
|
||||
```
|
||||
|
||||
Part of the Tree-sitter test suite involves parsing the *corpus* tests for several different languages and performing randomized edits to each example in the corpus. If you just want to run the tests for a particular *language*, you can pass the `-l` flag. And if you want to run a particular *example* from the corpus, you can pass the `-e` flag:
|
||||
|
||||
```sh
|
||||
script/test -l javascript -e Arrays
|
||||
```
|
||||
|
||||
## Published Packages
|
||||
|
||||
The main [`tree-sitter/tree-sitter`](https://github.com/tree-sitter/tree-sitter) repository contains the source code for several packages that are published to package registries for different languages:
|
||||
|
||||
* Rust crates on [crates.io](https://crates.io):
|
||||
* [`tree-sitter`](https://crates.io/crates/tree-sitter) - A Rust binding to the core library
|
||||
* [`tree-sitter-highlight`](https://crates.io/crates/tree-sitter-highlight) - The syntax-highlighting library
|
||||
* [`tree-sitter-cli`](https://crates.io/crates/tree-sitter-cli) - The command-line tool
|
||||
* JavaScript modules on [npmjs.com](https://npmjs.com):
|
||||
* [`web-tree-sitter`](https://www.npmjs.com/package/web-tree-sitter) - A WASM-based JavaScript binding to the core library
|
||||
* [`tree-sitter-cli`](https://www.npmjs.com/package/tree-sitter-cli) - The command-line tool
|
||||
|
||||
There are also several other dependent repositories that contain other published packages:
|
||||
|
||||
* [`tree-sitter/node-tree-sitter`](https://github.com/tree-sitter/py-tree-sitter) - Node.js bindings to the core library, published as [`tree-sitter`](https://www.npmjs.com/package/tree-sitter) on npmjs.com
|
||||
* [`tree-sitter/py-tree-sitter`](https://github.com/tree-sitter/py-tree-sitter) - Python bindings to the core library, published as [`tree-sitter`](https://pypi.org/project/tree-sitter) on [PyPI.org](https://pypi.org).
|
||||
|
||||
## Publishing New Releases
|
||||
|
||||
Publishing a new release of the CLI requires these steps:
|
||||
|
||||
1. Commit and push all outstanding changes and verify that CI passes:
|
||||
|
||||
```sh
|
||||
git commit -m "Fix things"
|
||||
git push
|
||||
```
|
||||
|
||||
2. Create a new tag:
|
||||
|
||||
```sh
|
||||
script/version patch
|
||||
```
|
||||
|
||||
This will determine the current version, increment the *patch* version number, and update the `Cargo.toml` and `package.json` files for the Rust and Node CLI packages. It will then create a commit and a tag for the new version. For more information about the arguments that are allowed, see the documentation for the [`npm version`](https://docs.npmjs.com/cli/version) command.
|
||||
|
||||
3. Push the commit and the tag:
|
||||
|
||||
```sh
|
||||
git push
|
||||
git push --tags
|
||||
```
|
||||
|
||||
4. Wait for CI to pass. Because of the git tag, the CI jobs will publish artifacts to [a GitHub release](https://github.com/tree-sitter/tree-sitter/releases). The npm module of `tree-sitter-cli` works by downloading the appropriate binary from the corresponding GitHub release during installation. So it's best not to publish the npm package until the binaries are uploaded.
|
||||
|
||||
5. Publish the npm package:
|
||||
|
||||
```sh
|
||||
cd cli/npm
|
||||
npm publish
|
||||
```
|
||||
Loading…
Add table
Add a link
Reference in a new issue