Reorganize language bindings

* Move rust binding: lib/binding -> lib/binding_rust
* Move wasm bindinig: lib/web -> lib/binding_web
* Add wasm readme
This commit is contained in:
Max Brunsfeld 2019-05-07 10:27:45 -07:00
parent a3ceb8f3a5
commit 3fc459a84b
23 changed files with 125 additions and 18 deletions

View file

@ -15,9 +15,10 @@ Tree-sitter is a parser generator tool and an incremental parsing library. It ca
There are currently bindings that allow Tree-sitter to be used from the following languages:
* [JavaScript](https://github.com/tree-sitter/node-tree-sitter)
* [Rust](https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding_rust)
* [JavaScript (Wasm)](https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding_web)
* [JavaScript (Node.js)](https://github.com/tree-sitter/node-tree-sitter)
* [Python](https://github.com/tree-sitter/py-tree-sitter)
* [Rust](https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding)
* [Ruby](https://github.com/tree-sitter/ruby-tree-sitter)
* [Haskell](https://github.com/tree-sitter/haskell-tree-sitter)

View file

@ -5,7 +5,7 @@ permalink: using-parsers
# Using Parsers
All of Tree-sitter's parsing functionality is exposed through C APIs. Applications written in higher-level languages can use Tree-sitter via binding libraries like [node-tree-sitter](https://github.com/tree-sitter/node-tree-sitter) or [rust-tree-sitter](https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding), which have their own documentation.
All of Tree-sitter's parsing functionality is exposed through C APIs. Applications written in higher-level languages can use Tree-sitter via binding libraries like [node-tree-sitter](https://github.com/tree-sitter/node-tree-sitter) or [rust-tree-sitter](https://github.com/tree-sitter/tree-sitter/tree/master/lib/binding_rust), which have their own documentation.
This document will describes the general concepts of how to use Tree-sitter, which should be relevant regardless of what language you're using. It also goes into some C-specific details that are useful if you're using the C API directly or are building a new binding to a different language.

View file

@ -4,13 +4,14 @@ description = "Rust bindings to the Tree-sitter parsing library"
version = "0.3.9"
authors = ["Max Brunsfeld <maxbrunsfeld@gmail.com>"]
license = "MIT"
readme = "binding/README.md"
readme = "binding_rust/README.md"
keywords = ["incremental", "parsing"]
categories = ["api-bindings", "parsing", "text-editors"]
build = "binding_rust/build.rs"
include = [
"/binding/*",
"/build.rs",
"/binding_rust/*",
"/Cargo.toml",
"/include/*",
"/src/*",
@ -27,4 +28,4 @@ serde_derive = "1.0"
cc = "1.0"
[lib]
path = "binding/lib.rs"
path = "binding_rust/lib.rs"

View file

@ -40,7 +40,7 @@ fn main() {
.include("include")
.include("utf8proc")
.file(src_path.join("lib.c"))
.file(Path::new("binding").join("helper.c"))
.file(Path::new("binding_rust").join("helper.c"))
.compile("tree-sitter");
}

105
lib/binding_web/README.md Normal file
View file

@ -0,0 +1,105 @@
tree-sitter.wasm
================
[![Build Status](https://travis-ci.org/tree-sitter/tree-sitter.svg?branch=master)](https://travis-ci.org/tree-sitter/tree-sitter)
Wasm bindings to the [Tree-sitter](https://github.com/tree-sitter/tree-sitter) parsing library.
### Basic Usage
You can either load the library as a standalone script:
```html
<script src="/public/js/tree-sitter.js"/>
<script>
const Parser = window.TreeSitter;
Parser.init().then(() => { /* the library is ready */ });
</script>
```
or using a packaging system like Webpack:
```js
const Parser = require('tree-sitter');
Parser.init().then(() => { /* the library is ready */ });
```
Create a parser:
```js
const parser = new Parser;
```
Then assign a language to the parser. Tree-sitter languages are packaged as individual `.wasm` files:
```js
const JavaScript = await Parser.Language.load('/path/to/tree-sitter-javascript.wasm');
parser.setLanguage(JavaScript);
```
Now you can parse source code:
```js
const sourceCode = 'let x = 1; console.log(x);';
const tree = parser.parse(sourceCode);
```
and inspect the syntax tree.
```javascript
console.log(tree.rootNode.toString());
// (program
// (lexical_declaration
// (variable_declarator (identifier) (number)))
// (expression_statement
// (call_expression
// (member_expression (identifier) (property_identifier))
// (arguments (identifier)))))
const callExpression = tree.rootNode.child(1).firstChild;
console.log(callExpression);
// { type: 'call_expression',
// startPosition: {row: 0, column: 16},
// endPosition: {row: 0, column: 30},
// startIndex: 0,
// endIndex: 30 }
```
### Editing
If your source code *changes*, you can update the syntax tree. This will take less time than the first parse.
```javascript
// Replace 'let' with 'const'
const newSourceCode = 'const x = 1; console.log(x);';
tree.edit({
startIndex: 0,
oldEndIndex: 3,
newEndIndex: 5,
startPosition: {row: 0, column: 0},
oldEndPosition: {row: 0, column: 3},
newEndPosition: {row: 0, column: 5},
});
const newTree = parser.parse(newSourceCode, tree);
```
### Parsing Text From a Custom Data Structure
If your text is stored in a data structure other than a single string, you can parse it by supplying a callback to `parse` instead of a string:
```javascript
const sourceLines = [
'let x = 1;',
'console.log(x);'
];
const tree = parser.parse((index, position) => {
let line = sourceLines[position.row];
if (line) return line.slice(position.column);
});
```

View file

@ -9,7 +9,7 @@ if [[ "$1" == "--debug" ]]; then
args="-s ASSERTIONS=1 -s SAFE_HEAP=1 -Os"
fi
exports=$(cat lib/web/exports.json)
exports=$(cat lib/binding_web/exports.json)
mkdir -p target/scratch target/release
@ -33,22 +33,22 @@ docker run \
-I lib/src \
-I lib/include \
-I lib/utf8proc \
--js-library lib/web/imports.js \
--pre-js lib/web/prefix.js \
--post-js lib/web/binding.js \
--js-library lib/binding_web/imports.js \
--pre-js lib/binding_web/prefix.js \
--post-js lib/binding_web/binding.js \
lib/src/lib.c \
lib/web/binding.c \
lib/binding_web/binding.c \
-o target/scratch/tree-sitter.js
if [[ "$minify" == "1" ]]; then
if [ ! -d lib/web/node_modules/terser ]; then
if [ ! -d lib/binding_web/node_modules/terser ]; then
(
cd lib/web
cd lib/binding_web
npm install
)
fi
lib/web/node_modules/.bin/terser \
lib/binding_web/node_modules/.bin/terser \
--compress \
--mangle \
--keep-fnames \

View file

@ -1,6 +1,6 @@
#!/bin/bash
output_path=lib/binding/bindings.rs
output_path=lib/binding_rust/bindings.rs
header_path='lib/include/tree_sitter/api.h'
bindgen \

View file

@ -2,7 +2,7 @@
set -e
cd lib/web
cd lib/binding_web
if [ ! -d "node_modules/chai" ] || [ ! -d "node_modules/mocha" ]; then
echo "Installing test dependencies..."