Max Brunsfeld
ffd3bdc4c1
Escape ? in C string literals
...
Fixes #714
2020-09-23 13:06:06 -07:00
Max Brunsfeld
5003064da7
Make supertypes automatically hidden, without underscore prefix
2020-09-23 09:35:14 -07:00
Max Brunsfeld
b5a9adb555
Allow queries to match on supertypes
...
Co-authored-by: Ayman Nadeem <aymannadeem@github.com>
2020-09-21 12:34:48 -07:00
Max Brunsfeld
ff488f89c9
Make the --prev-abi flag work w/ the newest abi change
2020-09-08 10:58:20 -07:00
Max Brunsfeld
2eb04094f8
Handle aliased parent nodes in query analysis
2020-08-21 14:12:04 -07:00
Max Brunsfeld
1ea29053e1
Merge branch 'master' into query-pattern-is-definite
2020-08-14 09:31:55 -07:00
Max Brunsfeld
81bbdf19f4
Fix handling of non-terminal extras that share non-extra rules
...
Fixes #701
2020-07-29 09:50:13 -07:00
Max Brunsfeld
32099050d6
node_types: Fix panic when field is associated with a hidden token
...
Fixes #695
2020-07-24 09:26:56 -07:00
Max Brunsfeld
82aa1462fd
Clean up get_variable_info function
2020-07-17 15:12:13 -07:00
Max Brunsfeld
c4fca5f73e
node types: Fix handling of repetitions inside of fields
...
Fixes #676
2020-07-17 14:19:59 -07:00
Max Brunsfeld
f4adf0269a
Propagate dynamic precedence correctly for inlined rules
...
Fixes #683
2020-07-17 09:53:01 -07:00
Max Brunsfeld
4c2f36a07b
Mark steps as definite on query construction
...
* Add a ts_query_pattern_is_definite API, just for debugging this
* Store state_count on TSLanguage structs, to allow for scanning parse tables
2020-06-25 15:06:27 -07:00
Max Brunsfeld
a6f71328fe
Avoid whitelist/blacklist terminology in test comments
2020-06-16 09:22:34 -07:00
Max Brunsfeld
ec870e9e66
Avoid extracting helpers for char sets that are only used once
2020-05-26 16:37:45 -07:00
Max Brunsfeld
911fb7f1b2
Extract helper functions to reduce the code size of the lexer function ( #626 )
...
* Extract helper functions to reduce code size of ts_lex
* Name char set helper functions based on token name
2020-05-26 13:39:11 -07:00
Max Brunsfeld
9d182bb078
node-types: Fix bug w/ required property when multiple rules aliased as same
2020-05-14 10:51:18 -07:00
Max Brunsfeld
b66d149b74
Fix inconsistent whitespace after '{' in generated parser
2020-05-13 15:56:49 -07:00
Max Brunsfeld
cdc973866f
Fix build-wasm command on latest emscripten
2020-05-12 15:42:11 -07:00
Riccardo Schirone
780e9cecc9
Do not use multiple unnamed structs inside of unions
2020-04-29 20:42:45 +02:00
Max Brunsfeld
a003e5f6bd
generate: Avoid duplicate string tokens in unique symbol map
2020-03-20 11:35:11 -07:00
Max Brunsfeld
6cb8d24de2
Merge pull request #542 from SKalt/issue-524-document-supertypes-in-grammar-schema
...
feat(cli): documented optional supertypes string[] in grammar schema
2020-02-24 16:14:49 -08:00
Steven Kalt
d82ee739e9
Update cli/src/generate/grammar-schema.json
...
Co-Authored-By: Max Brunsfeld <maxbrunsfeld@github.com>
2020-02-24 18:13:38 -05:00
Alyssa Verkade
0e689657b7
Add a language linkage declaration to parsers
...
Previously, in order to compile a `tree-sitter` grammar that contained
c++ source in the parser (ie the `scanner.cc` file), you would have to
compile the `parser.c` file separately from the c++ files. For example,
in rust this would result in a `build.rs` close to the following:
```
extern crate cc;
fn main() {
let dir: PathBuf = ["tree-sitter-ruby", "src"].iter().collect();
cc::Build::new()
.include(&dir)
.cpp(true)
.file(dir.join("scanner.cc"))
// NOTE: must have a name that differs from the c static lib
.compile("tree-sitter-ruby-scanner");
cc::Build::new()
.include(&dir)
.file(dir.join("parser.c"))
// NOTE: must have a name that differs from the c++ static lib
.compile("tree-sitter-ruby-parser");
}
```
This was necessary at the time for the following grammars: `ruby`,
`php`, `python`, `embedded-template`, `html`, `cpp`, `ocaml`,
`bash`, `agda`, and `haskell`.
To solve this, we specify an `extern "C"` language linkage declaration
to the functions that must be linked against to compile a parser with the
scanner, making parsers linkable against c++ source.
On all major compilers (gcc, clang, and msvc) this should be the only
change needed due to the combination of clang and gcc both supporting
designated initialization for years and msvc 2019 adopting designated
initializers as a part of the C++20 conformance push.
Subsequently, for rust projects, the necessary `build.rs` would become
(which also brings these parsers into sync with the current docs):
```
extern crate cc;
fn main() {
let dir: PathBuf = ["tree-sitter-ruby", "src"].iter().collect();
cc::Build::new()
.include(&dir)
.cpp(true)
.file(dir.join("scanner.cc"))
.file(dir.join("parser.c"))
.compile("tree-sitter-ruby");
}
```
2020-02-18 19:46:59 -08:00
Steven Kalt
c5ca259d09
feat(cli): documented optional supertypes Array<string> in grammar-schema.json
2020-02-15 11:23:14 -05:00
Max Brunsfeld
5922064b75
node-types: Fix ambiguity warning on rustc 1.41
2020-02-10 10:26:12 -08:00
Max Brunsfeld
8dd68c360a
Fix logic for generating unique symbol map
...
Previously, this didn't correctly handle the case where *multiple*
symbols were all simply-aliased to the same *other* symbol.
Refs #500
2020-01-27 12:06:48 -08:00
Steven Kalt
619d7cd65a
added field-rule to grammar-schema.json
2020-01-12 19:40:21 -05:00
Steven Kalt
e69430ae7d
removed duplicate key ( #521 )
...
The key "required" was duplicated on "symbol-rule". I removed the more permissive copy.
2020-01-09 17:34:07 -08:00
Max Brunsfeld
f0e4d630d5
Reliably avoid duplicate subtypes in node-types.json
2019-12-17 14:55:51 -08:00
Max Brunsfeld
3ac047679c
Fix children.required when multiple nodes are aliased as the same
2019-12-16 12:39:57 -08:00
Max Brunsfeld
7c711c5537
Move the private functions in node_types.rs to the end of the file
2019-12-16 12:38:40 -08:00
Max Brunsfeld
9a0cfa2376
Fix 'required' field when rules with diff fields are aliased to look the same
2019-12-12 11:26:58 -08:00
Max Brunsfeld
fc19312913
Fix node-types bugs involving aliases and external tokens
2019-12-12 10:06:18 -08:00
Max Brunsfeld
1b5ae380ce
Don't attempt to extract keywords that don't entirely match word token ( #505 )
2019-12-11 17:18:15 -08:00
Max Brunsfeld
a5a9000e29
generate: Ensure that field_map_slices array is long enough
2019-12-09 11:46:32 -08:00
Max Brunsfeld
7032dae4f6
Include alias symbols in unique symbol map
2019-12-06 12:11:09 -08:00
Max Brunsfeld
56c620c005
Store a mapping to ensure no two symbols map to the same metadata
2019-12-05 17:21:46 -08:00
Max Brunsfeld
5a979d1457
node-types: Add test for field merging with aliases
...
Co-Authored-By: Timothy Clem <timothy.clem@gmail.com>
2019-11-19 14:31:56 -08:00
Max Brunsfeld
1d63434664
Refactor node-type merging
2019-11-19 11:57:21 -08:00
Max Brunsfeld
5489bc4dc5
Fix small issues with merging node types
...
* Merge the `required` field with an 'and', not an 'or'
* Merge field info in addition to children info
2019-11-19 11:55:21 -08:00
Timothy Clem
e2325102d1
No printing
2019-11-19 09:03:35 -08:00
Timothy Clem
c346ce4a5e
Try not to loose existing children
2019-11-18 14:48:24 -08:00
Timothy Clem
5c72642634
A test demonstrating the issue with named aliases
2019-11-18 14:47:48 -08:00
Max Brunsfeld
660efd8c0f
Fix handling of token rules directly inside of alias, prec, or field.
...
Fixes #483
2019-11-15 10:53:17 -08:00
Max Brunsfeld
d45b98c7d9
node-types - Fix duplicate when a terminal is aliased with the same name as a non-terminal
2019-11-14 09:31:25 -08:00
Max Brunsfeld
5767bbc806
Avoid generating C char literals with control characters
...
Fixes #487
2019-11-13 10:54:34 -08:00
Max Brunsfeld
a2bbc7391b
node-types: Fix handling of simple aliased inlined rules
2019-11-13 10:38:47 -08:00
Max Brunsfeld
d765332c61
Don't rely on new eof ABI in parsers unless --next-abi is passed
2019-10-31 14:32:50 -07:00
Max Brunsfeld
d3b7caa565
Add a TSLexer.eof() API, use it in generated parsers
2019-10-31 14:11:52 -07:00
Max Brunsfeld
0cceca7b4e
Rename extra_tokens -> extra_symbols
2019-10-21 17:26:01 -07:00
