Max Brunsfeld
51a9f14f7d
Ensure symbols are written in a predictable order in conflict messages
2019-02-08 15:16:56 -08:00
Max Brunsfeld
1d1674811c
Fully implement ts_node_child_by_field_id
2019-02-08 15:16:56 -08:00
Max Brunsfeld
18a13b457d
Get basic field API working
2019-02-08 15:16:56 -08:00
Max Brunsfeld
108ca989ea
Start work on including child refs in generated parsers
2019-02-08 15:16:56 -08:00
Max Brunsfeld
bf4e1304f8
Start work on new ref API, for giving names to nodes' children
...
Co-Authored-By: Ayman Nadeem <aymannadeem@gmail.com>
2019-02-08 15:16:56 -08:00
Max Brunsfeld
dac13af206
cli: Always write parser.h in generate command
2019-02-08 15:16:38 -08:00
Max Brunsfeld
acd30d4693
Write grammar.json file when generating based on grammar.js
2019-02-08 15:15:47 -08:00
Max Brunsfeld
9b8bf8dfe2
cli: Compute fragile tokens *after* merging compatibile parse states
...
Previously, we failed to mark as fragile some tokens that *should* be
fragile because of tokens that were introduced during parse state
merging.
2019-02-06 16:18:49 -08:00
Max Brunsfeld
4badd7cc40
Disable compiler optimizations for lex functions in more cases
...
* Reduce the lexer state count threshold from 500 to 300
* Disable optimizations on clang and gcc in addition to MSVC
Optimizations in these source files don't seem to make any impact on
parsing performance, but they slow down compile time substantially.
2019-02-06 11:50:37 -08:00
Max Brunsfeld
5c925b3868
cli: Check that extra rules aren't passed to functions like 'optional'
...
Fixes #247
2019-02-06 09:31:32 -08:00
Max Brunsfeld
d7bc61cada
Include parser header content in library crate
2019-02-05 11:59:34 -08:00
Max Brunsfeld
4a98f0b87e
Port unit test for missing tokens and included ranges
2019-02-04 14:44:06 -08:00
Max Brunsfeld
e143710f4a
Move properties module out of generate
2019-02-01 19:57:00 -08:00
Max Brunsfeld
4cac85fec4
Add benchmark script
...
* Structure `cli` crate as both a library and an executable, so that
benchmarks can import code from the crate.
* Import macros in the Rust 2018 style.
2019-02-01 15:17:35 -08:00
Max Brunsfeld
f6d014f3f4
Write tree_sitter/parser.h file in generate command
2019-01-28 14:23:41 -08:00
Max Brunsfeld
e305012b31
Loosen keyword identification criteria slightly
2019-01-21 15:33:43 -08:00
Max Brunsfeld
f6cdd5e3d4
Loosen criteria for identifying conflict-free tokens for error recovery
2019-01-20 16:58:31 -08:00
Max Brunsfeld
ff41f05a20
Fix computation of following tokens
2019-01-18 15:16:20 -08:00
Max Brunsfeld
31bdf5eb97
Fix handling of JavaScript errors
...
Refs #258
2019-01-18 09:40:09 -08:00
Max Brunsfeld
ed195de8b6
rustfmt
2019-01-17 17:16:04 -08:00
Max Brunsfeld
c27f776d41
Fix word token index issue in a different way
...
Refs https://github.com/tree-sitter/tree-sitter/issues/258
2019-01-17 13:18:59 -08:00
Max Brunsfeld
8f4096e5cb
Give more informative error messages when failing to write files
2019-01-17 12:52:05 -08:00
Max Brunsfeld
bb5dedfb1e
Fix another token conflict detection bug
2019-01-17 12:44:35 -08:00
Max Brunsfeld
9f7079c9c5
Ensure that the word token has a low numerical index
...
Fixes https://github.com/tree-sitter/tree-sitter/issues/258
2019-01-17 12:44:14 -08:00
Max Brunsfeld
d52a11fd03
Avoid using a string literal to pass grammar path to JS
...
Backslashes in windows path were getting interpeted as escape characters.
2019-01-17 10:09:03 -08:00
Max Brunsfeld
d903371709
Remove noisy logging
2019-01-17 10:07:58 -08:00
Max Brunsfeld
e4b9d9dfa9
Fix token conflict detection bugs
2019-01-16 20:57:31 -08:00
Max Brunsfeld
a0a3903f76
Generate binding.gyp, binding.cc, and index.js
2019-01-16 13:53:01 -08:00
Max Brunsfeld
d23a03bdf1
Represent ParseItemSet as a sorted Vec, not a BTreeMap
2019-01-15 16:37:54 -08:00
Max Brunsfeld
ceff3936ef
Unify logic for handling tokens that match separators into one place
2019-01-15 16:10:52 -08:00
Max Brunsfeld
522021b107
Fix NFA generation w/ nested groups
2019-01-15 15:57:29 -08:00
Max Brunsfeld
d8ab36b2a5
Fix bugs in handling tokens that overlap with separators
2019-01-15 13:21:48 -08:00
Max Brunsfeld
b799b46f79
Handle repetition range operators with commas in regexes
2019-01-15 13:21:48 -08:00
Max Brunsfeld
a8292f4fe9
Load all fixture grammars dynamically
...
This way the build doesn't take forever any time a single grammar has
been regenerated.
2019-01-15 13:21:48 -08:00
Max Brunsfeld
19b2addcc4
Fix bug in symbol enum code generation
2019-01-14 14:08:07 -08:00
Max Brunsfeld
def5884b59
Allow passing grammar JS or JSON path to generate command
2019-01-14 14:07:42 -08:00
Max Brunsfeld
e2717a6ad1
Preprocess regexes to allow non-standard escape sequences
...
Also allow unescaped curly braces to match literal curly braces when
they don't form a valid repetition operator.
2019-01-14 14:05:19 -08:00
Max Brunsfeld
2e009f7177
Avoid writing empty initializer list for alias sequences
2019-01-12 21:57:34 -08:00
Max Brunsfeld
545e840a08
Remove stray single quotes in symbol name strings
2019-01-12 21:42:31 -08:00
Max Brunsfeld
6f242fda0c
Fix edge case in flatten rule
2019-01-11 17:43:42 -08:00
Max Brunsfeld
c76a155174
Fix escaping of characters in C strings
2019-01-11 17:43:27 -08:00
Max Brunsfeld
1468b349b5
Ensure 'src' directory exists before writing src/parser.c
2019-01-11 17:39:16 -08:00
Max Brunsfeld
6592fdd24c
Fix parser generation error messages
2019-01-11 17:26:45 -08:00
Max Brunsfeld
e64f7a64a1
Start work on running test corpus tests
2019-01-11 13:31:06 -08:00
Max Brunsfeld
cffe80bfad
Fix tie-breaking via cascade ordering in property sheets
2019-01-11 09:48:19 -08:00
Max Brunsfeld
272046a250
Reorganize tests - move them all into the CLI crate
2019-01-10 17:11:57 -08:00
Max Brunsfeld
38417fc8a1
Port over remaining tests about property sheet generation
2019-01-10 15:17:38 -08:00
Max Brunsfeld
1dfbe495ed
Get property sheet compilation working
2019-01-10 13:12:16 -08:00
Max Brunsfeld
6bd550ca87
Start work on property sheet compilation
2019-01-09 18:10:32 -08:00
Max Brunsfeld
c0fad8b3c4
Write parser.c in generate command
2019-01-09 14:43:49 -08:00
