Max Brunsfeld
3498498449
Merge branch 'master' into simpler-large-char-set-code
2024-04-12 10:03:46 -07:00
Amaan Qureshi
b35efa8f33
style: format imports
2024-04-11 22:35:43 -04:00
Max Brunsfeld
3d088888f5
Derive large character sets from lex states for individual tokens
2024-04-10 16:53:39 -07:00
Max Brunsfeld
be34bc9430
Identify large char sets for lexer using NFA transitions
2024-04-09 17:53:37 -07:00
Amaan Qureshi
754aa62e89
chore: turbofish styling
2024-03-17 05:36:30 -04:00
Amaan Qureshi
59be1edaa1
refactor: swap &Vec[T] with &[T] where appropriate
2024-02-07 02:50:31 -05:00
Amaan Qureshi
04ff704bca
chore(cli): apply clippy fixes
2024-02-04 04:18:48 -05:00
Douglas Creager
d2d01e77e3
cli: Use anyhow and thiserror for errors
...
This patch updates the CLI to use anyhow and thiserror for error
management. The main feature that our custom `Error` type was providing
was a _list_ of messages, which would allow us to annotate "lower-level"
errors with more contextual information. This is exactly what's
provided by anyhow's `Context` trait.
(This is setup work for a future PR that will pull the `config` and
`loader` modules out into separate crates; by using `anyhow` we wouldn't
have to deal with a circular dependency between with the new crates.)
2021-06-09 16:17:23 -04:00
Max Brunsfeld
ab78ab3f9b
Represent CharacterSet internally as a vector of ranges
2021-01-28 16:10:39 -08:00
Max Brunsfeld
1b5ae380ce
Don't attempt to extract keywords that don't entirely match word token ( #505 )
2019-12-11 17:18:15 -08:00
Max Brunsfeld
aeb2f895b4
Add --report-states flag for reporting state counts for each rule
2019-08-29 17:36:39 -07:00
Max Brunsfeld
48a883c1d4
Move external token state id computation out of render module
2019-08-29 15:48:22 -07:00
Max Brunsfeld
f371507d39
Move TokenSet to rules module
2019-08-29 15:25:45 -07:00
Max Brunsfeld
f4740a1beb
Make properties generation aware of which nodes are leaves
2019-07-22 09:30:05 -07:00
Max Brunsfeld
0a3f2131c8
Move state splitting algorithm into its own file
2019-07-19 12:39:52 -07:00
Max Brunsfeld
f85ce2fbfa
Remove state-ids-to-log flag
2019-07-18 16:06:13 -07:00
Max Brunsfeld
223a656fc8
Fix another bug in lex state merging
...
Reuse more logic for lex and parse state merging algorithms
2019-06-21 13:12:09 -07:00
Max Brunsfeld
eb96dd6ddb
node types: Preserve all supertypes in field type lists
2019-03-27 16:32:02 -07:00
Max Brunsfeld
e9afdd72b4
node-types: Fix incorrect named value for subclasses
2019-03-26 15:27:07 -07:00
Max Brunsfeld
9b8bf8dfe2
cli: Compute fragile tokens *after* merging compatibile parse states
...
Previously, we failed to mark as fragile some tokens that *should* be
fragile because of tokens that were introduced during parse state
merging.
2019-02-06 16:18:49 -08:00
Max Brunsfeld
4cac85fec4
Add benchmark script
...
* Structure `cli` crate as both a library and an executable, so that
benchmarks can import code from the crate.
* Import macros in the Rust 2018 style.
2019-02-01 15:17:35 -08:00
Max Brunsfeld
e305012b31
Loosen keyword identification criteria slightly
2019-01-21 15:33:43 -08:00
Max Brunsfeld
f6cdd5e3d4
Loosen criteria for identifying conflict-free tokens for error recovery
2019-01-20 16:58:31 -08:00
Max Brunsfeld
c27f776d41
Fix word token index issue in a different way
...
Refs https://github.com/tree-sitter/tree-sitter/issues/258
2019-01-17 13:18:59 -08:00
Max Brunsfeld
9f7079c9c5
Ensure that the word token has a low numerical index
...
Fixes https://github.com/tree-sitter/tree-sitter/issues/258
2019-01-17 12:44:14 -08:00
Max Brunsfeld
e4b9d9dfa9
Fix token conflict detection bugs
2019-01-16 20:57:31 -08:00
Max Brunsfeld
6592fdd24c
Fix parser generation error messages
2019-01-11 17:26:45 -08:00
Max Brunsfeld
e64f7a64a1
Start work on running test corpus tests
2019-01-11 13:31:06 -08:00
Max Brunsfeld
2e8b2ab8fb
Give strings more implicit precedence than immediate tokens
2019-01-09 09:59:46 -08:00
Max Brunsfeld
f059557a9d
Move parser generation code in to 'generate' module within CLI crate
2019-01-07 10:23:01 -08:00
