Max Brunsfeld
8dd68c360a
Fix logic for generating unique symbol map
...
Previously, this didn't correctly handle the case where *multiple*
symbols were all simply-aliased to the same *other* symbol.
Refs #500
2020-01-27 12:06:48 -08:00
Max Brunsfeld
fc19312913
Fix node-types bugs involving aliases and external tokens
2019-12-12 10:06:18 -08:00
Max Brunsfeld
a5a9000e29
generate: Ensure that field_map_slices array is long enough
2019-12-09 11:46:32 -08:00
Max Brunsfeld
7032dae4f6
Include alias symbols in unique symbol map
2019-12-06 12:11:09 -08:00
Max Brunsfeld
56c620c005
Store a mapping to ensure no two symbols map to the same metadata
2019-12-05 17:21:46 -08:00
Max Brunsfeld
5767bbc806
Avoid generating C char literals with control characters
...
Fixes #487
2019-11-13 10:54:34 -08:00
Max Brunsfeld
d765332c61
Don't rely on new eof ABI in parsers unless --next-abi is passed
2019-10-31 14:32:50 -07:00
Max Brunsfeld
d3b7caa565
Add a TSLexer.eof() API, use it in generated parsers
2019-10-31 14:11:52 -07:00
Max Brunsfeld
fcaabea0cf
Allow non-terminal extras
2019-10-21 16:08:59 -07:00
Max Brunsfeld
69ab405325
In next ABI, group symbols by action in small parse state table
...
This is a more compact representation because in most states, many
symbols share the same actions.
2019-08-30 20:29:55 -07:00
Max Brunsfeld
8037607583
Only generate the new parse table format if --next-abi flag is used
2019-08-29 17:37:33 -07:00
Max Brunsfeld
82ff542d3b
Appease MSVC by avoiding empty arrays
2019-08-29 17:31:44 -07:00
Max Brunsfeld
09a2755399
Store parse states with few lookahead symbols in a more compact way
2019-08-29 15:52:23 -07:00
Max Brunsfeld
48a883c1d4
Move external token state id computation out of render module
2019-08-29 15:48:22 -07:00
Max Brunsfeld
2430733ee8
Avoid iterating hashmaps in places where order matters
2019-08-29 15:26:05 -07:00
Max Brunsfeld
56ce4e5d50
Upgrade rsass, remove hashbrown
2019-08-13 10:08:58 -07:00
Max Brunsfeld
5f369a5870
Fix another empty array literal for MSVC compatibility
2019-08-12 15:13:41 -07:00
Max Brunsfeld
13c0aa7dbb
Avoid empty initializer list for ts_alias_sequences
...
Fixes a bug introduced in 68b089b41e
2019-08-12 14:11:40 -07:00
Max Brunsfeld
68b089b41e
cli: Fix generation of parsers with fields but no aliases
...
Fixes #419
2019-08-11 09:22:30 -07:00
Max Brunsfeld
5b38ff5f78
Loosen lex state equality check to catch some spurious duplicates
2019-06-20 09:57:38 -07:00
Max Brunsfeld
e4873191d6
Refactor generated lex function to use fewer instructions per state
2019-06-20 09:57:38 -07:00
Max Brunsfeld
5035e194ff
Merge branch 'master' into node-fields
2019-03-26 11:58:21 -07:00
Max Brunsfeld
5a59f19b69
Use explicit syntax for functions with no parameters
2019-03-21 16:06:06 -07:00
Max Brunsfeld
56309a1c28
Generate node-fields.json file
2019-02-12 11:06:18 -08:00
Max Brunsfeld
79d90f0d3e
Restore naming of alias sequence lengths
...
Fields aren't stored in sequences now, so the max length
is back to being just for aliases.
2019-02-08 16:14:18 -08:00
Max Brunsfeld
d8a2c0dda2
Use a separate type for storing field map headers
2019-02-08 16:06:29 -08:00
Max Brunsfeld
1d1674811c
Fully implement ts_node_child_by_field_id
2019-02-08 15:16:56 -08:00
Max Brunsfeld
18a13b457d
Get basic field API working
2019-02-08 15:16:56 -08:00
Max Brunsfeld
108ca989ea
Start work on including child refs in generated parsers
2019-02-08 15:16:56 -08:00
Max Brunsfeld
4badd7cc40
Disable compiler optimizations for lex functions in more cases
...
* Reduce the lexer state count threshold from 500 to 300
* Disable optimizations on clang and gcc in addition to MSVC
Optimizations in these source files don't seem to make any impact on
parsing performance, but they slow down compile time substantially.
2019-02-06 11:50:37 -08:00
Max Brunsfeld
ed195de8b6
rustfmt
2019-01-17 17:16:04 -08:00
Max Brunsfeld
19b2addcc4
Fix bug in symbol enum code generation
2019-01-14 14:08:07 -08:00
Max Brunsfeld
2e009f7177
Avoid writing empty initializer list for alias sequences
2019-01-12 21:57:34 -08:00
Max Brunsfeld
545e840a08
Remove stray single quotes in symbol name strings
2019-01-12 21:42:31 -08:00
Max Brunsfeld
c76a155174
Fix escaping of characters in C strings
2019-01-11 17:43:27 -08:00
Max Brunsfeld
6592fdd24c
Fix parser generation error messages
2019-01-11 17:26:45 -08:00
Max Brunsfeld
f059557a9d
Move parser generation code in to 'generate' module within CLI crate
2019-01-07 10:23:01 -08:00
