Commit graph

24 commits

Author SHA1 Message Date
Paul Gey
965e3c9e5e Generator::add_parse_table: Store entries in hash map
This avoids a quadratic behaviour due to repeatedly using `find` on a
growing `Vec`.
2021-08-08 21:45:43 +02:00
Paul Gey
cf69a2c94c Use IndexMap and FxHash for some hot hash maps 2021-08-08 21:45:43 +02:00
Max Brunsfeld
d40f118370 Generalize precedence datatype to include strings
Right now, the strings are not used in comparisons, but they
are passed through the grammar processing pipeline, and are
available to the parse table construction algorithm.

This also cleans up a confusing aspect of the parse table
construction, in which precedences and associativities were
temporarily stored in the parse table data structure itself.
2021-02-23 20:48:39 -08:00
Max Brunsfeld
29bc26ecd5 Fix test failure after non-terminal extras change 2021-02-18 15:43:01 -08:00
Max Brunsfeld
86a891fa63 Fix bugs in parser generation for non-terminal extras
Previously, we attempted to completely separate the parse states
for item sets with non-terminal extras from the parse states
for other rules. But there was not a complete separation.

It actually isn't necessary to separate the parse states in this way.
The only special behavior for parse states with non-terminal extra rules
is what happens at the *end* of the rule: these parse states need to
perform an unconditional reduction.

Luckily, it's possible to distinguish these *non-terminal extra ending*
states from other states just based on their normal structure, with
no additional state.
2021-02-18 14:14:22 -08:00
Max Brunsfeld
d3b7caa565 Add a TSLexer.eof() API, use it in generated parsers 2019-10-31 14:11:52 -07:00
Max Brunsfeld
fcaabea0cf Allow non-terminal extras 2019-10-21 16:08:59 -07:00
Max Brunsfeld
69ab405325 In next ABI, group symbols by action in small parse state table
This is a more compact representation because in most states, many 
symbols share the same actions.
2019-08-30 20:29:55 -07:00
Max Brunsfeld
09a2755399 Store parse states with few lookahead symbols in a more compact way 2019-08-29 15:52:23 -07:00
Max Brunsfeld
48a883c1d4 Move external token state id computation out of render module 2019-08-29 15:48:22 -07:00
Max Brunsfeld
56ce4e5d50 Upgrade rsass, remove hashbrown 2019-08-13 10:08:58 -07:00
Max Brunsfeld
223a656fc8 Fix another bug in lex state merging
Reuse more logic for lex and parse state merging algorithms
2019-06-21 13:12:09 -07:00
Max Brunsfeld
fe6a69a626 Sort lex states 2019-06-20 13:01:28 -07:00
Max Brunsfeld
5b38ff5f78 Loosen lex state equality check to catch some spurious duplicates 2019-06-20 09:57:38 -07:00
Max Brunsfeld
f7d25a5934 Fix missed opportunities to merge parse states 2019-06-06 15:29:22 -07:00
Max Brunsfeld
eb96dd6ddb node types: Preserve all supertypes in field type lists 2019-03-27 16:32:02 -07:00
Max Brunsfeld
6c65d74810 Restructure node-types.json output 2019-03-26 13:43:10 -07:00
Max Brunsfeld
b79bd8693b Start work on handling node supertypes 2019-03-26 11:51:02 -07:00
Max Brunsfeld
56309a1c28 Generate node-fields.json file 2019-02-12 11:06:18 -08:00
Max Brunsfeld
79d90f0d3e Restore naming of alias sequence lengths
Fields aren't stored in sequences now, so the max length
is back to being just for aliases.
2019-02-08 16:14:18 -08:00
Max Brunsfeld
1d1674811c Fully implement ts_node_child_by_field_id 2019-02-08 15:16:56 -08:00
Max Brunsfeld
18a13b457d Get basic field API working 2019-02-08 15:16:56 -08:00
Max Brunsfeld
108ca989ea Start work on including child refs in generated parsers 2019-02-08 15:16:56 -08:00
Max Brunsfeld
f059557a9d Move parser generation code in to 'generate' module within CLI crate 2019-01-07 10:23:01 -08:00
Renamed from cli/src/tables.rs (Browse further)