traxys/tree-sitter
1
0
Fork 0
Code Issues Pull requests Projects Releases Packages Wiki Activity Actions

Fix bugs in parser generation for non-terminal extras

Browse source 
Previously, we attempted to completely separate the parse states
for item sets with non-terminal extras from the parse states
for other rules. But there was not a complete separation.

It actually isn't necessary to separate the parse states in this way.
The only special behavior for parse states with non-terminal extra rules
is what happens at the *end* of the rule: these parse states need to
perform an unconditional reduction.

Luckily, it's possible to distinguish these *non-terminal extra ending*
states from other states just based on their normal structure, with
no additional state.
This commit is contained in:
Max Brunsfeld 2021-02-18 14:14:20 -08:00
parent b46d51f224
commit 86a891fa63
4 changed files with 50 additions and 28 deletions
Download patch file Download diff file Expand all files Collapse all files

5
cli/src/generate/render.rs
View file

@ -970,10 +970,7 @@ impl Generator {
add_line!(self, "static TSLexMode ts_lex_modes[STATE_COUNT] = {{");
indent!(self);
for (i, state) in self.parse_table.states.iter().enumerate() {
if state.is_non_terminal_extra
&& state.terminal_entries.len() == 1
&& *state.terminal_entries.iter().next().unwrap().0 == Symbol::end()
{
if state.is_end_of_non_terminal_extra(&self.syntax_grammar) {
add_line!(self, "[{}] = {{(TSStateId)(-1)}},", i,);
} else if state.external_lex_state_id > 0 {
add_line!(