Max Brunsfeld
2b3da512a4
Add serialize, deserialize and reset callbacks to external scanners
...
Signed-off-by: Nathan Sobo <nathan@github.com>
2016-12-20 13:12:01 -08:00
Max Brunsfeld
a1770ce844
Allow external tokens to be used as extras
2016-12-12 22:06:01 -08:00
Max Brunsfeld
10b51a05a1
Allow external scanners to refer to (and return) internally-defined tokens
...
Tokens that are defined in the grammar's rules may now be included in the
externals list also, so that external scanners can check if they are valid
lookaheads or not, and if so, can return them to the parser if needed.
2016-12-09 13:32:58 -08:00
Max Brunsfeld
c4fe8ded95
Remove state argument to Lexer advance method
2016-12-05 16:36:34 -08:00
Max Brunsfeld
0f8e130687
Call external scanner functions when lexing
2016-12-02 22:03:48 -08:00
Max Brunsfeld
c966af0412
Start work on external tokens
2016-12-02 16:24:19 -08:00
