In lexer, prefer tokens to skipped separator characters

This was causing newlines in go and javascript to be parsed as
meaningless separator characters instead of statement terminators
This commit is contained in:
Max Brunsfeld 2014-05-30 13:29:54 -07:00
parent 220e081c49
commit e93e254518
26 changed files with 5559 additions and 6650 deletions

View file

@ -80,6 +80,16 @@ try {
(statement_block (expression_statement (function_call (identifier) (identifier))))))
===========================================
parses indented code after blocks
===========================================
function x() {}
return z;
---
(program
(expression_statement
(function_expression (identifier) (formal_parameters) (statement_block)))
(return_statement (identifier)))
===========================================
parses switch statements
===========================================
switch(x) {