SQLite Forum

lemon cannot shift-reduce errors
I have been trying to use lemon with the 'error' keyword to enable error recovery, but I've run into something that might be a bug. 

TLDR: Are reduction rules allowed to END with 'error'? If not, disregard the rest of this post.

Given a grammar like this:
    start ::= top.
    top ::= .
    top ::= top error.
    top ::= top topdef.
    ... a bunch of legal topdefs

When the generated parser encounters an error, the loop:
        while( yypParser->yytos >= yypParser->yystack
            && (yyact = yy_find_reduce_action(
                        YYERRORSYMBOL)) > YY_MAX_SHIFTREDUCE
          /* added for debug by me: */ if (yyact >= YY_MIN_REDUCE) printf("CONSIDERING %d: %s\n", yyact - YY_MIN_REDUCE, yyRuleName[yyact-YY_MIN_REDUCE]);
unwinds the stack until it gets an assertion failure due to popping the end of the stack:
    Return. Stack=[top global export KW_DEF]
    Input 'KW_DATA' in state 21
    Syntax Error!
    foo.wake:133:[5-8]: syntax error; found 'data', but was expecting one of:
        if    \    (    subscribe    prim    match
    Popping KW_DEF
    Popping export
    CONSIDERING 41: export ::=
    Popping global
    CONSIDERING 2: top ::= top error
    Popping top
    CONSIDERING 1: top ::=
    parser: parser.c:2096: void yy_pop_parser_stack(yyParser*): Assertion `pParser->yytos > pParser->yystack' failed.

As you can see in the trace of 'yyact', it DID consider the rule `top ::= top error.`, but the number it had assigned to this rule was in the range of `[YY_MIN_REDUCE, YY_MAX_REDUCE)` instead of `[YY_MIN_SHIFTREDUCE, YY_MAX_SHIFTREDUCE)`. Thus, the loop popped past the point it could have shift-reduced. This led me to the work-around of not having 'error' as the last term in the error-handling rule, and that seems to have worked, except that now I cannot tolerate an error at the end of the input, which is quite unfortunate, as people often edit files at the bottom.

I've also noticed that when I try to create a minimal grammar reproducing this behavior, the bug(?) goes away and the reduction works. When I use the '-c' option, the bug(?) also goes away and the error reduction works.

I'm not sure where to post the full grammar that causes this issue. I did include it attached to the original email I sent.