From the document, I noticed that both expression and constant integer are allowed in the Group By clause. I am a bit confused why the following expression is identified as constant integer, instead of expression.

CREATE TABLE t0(c0);
SELECT COUNT(*) FROM t0 ORDER BY (t0.c0 IN ()); 
-- Parse error: 1st ORDER BY term out of range - should be between 1 and 1

As an optimization, the SQLite parser treats expressions of the form "<expr> IN ()" and "<expr> NOT IN ()" as alternative spellings for constants 0 and 1, respectively. See the parse.y sources for details.

Dan has now checked in a change that causes "<expr> IN ()" to be an alternative spelling for "false" instead of "0", which should fix your problem, I think.