SQLite Forum

BUG REPORT: format() rounding errors using %f or %g with 0 precision
Login

BUG REPORT: format() rounding errors using %f or %g with 0 precision

(1.3) By Mark Brand (mabrand) on 2024-02-17 09:42:47 edited from 1.2 [source]

Dear SQLite developers,

From version 3.43, I see what appears to be incorrect rounding when using the f and g conversions with precision 0 in the format() function:


> SELECT format('%0.0f', 0.9);
--SQLite version 3.42.0
1 --ok
--SQLite version 3.45.1
0 --wrong


> SELECT format('%0.0g', 0.09);
--SQLite version 3.42.0
0.09 --ok
--SQLite version 3.45.1
0.08 -- wrong

> SELECT format('%0.0g', 1.9);
--SQLite version 3.42.0
2 --ok
--SQLite version 3.45.1
1 --wrong

Regarding the %g conversion, there is the additional question of what 0 precision means. According to https://sqlite.org/printf.html:

"For floating-point substitutions (%e, %E, %f, %g, %G) the precision specifies the number of digits to display to the right of the decimal point."

I assume this to be a documentation error because, in practice, SQLite %g appears to follow C printf %g where precision specifies the number of significant digits.

Given that precision for %g specifies the number of significant digits, the meaning of 0 precision may not be obvious, but at least this source explicitly states that 0 precision is treated the same as 1:
  https://en.cppreference.com/w/c/io/fprintf
SQLite agrees, except for "buggy" cases like the ones above.

(2) By Richard Hipp (drh) on 2024-02-17 03:39:42 in reply to 1.2 [link] [source]

Should be fixed by check-in 7fca1bc482fc2456. Please verify that rounding is now working correctly for zero-precision %f and %g conversions and post a follow-up if there are any further issues.

(3.1) By Mark Brand (mabrand) on 2024-02-17 15:44:56 edited from 3.0 in reply to 2 [link] [source]

Thanks! That fixes it as far as I can tell.

Do you agree https://sqlite.org/printf.html section 2.4 needs a correction with respect to the g conversion specifier? (Precision specifies significant digits, not places after the decimal point.)

(4) By Larry Brasfield (larrybr) on 2024-02-17 11:00:16 in reply to 3.0 [link] [source]

That's not quite the right term, either. The term, "significant digits", can and often does refer to digits with more weight, to the left of the radix.

Rewording suggestions are welcome.

(5.6) By Mark Brand (mabrand) on 2024-03-11 09:14:55 edited from 5.5 in reply to 4 [link] [source]

"The precision specification is used to control ... the number of significant digits in the g and G Conversions."
C, A Reference Manual, Fourth Edition, 15.11.5 Precision

"This [precision] gives ...  the maximum number of significant digits for g and G conversions"
printf man page
https://linux.die.net/man/3/printf

As noted earlier, https://en.cppreference.com/w/c/io/fprintf covers the case of 0 precision.

Based on the above, I would suggest the following wording for
https://sqlite.org/printf.html#the_optional_precision_field

For floating-point substitutions (%e, %E, %f) the precision specifies the number of digits to display to the right of the decimal point.

For floating-point substitutions (%g, %G) the precision specifies the number of significant digits. 0 is treated as 1.

Any objections to this?

(6) By Mark Brand (mabrand) on 2024-03-11 09:15:53 in reply to 5.6 [link] [source]

Just pinging this since it is an outstanding documentation bug. Please see the suggested correction in the previous post.