SQLite User Forum

JSON numbers order by
Login

JSON numbers order by

(1) By Ben (ben123) on 2022-10-13 16:06:51 [link] [source]

It looks like SQLite doesn't use numeric-ordering when extracting a number from JSON. Could it?

SQLite version 3.39.4 2022-09-29 15:55:41
Enter ".help" for usage hints.
sqlite> SELECT json -> '$."6"', json_type(json -> '$."6"') from testjson;
0|integer
-1|integer
-1000|integer
534|integer
sqlite> SELECT json -> '$."6"', json_type(json -> '$."6"') from testjson ORDER BY (json -> '$."6"') ASC;
-1|integer
-1000|integer
0|integer
534|integer
sqlite> SELECT json -> '$."6"', json_type(json -> '$."6"') from testjson ORDER BY (json -> '$."6"') + 0 ASC;
-1000|integer
-1|integer
0|integer
534|integer

(2) By David Raymond (dvdraymond) on 2022-10-13 16:39:21 in reply to 1 [link] [source]

Have you tried using ->> to get the actual number, and not text?

The -> and ->> operators

(3) By Ben (ben123) on 2022-10-13 17:21:21 in reply to 2 [link] [source]

Thanks, that works!

(4) By Ben (ben123) on 2022-10-13 18:35:54 in reply to 3 [link] [source]

Unfortunately, it looks like it works the opposite way for Postgresql:

test_django=# SELECT id, value FROM model_fields_nullablejsonmodel ORDER BY value -> 'ord' ASC;
 id |             value            
----+------------------------------- 96 | {"ord": -1000, "name": "bot"} 95 | {"ord": -1, "name": "baz"} 94 | {"ord": 22.1, "name": "foo"} 93 | {"ord": 93, "name": "bar"} (4 rows)

test_django=# SELECT id, value FROM model_fields_nullablejsonmodel ORDER BY value ->> 'ord' ASC; id | value
----+------------------------------- 95 | {"ord": -1, "name": "baz"} 96 | {"ord": -1000, "name": "bot"} 94 | {"ord": 22.1, "name": "foo"} 93 | {"ord": 93, "name": "bar"} (4 rows)

(5) By RandomCoder on 2022-10-13 18:51:39 in reply to 4 [link] [source]

It appears SQLite is treating JSON numbers as strings, at least when it comes to sorting them:

sqlite> create table test(value);
sqlite> insert into test values ('{"ord":3}'), ('{"ord":100}'), ('{"ord":20}');
sqlite> select value from test order by value -> 'ord' asc;
sqlite> select value from test order by value -> 'ord' asc;
{"ord":100}
{"ord":20}
{"ord":3}
sqlite> select value from test order by cast(value -> 'ord' as int) asc;
{"ord":3}
{"ord":20}
{"ord":100}

(6) By Richard Hipp (drh) on 2022-10-13 19:05:53 in reply to 4 [link] [source]

No. The ->> operator in PG always gives a TEXT result. (See https://www.postgresql.org/docs/15/functions-json.html.) Your ->> output is being sorted in in TEXT order, not in numeric order. The -> operator in PG always returns JSON. I don't know what PG considers JSON sort order to be, but I suspect that if you added a new field in front of "ord", it would mess up your sort.

The fact that the PG ->> operator always returns TEXT, rather than the type of the underlying JSON object, is a limitation in PG that derives from its rigid type system. In PG, the return type of an operator is always determined by the type of its operands, so that the type can be computed at compile-time. To my mind, this is a bug in PG.

SQLite computes datatypes at run-time, not a compile-time. Hence, the ->> operator in SQLite can (and does) return values of different types depending on the values of its operands, not just the datatype of its operands. Hence, SQLite really is able to return an INT or REAL or TEXT value from ->> if the field being selected is an integer or real or text value.

(7) By Richard Hipp (drh) on 2022-10-13 19:06:59 in reply to 5 [link] [source]

Yes. SQLite sorts JSON values according to the string representation of the JSON. How else do you propose to sort JSON?

(8) By David Raymond (dvdraymond) on 2022-10-13 19:11:09 in reply to 5 [link] [source]

Inexperienced summary that's probably got wrong terminology and could be put better.

Postgres:
Strongly typed. It has an actual json type, and functions need to know what type they're going to return before they're called.

json -> [int or text] returns another json object.
json ->> [int or text] returns text, because it has to know the type, and can't sometimes return text, sometimes return an int, etc. If you the human know what type it is you can manually cast it from there.

SQLite:
More dynamically typed. Only has text, int, real, blob as actual types. There is no json type, only text that's in a valid json format.

<valid json text> -> [int or text] returns <valid json text>  This is as close as it can get to Postgres without having a json type. It will return valid json text that can go right into any other json function.
<valid json text> ->> [int or text] returns whatever type that element is. Because it's dynamically typed this can be an actual int, float, text, anything.

(9) By Ben (ben123) on 2022-10-13 19:35:25 in reply to 6 [source]

The -> operator in PG always returns JSON. I don't know what PG considers JSON sort order to be, but I suspect that if you added a new field in front of "ord", it would mess up your sort.

I tried that, and here are the results. It seems to be able to sort by 'ord' as numbers?

test_django=# create table test_jsonb (id INT PRIMARY KEY, data JSONB);
CREATE TABLE
test_django=# INSERT INTO test_jsonb (id, data) VALUES (1, '{"a": 1, "ord": 93}');
INSERT 0 1
test_django=# INSERT INTO test_jsonb (id, data) VALUES (2, '{"a": 2, "ord": 22.1}');
INSERT 0 1
test_django=# INSERT INTO test_jsonb (id, data) VALUES (3, '{"a": 3, "ord": -1}');
INSERT 0 1
test_django=# INSERT INTO test_jsonb (id, data) VALUES (4, '{"a": 4, "ord": -1000}');
INSERT 0 1
test_django=# SELECT id,data FROM test_jsonb ORDER BY data -> 'ord' ASC;
 id |          data          
----+------------------------
  4 | {"a": 4, "ord": -1000}
  3 | {"a": 3, "ord": -1}
  2 | {"a": 2, "ord": 22.1}
  1 | {"a": 1, "ord": 93}
(4 rows)

test_django=# SELECT id,data FROM test_jsonb ORDER BY data ->> 'ord' ASC;
 id |          data          
----+------------------------
  3 | {"a": 3, "ord": -1}
  4 | {"a": 4, "ord": -1000}
  2 | {"a": 2, "ord": 22.1}
  1 | {"a": 1, "ord": 93}
(4 rows)

(10.1) By David Raymond (dvdraymond) on 2022-10-13 20:11:33 edited from 10.0 in reply to 6 [link] [source]

In Postgres, json has no sort order, if you try you get

ERROR: could not identify an ordering operator for type json

jsonb though does have a sort order that's listed at the bottom of the jsonb Indexing section

jsonb also supports btree and hash indexes. These are usually useful only if it's important to check equality of complete JSON documents. The btree ordering for jsonb datums is seldom of great interest, but for completeness it is:

Object > Array > Boolean > Number > String > Null

Object with n pairs > object with n - 1 pairs

Array with n elements > array with n - 1 elements

Objects with equal numbers of pairs are compared in the order:

key-1, value-1, key-2 ...

Note that object keys are compared in their storage order; in particular, since shorter keys are stored before longer keys, this can lead to results that might be unintuitive, such as:

{ "aa": 1, "c": 1} > {"b": 1, "d": 1}

Similarly, arrays with equal numbers of elements are compared in the order:

element-1, element-2 ...

Primitive JSON values are compared using the same comparison rules as for the underlying PostgreSQL data type. Strings are compared using the default database collation.

(11) By Richard Hipp (drh) on 2022-10-13 20:12:42 in reply to 10.0 [link] [source]

In Postgres, json has no sort order, if you try you get "ERROR: could not identify an ordering operator for type json"

That's not what is happening for Ben in post #6 above. He has an ORDER BY on the result of the -> operator (which is JSON according to the documentation) and it appears to work and give the answer that he wants.

(12) By David Raymond (dvdraymond) on 2022-10-13 20:31:38 in reply to 11 [link] [source]

In his example he declared his data field as jsonb.

and jsonb -> [text or int] gives jsonb

So when he did ORDER BY data -> 'ord' ASC it went through and said

1: Compare the types of the jsonb values. (All of these are NUMBERs, so nothing sorts before or after each other based on that.)

2: Compare each of these NUMBERs using the numeric data type.

Thus it came out in nice numeric order. If 'ord' had any text or other types of values the weird ordering would have shown up.

(13) By Ben (ben123) on 2022-10-13 21:01:48 in reply to 12 [link] [source]

Yup, sounds right.

And I did get the "ERROR: could not identify an ordering operator for type json" issue when I declared the field as JSON instead of JSONB.