Any way to preserve subquery result order in outer query
(1) By Swastik (961679) on 2022-08-16 10:41:52 [link] [source]
For the below query I'm using ORDER BY in one of the subqueries to fetch recently added products for some E-Commerce use case. This order is not preserved in the outer query when the DB executes `AND v.product_id IN ( SELECT * ...` I executed the same query in MariaDb with same data and in MariaDb the order is preserved. Is there any way in SQLite also to preserve the order of subquery results? WITH products AS ( SELECT v.product_id, v.variant_id, v.title, v.price, v.position, v.image_url, i.url FROM shopify_variants v JOIN shopify_images i WHERE v.product_id = i.product_id AND v.product_id IN ( SELECT * FROM ( SELECT p.product_id FROM shopify_products p JOIN shopify_collections c WHERE p.product_id = c.product_id AND c.bot_ref = 27674 AND c.shop_domain = "mystore.com" AND p.status LIKE "ACTIVE" AND c.title="Casual" GROUP BY p.product_id ORDER BY p.created_at DESC LIMIT 30 ) AS limiter)) SELECT products.product_id, products.position, products.variant_id, products.title, products.price, products.url, products.image_url, pr.handle, pr.description, pr.title AS producttitle FROM products JOIN shopify_products pr WHERE products.product_id =pr.product_id
(2) By Gunter Hick (gunter_hick) on 2022-08-16 11:27:57 in reply to 1 [source]
If you want your query result produced in a certain order, you need to provide an ORDER BY clause. Otherwise, the order of query results is unspecified (meaning the visitation order according to the query plan executed). Use the EXPLAIN QUERY PLAN prefix to view the plan chosen by SQLite. That MariaDB happens to produce your desired order is probably purely accidental. I don't know MariaDB, but i am guessing it is using the limiter table to "drive" the WITH products JOIN. It may estimate that doing 30 index lookups in the shopify_variants table is faster than a full table scan and checking against a materialized subselect.
(3) By anonymous on 2022-08-16 14:01:27 in reply to 1 [link] [source]
Since you rejoin with shopify_products
in the outermost query, you can just append ORDER BY pr.created_at DESC
to get what you want.
(4) By Swastik (961679) on 2022-08-16 15:03:08 in reply to 3 [link] [source]
Yeah, this will work. Just wanted to know how it is different in SQLite when MariaDb is preserving the order.
(5) By Ryan Smith (cuz) on 2022-08-16 16:22:30 in reply to 4 [link] [source]
Every SQL RDBMS engine follows the SQL standard at some distance and implements functionality to satisfy those requirements. Often the requirement is not clear or not precise, or simply has no say or sway on a specific matter (or have now but did not have back when some DB engines were born).
All of this means that every implementation is different to every other but in general will supply data according to the standard and, at a minimum, will adhere to the guarantees it makes of itself.
In this light, there are myriad differences between SQLite and MySQL (or any other engine you can name). MariaDB could have calculated it will deliver better performance by materializing the sub-query (it typically has much more memory available and can expect not be constrained). Or it might be something else.
What we can say is this: The SQL standard doesn't have any thoughts on data order, and SQLite itself makes no guarantees on it, unless the ORDER BY is specified. For that matter, I don't think MariaDB guarantees it either, could well be by accident, in which case it may work today but the same query may work different in the next version. You'll do well to refactor that query using an ORDER BY clause so it doesn't depend on such a quirk for ordering and working correctly.
In fact, SQLite has a great pragma setting "reverse_unordered_selects" that force any select queries to deliver rows in the reverse order (unless you've specified an ORDER BY) so you can test your system and see if somewhere you depend on ordering which you did not explicitly specify, and rectify that mistake.
(6) By Swastik (961679) on 2022-08-17 05:24:35 in reply to 5 [link] [source]
Thank you for detailed explanation