Converting Text Time for Query...
(1) By Superfreak3 on 2022-05-21 22:07:54 [link] [source]
Hi there,
I have a sqlite Db with a Date field namded Date in the following format...
Friday, February 18, 2022
I need to retrieve data >= date1 and <= date2. I have tried everything I can find in a query, but nothing is working for the date conversion.
Can anyone help me get something like the following...
Select RiseBG, PeakBG from MealData where date >= date1 and <= date2.
I've seen the use of BETWEEN, but not sure if that includes the dates passed into the query or not.
Any help GREATLY appreciated and I thank you in advance.
(2) By Larry Brasfield (larrybr) on 2022-05-21 22:30:50 in reply to 1 [link] [source]
You would be ahead if you store the date in a sortable form, such as ISO 8601 or one of the numeric forms known to the data and time functions. You can use one of those functions to prettify your date when needed for presentation, but do comparisons in queries much more readily and speedily.
(3.1) By Superfreak3 on 2022-05-21 22:49:52 edited from 3.0 in reply to 2 [link] [source]
The problem is it's not our data, just need to convert it. I was hoping it was possible in the query.
I guess their date-time format was derived from a date-time picker in a .NET app just saved as a string.
(4) By Larry Brasfield (larrybr) on 2022-05-21 22:51:59 in reply to 3.0 [link] [source]
That data is certainly susceptible to conversion. With a few instr(...) and substring(...) function applications, and maybe a willingness to overlook the extraneous day-of-week subfield, it should be simple to extract the components, then use those along with a CASE expression to compute a numeric date or an ISO 8601 formatted date. I shudder to think of putting such a computation in multiple places in a query. It would be better to make a more data-like view or even an intermediate temporary table.
(5) By Simon Slavin (slavin) on 2022-05-22 07:03:34 in reply to 1 [link] [source]
- Create another column which will have the date stored in a useful format. Julian Day, stored as an integer, would be a good one, and is widely understood and used by programmers.
- Write software in your favourite language to read the existing date from each row, convert it to the better format, and update the row to set the value in the new column.
- Change your code so that new values are stored in both date columns. (If you have no plans to add or change rows to that table, you can skip this one.)
- Next time you create a database, think about this problem before you decide what to store and what format to store it in.
You may find the functions on this page useful. But there's no simple function which will do all the conversion for you.
(8) By Superfreak3 on 2022-05-23 14:02:03 in reply to 5 [link] [source]
Will try this. It is not our Database so we didn't decide to store it this way. We just have to deal with it now :(
(6.2) By MBL (UserMBL) on 2022-05-22 12:49:26 edited from 6.1 in reply to 1 [link] [source]
Check my recent post in following forum post 4/17/2022, 5:03:31 PM convert into YYYY-MM-DD HH:MM:SS.
Like described there you can use a substr(instr(...))
to get rid of the leading day string and the comma, then a sequence of 12 replace statements to change the month name into a month number and a comma.
Use a replace
to change the comma and space into slashes as separators.
Append with ||
a dummy time like '00:00:00'
and continue with my example in that post.
The rest of the work would be what I have shown in my posted example.
Once you understood the principle you will find an easier way for your specific format.
Edit: Attached another proposal of mine (with correction on final ordered output of the ISOnized):
D:\sandbox>sqlite3
SQLite version 3.38.5 2022-05-06 15:25:27
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .mode box
sqlite> with input(txt) as (
...> values('Friday, February 18, 2022')
...> ), JSONready as (
...> select txt
...> , substr(txt,instr(txt,', ')+2) as woDayName
...> , replace(substr(txt,instr(txt,', ')+2),', ',',') as singleSep
...> , replace(replace(substr(txt,instr(txt,', ')+2),', ',','),' ',',') as homogenized
...> , replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(
...> replace(replace(substr(txt,instr(txt,', ')+2),', ',','),' ',',')
...> ,'January','01'),'February','02'),'March','03'),'April','04'),'May','05'),'June','06')
...> ,'July','07'),'August','08'),'September','09'),'October','10'),'November','11')
...> ,'December','12') as digitized
...> , json( '["'|| replace(
...> replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(replace(
...> replace(replace(substr(txt,instr(txt,', ')+2),', ',','),' ',',')
...> ,'January','01'),'February','02'),'March','03'),'April','04'),'May','05'),'June','06')
...> ,'July','07'),'August','08'),'September','09'),'October','10'),'November','11')
...> ,'December','12')
...> ,',','","') || '"]' ) as jsonized
...> from input
...> )
...> select *
...> , jsonized->>'$[0]' as Month
...> , jsonized->>'$[1]' as Day
...> , jsonized->>'$[2]' as Year
...> , format('%s-%s-%s', jsonized->>'$[2]', jsonized->>'$[0]', jsonized->>'$[1]') as ISOnized
...> from JSONready;
┌───────────────────────────┬───────────────────┬──────────────────┬──────────────────┬────────────┬────────────────────┬───────┬─────┬──────┬────────────┐
│ txt │ woDayName │ singleSep │ homogenized │ digitized │ jsonized │ Month │ Day │ Year │ ISOnized │
├───────────────────────────┼───────────────────┼──────────────────┼──────────────────┼────────────┼────────────────────┼───────┼─────┼──────┼────────────┤
│ Friday, February 18, 2022 │ February 18, 2022 │ February 18,2022 │ February,18,2022 │ 02,18,2022 │ ["02","18","2022"] │ 02 │ 18 │ 2022 │ 2022-02-18 │
└───────────────────────────┴───────────────────┴──────────────────┴──────────────────┴────────────┴────────────────────┴───────┴─────┴──────┴────────────┘
sqlite>
(7) By MBL (UserMBL) on 2022-05-22 13:51:41 in reply to 1 [link] [source]
Another solution of mine using an evaluation of Lua embedded extension function eLua
and the substitution using a lambda function with 3 arguments using the pattern matching mechanism of the string.gsub
substitution methode for field separation:
D:\sandbox>sqlite3.exe
SQLite version 3.38.5 2022-05-06 15:25:27
Enter ".help" for usage hints.
Connected to a transient in-memory database.
Use ".open FILENAME" to reopen on a persistent database.
sqlite> .load sqlite3.dll
sqlite> .mode box
sqlite> with input(txt) as (
...> values('Friday, February 18, 2022')
...> )
...> select txt
...> ,eLua('return arg[1]', txt ) as LuaEcho
...> ,eLua('return (string.gsub(arg[1],".-, (.+) (%d+), (%d+)","%3-%1-%2"))', txt ) as LuaOrdered
...> ,eLua('return (string.gsub(arg[1],".-, (.+) (%d+), (%d+)", function(m,d,y)
...> return string.format("%s-%s-%s",y,m,d) end ))', txt ) as LuaFunction
...> ,eLua('return (string.gsub(arg[1],".-, (.+) (%d+), (%d+)", function(m,d,y)
...> local NameToNum = { January = "01", February = "02", March = "03"
...> , April = "04", May = "05", June = "06"
...> , July = "07", August = "08", September = "09"
...> , October = "10", November = "11", December = "12" }
...> return string.format("%s-%s-%s",y,NameToNum[m],d) end ))', txt ) as ISOnized
...> from input;
┌───────────────────────────┬───────────────────────────┬──────────────────┬──────────────────┬────────────┐
│ txt │ LuaEcho │ LuaOrdered │ LuaFunction │ ISOnized │
├───────────────────────────┼───────────────────────────┼──────────────────┼──────────────────┼────────────┤
│ Friday, February 18, 2022 │ Friday, February 18, 2022 │ 2022-February-18 │ 2022-February-18 │ 2022-02-18 │
└───────────────────────────┴───────────────────────────┴──────────────────┴──────────────────┴────────────┘
sqlite>
(10) By anonymous on 2022-05-23 16:25:49 in reply to 7 [link] [source]
Could you provide a link to this Lua extension?
(11) By MBL (UserMBL) on 2022-05-23 21:16:59 in reply to 10 [source]
I am very sorry for now, it exists (yet) only inside my complete work together with the other extensions and has not been published anywhere.
This means that I will have to extract that parte and prepare it a little bit for presentation to whomever wants to use it.
There are two important points to know: a) Where to create the Lua state and b) how to use it inside the function calls.
For my single user/single thread usage the answer to a) is: Either together with the opening of a connection to database or a) when loading the extension module. The Lua state pointer needs to be kept somewhere for first and reuse and also the closing of the state should be under control (either when closing the database connection or when unloading the module).
The answer to b) is: sqlite3_create_function_v2
gets the information as 5th parameter (*pApp).
sqlite3_create_function_v2(db, "eLua", -1, SQLITE_UTF8, &gLuaStateVM, eLuaExecFunc, 0, 0, NULL); // SQLITE_DETERMINISTIC not recommended
(9) By Ryan Smith (cuz) on 2022-05-23 15:58:02 in reply to 1 [link] [source]
To add to the other examples - this is pure SQL to read such a bad-formatted date from a table and fix it, or in this case, update it back into the table.
In the example I update a second column, but you can overwrite the origin column too, just make sure to keep backups of the DB before you do.
-- SQLite version 3.37.2 [ Release: 2022-01-06 ] on SQLitespeed version 2.1.3.11.
CREATE TABLE bad_dates(bad_date TEXT, good_date NUMERIC);
INSERT INTO bad_dates(bad_date) VALUES
('Friday, February 18, 2022')
,('Friday, February 4, 2022')
,('Previousday, January 1, 2022')
,('Wednesday, may 11, 2022')
,('Sameday, September 30, 2021')
,('Someday, APRIL 2, 2023')
,('Anotherday, December 31, 2022')
;
WITH B1(bd, sdt) AS (
SELECT bad_date, upper(trim(substr(bad_date,instr(bad_date, ',') + 1))) FROM bad_dates
), B2(bd, sMth, iDay, iYear) AS (
SELECT bd, trim(substr(sdt, 1, 3)),
trim(substr(sdt, instr(sdt, ',') - 2, 2)),
trim(substr(sdt, instr(sdt, ',') + 1))
FROM B1
), B3(bd, iYear,iMonth,iDay) AS (
SELECT bd, iYear, instr('---JANFEBMARAPRMAYJUNJULAUGSEPOCTNOVDEC',sMth)/3, iDay
FROM B2
), B4(bd, sYear, sMonth, sDay) AS (
SELECT bd, iYear,
CASE WHEN CAST(iMonth AS INT) > 9 THEN iMonth ELSE '0'||iMonth END,
CASE WHEN CAST(iDay AS INT) > 9 THEN iDay ELSE '0'||iDay END
FROM B3
)
UPDATE bad_dates
SET good_date = (SELECT date(sYear||'-'||sMonth||'-'||sDay) FROM B4 WHERE bd = bad_date)
WHERE good_date IS NULL
;
SELECT * FROM bad_dates;
-- bad_date |good_date
-- -------------------------------|------------
-- Friday, February 18, 2022 |2022-02-18
-- Friday, February 4, 2022 |2022-02-04
-- Previousday, January 1, 2022 |2022-01-01
-- Wednesday, may 11, 2022 |2022-05-11
-- Sameday, September 30, 2021 |2021-09-30
-- Someday, APRIL 2, 2023 |2023-04-02
-- Anotherday, December 31, 2022 |2022-12-31
Also, I've gone for clarity rather than compactness - and you can see what happens at each stage by simply selecting from the different stage CTEs (B1, B2.. etc.) such as this example:
WITH B1(bd, sdt) AS (
SELECT bad_date, upper(trim(substr(bad_date,instr(bad_date,',')+1))) FROM bad_dates
), B2(bd, sMth, iDay, iYear) AS (
SELECT bd, trim(substr(sdt, 1, 3)),
trim(substr(sdt, instr(sdt, ',') - 2, 2)),
trim(substr(sdt, instr(sdt, ',') + 1))
FROM B1
) -- trimmed off the rest of the statement....
SELECT * FROM B2;
-- bd | sMth|iDay| iYear
-- -------------------------------|-----|----|------
-- Friday, February 18, 2022 | FEB | 18 | 2022
-- Friday, February 4, 2022 | FEB | 4 | 2022
-- Previousday, January 1, 2022 | JAN | 1 | 2022
-- Wednesday, may 11, 2022 | MAY | 11 | 2022
-- Sameday, September 30, 2021 | SEP | 30 | 2021
-- Someday, APRIL 2, 2023 | APR | 2 | 2023
-- Anotherday, December 31, 2022 | DEC | 31 | 2022
(12) By Superfreak3 on 2022-05-24 03:01:02 in reply to 9 [link] [source]
THIS WORKED PERFECTLY!!!!!!!
THANK YOU!!