SQLite User Forum

some weird date side effects
Login

some weird date side effects

(1) By rbucker on 2022-03-30 22:40:44 [link] [source]

sqlite> select strftime('%Y-%m-%d', 'now');
2022-03-30
sqlite> select strftime('%Y-%m-%d', 'now', '-1 month');
2022-03-02

and

sqlite> select strftime('%Y-%m-%d', '2022-03-30');
2022-03-30
sqlite> select strftime('%Y-%m-%d', '2022-03-30', '-1 month');
2022-03-02

I have to look for the source... I've seen the exact same thing happen in golang. Frankly I'm not sure what was supposed to happen but I would have expected something like FEB.

(2) By Richard Hipp (drh) on 2022-03-30 23:57:26 in reply to 1 [link] [source]

One month before 2022-03-30 is 2022-02-30, but that date does not exist, so it wraps around to 2022-03-02.

What do you think that answer should be?

(3) By anonymous on 2022-03-31 00:58:23 in reply to 2 [link] [source]

I would expect that -1 month in the query would determine how many days in month -1 from the month where -1 is applied. 

In this case -1 would be FEB so, DaysInMonth -1 from March is 28 or 29 so the return value should be 28 or 29 / 02 depending on if said moth is in a leap year.

A human would know this is one calendar month before regardless of the fact that feb has 28 or 29 days. 

I think the DaysInMonth test should be done as a matter of good date management functions.

(4) By rbucker on 2022-03-31 01:17:51 in reply to 2 [link] [source]

I just looked at the code again... and sure enough. It subtracted 1 from the month then normalized the month/year... then normalized the day. Since the day was >28 it had to re-normalize the month.

Frankly I cannot see anything wrong with that. Thanks and sorry for distracting.

(5) By rbucker on 2022-03-31 01:31:43 in reply to 2 [link] [source]

Last note: It seems that TCL returns 2022-02-28 instead of 2022-03-02.

(6) By Karl (kbilleter) on 2022-03-31 02:52:13 in reply to 5 [link] [source]

That's what I'd do personally but GNU date also behaves as above (2022-03-02).

(19) By Holger J (holgerj) on 2022-04-01 15:52:05 in reply to 5 [link] [source]

Yes, and PostgreSQL does the same. So it's very likely that this is the correct behaviour according to ISO8601.

# select date '2022-03-30' - interval '1 month';
┌─────────────────────┐
│ ?column? │
├─────────────────────┤
│ 2022-02-28 00:00:00 │
└─────────────────────┘

(20) By Harald Hanche-Olsen (hanche) on 2022-04-01 17:05:17 in reply to 19 [link] [source]

I am in possession of the 2004 edition of ISO8601, and it does not specify the interpretation of the month interval to this level of precision. The latest version is from 2019, I believe, but given the ridiculous prices charged for standards, it's anybody's guess what (if anything) it says about it.

(7) By Ryan Smith (cuz) on 2022-03-31 07:59:20 in reply to 2 [link] [source]

I believe that is the worst possible month increment.

When someone states to increase the month, they typically mean increasing/decreasing the MONTH and is very willing to have the days curbed where the month-length cannot accommodate it, but curbing the days is a lot different from simply shoving the non-fitting days into another month.

Most languages* do this sensible thing and actually increases/decreases the MONTH counter in their Month-increment functions, and then picking the same day in the target month, or the last day if said target month does not have enough days. A very simple rule.

I can't imagine any human ever asking for the month to be increased/decreased being happy to see:

  30 March   - 1 Month --> 2 March, or
  31 May     - 1 Month --> 1 May,   or
  31 January + 1 Month --> 3 March, or
  31 May     + 1 Month --> 1 July,
  etc.

(I'm open to arguments in favor of the above result, but I'm betting on it not existing).

I submit the only correct way to "move the date to a next/previous month" is as described above, that would yield:

  30 March   - 1 Month --> 28 Feb (or 29 Feb on a leap-year), and
  31 May     - 1 Month --> 30 April, and
  31 January + 1 Month --> 28 Feb, and
  31 May     + 1 Month --> 30 June,
  etc.

Moreover, if you start on a variable day and then get the next month successively (base-date + n months), let's say for future projections, in the SQLite way, starting on December 31, your dates would go:

Dec 31
Jan 31
Mar 02
Mar 31
May 01
May 31
Jul 01
Jul 31
etc.

Where it should be clear the more accurate increment would be:
Dec 31
Jan 31
Feb 28
Mar 31
Apr 30
May 31
Jun 30
Jul 31
etc.

Again, I'm open to argument for the former.

That said, I'm not arguing for changing this in SQLite - understanding that legacy changes may break someone's code (though it's hard to imagine anyone depending on the above quirk), and Richard's question of "What do you think the answer should be?" might be an honest attempt to gather opinion, but if perhaps it is a doubling down on the current implementation being the "correct" way, or displaying surprise at something else being expected, I would find that surprising.

[*] "Most" in this context obviously means "Most that I am familiar with", I am not informed of ALL and acknowledge there is some probability of being wrong on this point, but if so, I'm still hoping it doesn't invalidate the reasoning.

(8) By jchd (jchd18) on 2022-03-31 09:42:23 in reply to 7 [link] [source]

This kind of date arithmetic defies logic, "month" and "year" not being formally defined.

Both answers 2022-02-28 and 2022-03-02 have their value but none can round-trip (+1 month (- 1 month)) since you get resp. 2022-03-28 and 2022-04-02, so you never get 2022-02-30 back.

One easy way to provide more flexibility would be to create a pair of new modifiers, say calmonth for calendar month and calyear or some other names, yielding 2022-02-28 in this case.

I believe this would still be compatible with 'Lite' (probably little code involved) and leave the choice of behavior to the user.

(9) By Rowan Worth (sqweek) on 2022-03-31 10:03:58 in reply to 7 [link] [source]

I submit the only correct way to "move the date to a next/previous month" is as described above, that would yield:

  30 March   - 1 Month --> 28 Feb (or 29 Feb on a leap-year), and
  31 May     - 1 Month --> 30 April, and
  31 January + 1 Month --> 28 Feb, and
  31 May     + 1 Month --> 30 June,
  etc.

If 30 March - 1 Month --> 28 Feb in a non-leap-year, does that mean that 30 March - 1 Month - 1 Day --> 27 Feb? And yet 30 March - 1 Day - 1 Month is presumably 28 Feb?

IMO trying to do any kind of arithmetic with a non-constant unit such as "Month" is a terrible idea; there's no way to implement sensible and consistent semantics. So personally I would advocate for disallowing "± Month," but I do agree with you though, that if it is allowed then interpreting in as "advance/precede month" best satisfies the principle of least surprise.

Most languages* do this sensible thing

I'm not sure that's true.

C: doesn't really provide date arithmetic, but if you start with 2022-01-31 and increment .tm_mon then strftime will happily format an invalid date for you: 2022-01-31T00:00:00 -> 2022-02-31T00:00:00

python: the datetime module doesn't support timedeltas with units greater than day

GNU date: same as sqlite's behaviour: date -d '2022-01-30 + 1 month' +%FT%T -> 2022-03-02T00:00:00

java: clips the day into the target month: java.time.LocalDate.of(2022, 01, 31).plusMonths(1) -> 2022-02-28

ruby: the Date type only allows addition/subtraction of days/seconds. Does provide an explicit "next_month" function that clips the day

golang: time.Date has its own normalisation scheme which is applied after AddDate(0 /* years */, 1 /* months */, 0 /* days */): 2022-01-31T00:00:00 -> 2022-03-03T00:00:00

1.5 out of 6? Anyway the point is, never trust a computer to do what you expect :)

(11) By Gunter Hick (gunter_hick) on 2022-03-31 10:40:30 in reply to 9 [link] [source]

Please note that "Year" is also a non-constant unit. This includes the obvious leap year (not only for Feb 29th where naively incrementing the year number produces an invalid date, but also the year around it when 1 year = 366 days) and the less obvious leap seconds (27 between 1972 and 2016) and historical calender switches (r.g. gregorian and julian, which included jumps in the date by 10 and a whopping 80 days; and the latter also changed the number of days in 7 of 12 months).

(12.1) By Harald Hanche-Olsen (hanche) on 2022-03-31 10:58:50 edited from 12.0 in reply to 9 [link] [source]

Given how ill defined the notion of an “N months” increment is, I have to ask: What are the use cases where this is important? That someone finds the current behaviour surprising, does not seem like a sufficient reason to change anything. And I would argue that any date logic that depends on a particular interpretation of N month increments is flawed, and should be replaced by logic counting days or weeks.

Speaking of which, I think a new “N weeks” modifier would be more useful, even though you could easily multiply by 7 and use the “N days” modifier instead. It lets you easily specify the Nth Wednesday of any month by applying the “beginning of month” modifier followed by “weekday 3” and “N weeks” modifiers.

But also, having a way to specify the last occurrence of a weekday within a month seems useful. I have encountered several monthly events scheduled for the last Friday (for example) of each month.

Edited to add a rhetorical question: What date is one month prior to February 28? Currently, you'll get January 28. I think there is no strong reason, other than the look-alike dates, to pick that answer over January 31. I mean, one month before the last day of any given month should be the last day of the previous month, right? It stands to reason. Or not, as you may see fit. ;-)

(13) By rbucker on 2022-03-31 12:11:01 in reply to 12.1 [link] [source]

A modifier is a good idea... but a pragma might also be a good idea for the default behavior.

(10) By rbucker on 2022-03-31 10:32:54 in reply to 7 [link] [source]

Unfortunately this issue is not limited to +1/-1 month... it's any month where the current DOM is greater than the DIM of the target month. (Moving from Dec 31 to Feb (-3 month) being an example).

(14) By rbucker on 2022-03-31 12:15:01 in reply to 2 [link] [source]

Found this article: date calc breaks prod

It encapsulates the issue and gives examples in different langs.

(15) By rbucker on 2022-03-31 13:20:21 in reply to 1 [link] [source]

One very interesting side effect... have you ever used VIM where you moved the cursor to the end of the current line and then moved the cursor up/down the document... notice that it always tracks with the last char/original col depending on the length of the line.

pseudo code:

date = '2022-01-31'
for loop from 1 to 6
  newdate = strftime('%Y-%m-%d', date, '1 month')
  print "date -> newdate"
  date = newdate
loop

results:

2022-01-31 -> 2022-03-03
2022-03-02 -> 2022-04-03
2022-04-31 -> 2022-05-03
...

alternately

date = '2022-01-31'
for loop from 1 to 6
  newdate = strftime('%Y-%m-%d', date, '${loop} month')
  print "date -> newdate"
  date = newdate
loop

results:

2022-01-31 -> 2022-03-03
2022-01-31 -> 2022-03-31
2022-01-31 -> 2022-05-01
...

Now let's just ASSUME for a moment that the date math is changed (not corrected) to the other method. Now what will we see?

pseudo code:

date = '2022-01-31'
for loop from 1 to 6
  newdate = strftime('%Y-%m-%d', date, '1 month')
  print "date -> newdate"
  date = newdate
loop

results:

2022-01-31 -> 2022-02-28
2022-02-28 -> 2022-03-28
2022-03-28 -> 2022-04-28
...

alternately

date = '2022-01-31'
for loop from 1 to 6
  newdate = strftime('%Y-%m-%d', date, '${loop} month')
  print "date -> newdate"
  date = newdate
loop

results:

2022-01-31 -> 2022-02-28
2022-01-31 -> 2022-03-31
2022-01-31 -> 2022-04-30
...

if you think you understand dates. You don't! --anon

They all require consideration. Theirs just no way NOT to think about dates and times with great care.

(16) By anonymous on 2022-03-31 13:38:08 in reply to 1 [source]

Midnight on March 30 is 30/31 of the way through March. One month earlier would be 30/31 of the way through February, or about 02:19:21 on February 28...

(17) By Gunter Hick (gunter_hick) on 2022-03-31 13:57:33 in reply to 1 [link] [source]

Please note that this is all documented here https://sqlite.org/lang_datefunc.html

Specifically:

"Note that "±NNN months" works by rendering the original date into the YYYY-MM-DD format, adding the ±NNN to the MM month value, then normalizing the result. Thus, for example, the date 2001-03-31 modified by '+1 month' initially yields 2001-04-31, but April only has 30 days so the date is normalized to 2001-05-01. A similar effect occurs when the original date is February 29 of a leapyear and the modifier is ±N years where N is not a multiple of four."

"The computation of local time depends heavily on the whim of politicians and is thus difficult to get correct for all locales. In this implementation, the standard C library function localtime_r() is used to assist in the calculation of local time. The localtime_r() C function normally only works for years between 1970 and 2037. For dates outside this range, SQLite attempts to map the year into an equivalent year within this range, do the calculation, then map the year back.

These functions only work for dates between 0000-01-01 00:00:00 and 9999-12-31 23:59:59 (julian day numbers 1721059.5 through 5373484.5). For dates outside that range, the results of these functions are undefined.

Non-Vista Windows platforms only support one set of DST rules. Vista only supports two. Therefore, on these platforms, historical DST calculations will be incorrect. For example, in the US, in 2007 the DST rules changed. Non-Vista Windows platforms apply the new 2007 DST rules to all previous years as well. Vista does somewhat better getting results correct back to 1986, when the rules were also changed.

All internal computations assume the Gregorian calendar system. They also assume that every day is exactly 86400 seconds in duration; no leap seconds are incorporated."

And the beauty of SQLite is that you can create your own datetime functions and even overload the provided functions to make them do exactly what you want, completely baffling the casual reader of your SQL code.

(18.1) By rbucker on 2022-03-31 17:42:57 edited from 18.0 in reply to 17 [link] [source]

I appreciate that there is a sqlite doc that addresses the question... and that Richard Hipp replied first asking "what did I expect?".

As there are other SQLite docs that talk about SQLite having technical parity with other DBs including postgres I ran the following:

(pg)

select date('2022-03-30') + ('-1 month')::interval;
2022-02-28T00:00:00.000Z

(mysql6.5)

SELECT DATE_ADD("2022-03-30", INTERVAL -1 month);
2022-02-28

(sql server)

SELECT DATEADD(month, -1, '2022/03/30') AS DateAdd
2022-02-28T00:00:00Z

It's perfectly fine that SQLite wants to do this differently. However, for compatibility reasons I would strongly recommend a PRAGMA for global configuration and a strftime() modifier for local... The main justification is there is no simple sql expression that could be constructed to do exactly that and the CLI does not support variables or variable expansion(computing the value outside the sql query).

I suppose anyone could implement an extension.