Suggestion: Use strong hashes on the download page
(1) By anonymous on 2020-07-26 14:52:03 [link] [source]
(2) By Ryan Smith (cuz) on 2020-07-26 16:14:08 in reply to 1 [link] [source]
I'd like to suggest switching to a stronger hash that is not broken.
Calling SHA1 "broken" are we? Please demonstrate.
Quoted from the linked Site:
Who is capable of mounting this attack? This attack required over 9,223,372,036,854,775,808 SHA1 computations. This took the equivalent processing power as 6,500 years of single-CPU computations and 110 years of single-GPU computations.
- and this:
How widespread is this? As far as we know our example collision is the first ever created.
- and this:
Has this been abused in the wild? Not as far as we know.
110 CPU years... SQLite's release cycle alone will thwart any prospective attacker.
To leverage this attack against SQLite someone will have to spend considerably more resources than the average attacker possesses and even then, after having successfully crafted a SHA1 for the code, all of which are considerably larger than the PDF used in the demonstration (upon which the attack figures is based), then such an attacker also has to successfully hack the SQLite servers and post such code on there.
I'm all for changing to a different completely un-attacked hash algorithm, but the notion that the current implementation is in any way unsafe or broken is just silly. When people start calling security of any system "broken", "shattered" or "we're just inviting criminals now", my normal response to such claims is: Prove it. Break it.
I'll afford the OP an even easier challenge: Forget for a moment the need to break into SQLite's servers - but simply show me an amalgamation (freely downloadable) with added demonstrably malicious code and the same SHA1 hash as the on-line posted one. (I'll even accept it if you got help from those folks who fashioned the PDF's colliding hash). Upon that I will rescind my voice on the matter, eat my own hat, and completely join the band of protestors wanting an upgrade to that hashing algorithm soonest.
Until then, when requesting people to do work for no valid proven reason at all, kindly accompany any such request with a substantial monetary donation.
(3) By Warren Young (wyoung) on 2020-07-26 18:46:28 in reply to 2 [link] [source]
In addition to the SHAttered attack, we also have more recent the SHAmbles attack.
110 CPU years... SQLite's release cycle alone will thwart any prospective attacker.
You're making the assumption that an attacker can use only one CPU. There are several cloud services that will happily rent you 110 CPUs for a 1-year attack, 220 for a half-year attack, 440 for a quarter-year attack, etc.
Aside from that, you're using numbers from an old attack. The SHAmbles attack was made using US $45000 in processing time last year. Tell me there are no entities willing to spend that kind of money to provide vulnerable versions of SQLite to a large chunk of the Internet population.
after having successfully crafted a SHA1 for the code, all of which are considerably larger than the PDF used in the demonstration
I doubt the cost of the attack is dominated by the size of the object to be hashed, particularly for any sort of aggregating file format like Zip or Tar. I expect that an attacker should be able to produce the "base" file and then extend it until they get a crafted match.
Forget for a moment the need to break into SQLite's servers
...as you must, since if you can do that, then you don't need any of these attacks at all! You can just upload whatever you want and change the hash, since they're served from the same place.
That's the problem with this whole idea of hashed downloads. I don't understand why anyone has any confidence in them.
Now, if there were some sort of trusted third party who would download things, check them, hash them, and serve up their own hashes, that might be valuable, but I'd expect to pay enterprise IT service sort of prices to get it.
Upon that I will rescind my voice on the matter, eat my own hat, and completely join the band of protestors wanting an upgrade to that hashing algorithm soonest.
You're asking someone to spend another $45000 when someone's already done something equivalent.
(4) By Richard Damon (RichardDamon) on 2020-07-26 19:02:13 in reply to 3 [link] [source]
Forget for a moment the need to break into SQLite's servers ...as you must, since if you can do that, then you don't need any of these attacks at all! You can just upload whatever you want and change the hash, since they're served from the same place.
I would think this argument pretty much defeats the request. A stronger hash provides NO more security as, in effect the hash provide NO security beyond knowing that the file came from the official domain. Anyone that can put a false file on the server can change the hash to compare to. The only use of the hash is to check a download from a mirror.
(5) By Rowan Worth (sqweek) on 2020-07-27 02:18:44 in reply to 4 [link] [source]
The only use of the hash is to check a download from a mirror.
Right, but if a weak hashing algorithm is in use then mirrors are free to publish whatever crafted code they like and go "hey look the file matches the official hash therefore you can trust it."
(9) By Keith Medcalf (kmedcalf) on 2020-12-28 22:18:49 in reply to 2 [link] [source]
That attack required adding gigabytes of drivel (padding) to the file in order to create the collision. It was plainly obvious to any non-intellectually challenged person that this was a forgery.
In this particular case one would have to add shitloads of drivel (padding) that was "invisible" yet still permitted the underlying tar.gz to be intelligible.
With the Proprietary Document Format (PDF) file this is trivial, simply put a postscript comment in the file to hide the drivel.
Putting a couple of gigabytes in a .tar.gz or .zip in such a manner that it does not "fuck up" the un-gz-ing, unzipping, or un-tar-ring is not trivial. You you could put a bloody big "Forgery Drivel -- Please Ignore Me" file in the tar or zip, but IT WOULD NOT BE INVISIBLE. This is not to mention that the it is the compressed "drivel" that must participate in the forgery. Computing the drivel itself is a significant task -- computing the "uncompressed" version of the drivel would probably take a few millenium.
(6) By anonymous on 2020-07-30 20:51:54 in reply to 1 [link] [source]
I'm the OP. It's lovely to see that the download page now uses SHA-3, thank you.
(7) By David Jones (vman59) on 2020-08-02 13:45:18 in reply to 6 [link] [source]
If verifying the hash with OpenSSL, specify sha3-256 as the digest algorithm.
(8) By anonymous on 2020-12-28 21:43:14 in reply to 7 [link] [source]
Any chance of showing more than one hash on the downloads page? Or maybe just for the windows download? :) I can check sha3 on linux or macos but not on windows -- because it's a managed computer and windows does not yet have a built-in way to check. I approve of strong hashes, but I need a weaker hash...
(10) By Richard Hipp (drh) on 2020-12-28 23:00:37 in reply to 8 [link] [source]
The stand-along "fossil.exe" binary that you can download from the Fossil website supports to the "fossil sha3sum" command. It does not require any installation in any managed folders. It's just an EXE that you run from anyplace.
(11) By anonymous on 2020-12-29 16:32:58 in reply to 10 [link] [source]
Thanks for understanding my situation perfectly. And thanks for the suggestion, fossil could work for me, except I don't see any checksums at all on their download page, nor could I find any by searching their website.
Anyway, doesn't this just punt the problem? Suppose I ask fossil to provide a checksum and they give me a sha3sum? I can't be the only one gnashing my teeth that MS is behind the times. I'm on Server 2016, but I understand Windows 10 is in the same situation.
(13) By Donald Griggs (dfgriggs) on 2020-12-29 22:13:51 in reply to 11 [link] [source]
Another pre-compiled option for Windows is OpenSSL.
From page: https://wiki.openssl.org/index.php/Binaries
I took the link to:
The page asserts: The OpenSSL DLLs and EXE files are digitally code signed 'Open Source Developer, François PIETTE'
Example of use: openssl dgst -sha3-256 sqlite-tools-win32-x86-3340000.zip
(16) By anonymous on 2020-12-30 18:37:15 in reply to 13 [source]
Thanks for the suggestion of an OpenSSL binary. It could work, except ... the binary is from an unknown third party and "Use these OpenSSL derived products at your own risk; these products have not been evaluated or tested by the OpenSSL project." I don't see why I should be required to trust and verify TWO downloads instead of one. So I'm going to pass on the third party OpenSSL exe.
I explained my situation, which is that Windows out-of-the-box does not provide a tool to verify the provided sha3sum. There are numerous possible workarounds, none of them are ideal. My suggestion was to provide a second hash, which seems reasonable to me, although I will admit it's also not ideal.
Some people have provided other reasonable workarounds. Not all of them work for me, and I choose not to use the possible ones because in my view it's illogical to engage in risky behavior B in order to "reduce risk" in scenario A. But I wish to give thanks for all the helpful links so far. If this were my own pc rather than my employer's I probably would use one of them.
In the absence of any ideal solution, I'm not going to get upset if my own non-ideal suggestion is not taken up. In the meantime I'll continue to use the older working sqlite here, albeit without the option of a clustered index. I am upset though by a more recent comment, and will reply to it separately. Maybe it won't pass moderation...
Anonymous because on a work pc and I'm note supposed to risk the reputation of my employer.
(17) By Richard Hipp (drh) on 2020-12-30 19:07:32 in reply to 16 [link] [source]
I explained my situation, which is that Windows out-of-the-box does not provide a tool to verify the provided sha3sum. There are numerous possible workarounds, none of them are ideal.
Here is yet another suggestion:
Since security is important to your company, and in particular security associated with SQLite databases and the programs used to access those databases, perhaps your company could enter into a paid technical support arrangement with the SQLite developers. In that way, you would help to fund our continuous and ongoing efforts to protect the SQLite code base from unauthorized tampering. And as a paying client, you can probably convince us to send full-vetted SQLite source files or builds to you via whatever secure channel you find most convenient.
By returning a very small percentage of the value you receive from SQLite back to the SQLite developers (still keeping the vast majority of that value for yourselves), you will be helping to protect your supply chain and at the same time simplifying your own operations by making it easier to obtain validated and trusted SQLite code.
(19) By Warren Young (wyoung) on 2020-12-31 00:33:43 in reply to 17 [link] [source]
you can probably convince us to send full-vetted SQLite source files or builds
…which would also solve the problem with using such hashes in the first place: if you're worried that the remote host might be serving untrustworthy binaries, why are you trusting the hashes posted right alongside those binaries? If someone can modify one, then why not the other?
drh's suggestion is a much more reliable option than any of this faffing about with SHA-3 programs.
(20) By Keith Medcalf (kmedcalf) on 2020-12-31 01:59:40 in reply to 19 [link] [source]
The same utility as providing the password to an e-mail attached zip file in the same (or different) e-mail as sent the zip file in the first place.
This is, it is for decorum, deniability, and CYA purposes. It is not intended to have any actual security benefit. Like searches by Airport Security, it is intended primarily as Theater to calm the hysterical masses.
It can be argued that such measures cull out the totally and completely incompetenr, and that probably covers most of the threat landscape. The remainder deliberately install malware (see Solarwinds) and are thus hoisted by their own petards.
The least risky version of any software to use is the most recently discontinued/unsupported version because: (a) that version will suffer no more fiddling; (b) will work the same forever; (c) has a high probability that all vulnerabilities are well-known; and, (d) since it is well known, support technicians can be gotten for a pittance at the local pub.
(23) By anonymous on 2021-01-02 03:29:03 in reply to 20 [link] [source]
"This is, it is for decorum, deniability, and CYA purposes." --Exactly my intent.
"Like searches by Airport Security, it is intended primarily as Theater to calm the hysterical masses." --Since I am employed by the hysterical masses, I'm all for it.
(21) By anonymous on 2020-12-31 02:02:41 in reply to 19 [link] [source]
Absolutely correct. The hashes could be posted to something like Twitter or to the forum, separate from the download page.
I second (or third) drh’s suggestion to support the project via a paid subscription and you’ll get the hashes, source and binary you preferred method.
(22) By anonymous on 2021-01-01 10:47:54 in reply to 19 [link] [source]
I'm told that hashes used to make sense in cases when there were fast, easily accessible local mirrors vs. slow and expensive access to the rest of the Internet. In that case, visiting the official web site just for the hash of the file downloaded from the local mirror kinda makes sense. I used to sometimes get truncated or otherwise damaged files back when my Internet connection was much less reliable, as was
Range: header support in many servers, but things like rolling hashes are much more useful for that.
(12) By anonymous on 2020-12-29 17:32:39 in reply to 8 [link] [source]
but I need a weaker hash
Can you verify the windows sha3 hash on linux prior to downloading on Windows?
(14) By anonymous on 2020-12-29 23:20:42 in reply to 12 [link] [source]
No, I can't verify on linux first because "managed computer". I'll look at OpenSSL now, my previous web search told me I would have to build from source.
But I do want to keep going back to the original point... It can't be just me. Is it really impossible to make available a secure modern hash AND a backwards-compatible "mostly" secure older hash? As far as I know, sha2 as of today is only compromised theoretically, not practically. In the meantime, some Windows users are a little stuck.
(15) By RandomCoder on 2020-12-30 00:16:28 in reply to 14 [link] [source]
Any user that can download and run executables can download Fossil, OpenSSL, RHash, Python, Perl, or any number of other tools to verify the hash.
And since a common user of the SQLite downloads will have access to a compiler, they could just compile their own version of sha3sum.
Between that, and my assumption that only a small minority of users bother to verify the any hash or CRC beyond whatever verification their zip program does on its own, I'm not sure that means any Windows users are stuck.
(18) By Jason Gibson (Jason_Gibson) on 2020-12-30 21:24:05 in reply to 8 [link] [source]
(24) By anonymous on 2021-01-02 03:48:52 in reply to 18 [link] [source]
This is a good idea, if the C# path doesn't work out I will try this. Thanks.
(25) By anonymous on 2022-03-11 18:41:26 in reply to 1 [link] [source]
I arrived to find only SHA3 hashes, and would request some other options that a Windows user could use to confirm a clean download. That's all I want.
Many others here already pointed to the flaws of posting the hash alongside the download file. Even if SHA3 is the only secure "solution" to the "problem" there's still no reason to inconvenience many users of one of the platforms for which SQLite is offered. I just downloaded Powershell 7 and the updated (?) >Get-Filehash -algorithm options do not include SHA3.
Fortunately I could go to my linux VM and find the necessary tool but here I am almost an hour later because "security concerns".
p.s. Generally it would be a great improvement to have hashes posted SEPARATELY from the files, on another domain perhaps.
Thank you for your consideration! New SQLite user and potential bug-reporter.
(26) By anonymous on 2022-03-11 19:29:43 in reply to 25 [link] [source]
Since I already had git on this machine, I was able to use openssl: C:\Program Files\Git\usr\bin>openssl < "C:Users\anonymous\Downloads\sqlite-dll-win64-x64-3380000.zip" dgst -sha3-256
This confirms that I have a clean download of something
(27) By Donal Fellows (dkfellows) on 2022-03-14 12:33:36 in reply to 25 [link] [source]
here I am almost an hour later because "security concerns"
If you want to be paranoid, you're going to have to pay the costs of doing so. After all, you can't really trust someone else's SHA3-256 implementation, can you?