SQLite User Forum

[ANN] SQLite in the Cloud: The Future of Lightweight Databases
Login

[ANN] SQLite in the Cloud: The Future of Lightweight Databases

(1) By Marco Bambini (marcobambini) on 2023-03-16 13:38:01 [link] [source]

Hi all,

We are excited to announce the launch of a beta version of our new SQLite Cloud service. This service allows you to easily and securely share your SQLite databases in the cloud, eliminating the need for local storage or expensive server setups.

With our SQLite Cloud service, you can access your databases from anywhere, anytime, and on any device with internet access. Our platform is built on top of the latest security technologies, ensuring your data is always safe and protected.

Our easy-to-use interface lets you create and manage SQLite databases quickly and easily without coding or technical expertise. You can also share your databases and collaborate in real-time, making it the perfect solution for businesses, developers, and individuals.

Sign up today and start enjoying the benefits of our fast, secure, and reliable SQLite Cloud service (while in beta, the service will be completely free).

More on: https://sqlitecloud.io

For any information please contact us from: https://www.sqlitecloud.io/support

Thanks, Marco Bambini SQLite Cloud, Inc.

(2) By Simon Slavin (slavin) on 2023-03-17 16:49:16 in reply to 1 [link] [source]

For those who care about this sort of thing …

This is an example of SQLite over Raft, with Raft ensuring database promulgation and consistency (ACID). So you can have multiple copies of your database around the world, local to each of your locations, or take backups without interrupting production.

https://raft.github.io

As you'd expect, you need an account. There's a web-facing dashboard so an admin can control accounts and databases.

With a one-line MACRO change you can recompile your program switching using a local database and a Cloud one, which allows for easy development. Documentation, but not full source code, is on github:

https://github.com/sqlitecloud

The website is unfinished, but that just reflects the beta status of the project, as announced in the pst.

(3) By Keith Medcalf (kmedcalf) on 2023-03-17 17:00:20 in reply to 2 [link] [source]

Isn't RAFT one of those "usually inconsistent" things that the kiddies are enamoured with?

(4) By Keith Medcalf (kmedcalf) on 2023-03-17 17:07:17 in reply to 3 [link] [source]

My mistake, RAFT is a general distributed consensus algorithm, which if implemented properly means "always consistent". The "usually inconsistent" (called "eventually consistent" by the kiddies) is an entirely different kettle of fish derived from the "no authoritative master" principle of mayhaps design.

(6) By Marco Bambini (marcobambini) on 2023-03-18 06:52:55 in reply to 4 [link] [source]

We guarantee strong consistency across all the cluster nodes. I'll explain all the technical details in a blog post.

(7) By Marco Bambini (marcobambini) on 2023-03-21 16:07:32 in reply to 6 [link] [source]

I just published a blog post with more technical information: https://blog.sqlitecloud.io/strong-consistency-with-raft-and-sqlite

(9) By Keith Medcalf (kmedcalf) on 2023-03-21 16:40:46 in reply to 7 [link] [source]

Good explanation. I was in process control systems before retirement which will explain why I refer to what the kiddies call "eventually consistent" as "usually inconsistent" which generally equates to "useless".

(10) By Warren Young (wyoung) on 2023-03-21 18:43:16 in reply to 9 [link] [source]

The very software hosting this discussion is an AP-mode system, thus "eventually consistent."

Care to walk that charge of "useless" back a bit?

(11) By Keith Medcalf (kmedcalf) on 2023-03-21 18:50:10 in reply to 10 [link] [source]

No. I would not run my refinery on it though, no matter how you feel.

(12) By jose isaias cabrera (jicman) on 2023-03-21 19:18:23 in reply to 11 [link] [source]

No. I would not run my refinery on it though, no matter how you feel.

Huh. Ok, Keith. I will bite, and ask: Why wouldn't you? It's all over the world in billions of electrical devices. And they are working ok. So, since I am a big fan of your work, I would like to see a few reasons why, if you don't mind. Thanks.

(13) By Keith Medcalf (kmedcalf) on 2023-03-21 19:21:32 in reply to 12 [link] [source]

Fossil is on billions of electrical devices? I don't think so.

(14) By jose isaias cabrera (jicman) on 2023-03-21 19:27:33 in reply to 13 [link] [source]

Ooooohhh, now I get it. Ok, thanks. Wrong software.

(15.2) By Keith Medcalf (kmedcalf) on 2023-03-21 19:55:01 edited from 15.1 in reply to 12 [source]

I will also note that Fossil is not an AP-system. My experience is that it is only an A system. It is not "eventually consistent" in the presence of partitioning. It is, of course, always a possibility of being an expectation problem. However, I have multiple times had to rebuild the repository because it got "inconsistent" and unable to "fix itself" without "manual intervention"; that "maual intevention" often being to delete the fubar clone and start anew.

(16) By jose isaias cabrera (jicman) on 2023-03-21 19:44:22 in reply to 15.0 [link] [source]

I am not an expert, but I use Fossil at work, and I love it. I also use the ticketing system, and when people say what are you working on, I send them a link with my opened tickets. They love it, as I. But, that is the depth of my insight.

(17) By Warren Young (wyoung) on 2023-03-21 20:45:24 in reply to 15.2 [link] [source]

It is not "eventually consistent" in the presence of partitioning.

You're misapplying the CAP theorem. AP means it is always Available in the presence of Partitioning, specifically sacrificing Consistency. Thus fossil set autosync off, or the ability to say "no" to the sync failure prompt when it fails to sync due to a network error. When the partition is fixed, then it can become consistent, requiring some number of syncs depending on which repos cloned from which others.

Thus "eventual."

I have multiple times had to rebuild the repository

Simply needing to rebuild isn't a sign of problems. One of the things it does is vacuum the underlying SQLite DB, for instance. It also rebuilds delta information in the most efficient manner, and other helpful maintenance tasks.

it got "inconsistent" and unable to "fix itself" without "manual intervention";

It would have been good to have that repo so we could see it and debug the problem.

I've been using Fossil longer than you, and I've never seen it do that sort of thing.

delete the fubar clone and start anew.

I can only recall ever having one repo DB so badly messed up it couldn't be recovered, and that could easily have been due to the normal ways any SQLite DB can be messed up.

The way you write it, you'd think this happened frequently. In my experience, it's exceedingly rare, and I have yet to ascribe the source of the problem to Fossil itself.

(18) By Keith Medcalf (kmedcalf) on 2023-03-21 22:50:28 in reply to 17 [link] [source]

Yes, it is rare. And yes, in order to fix it you have to delete the whole repo and re-clone it. I think the root of the problem is that it is possible to "unmerge" a commit on a branch in the "master" where the commit has already been replicated and then is attempted to be "reverted" on a later sync after it has already been propagated.

I have asked several times when this happened how to fix it, but no one knows how other than to delete the entire repository and start afresh.

(19) By Vadim Goncharov (nuclight) on 2023-03-29 07:07:54 in reply to 9 [link] [source]

There are many-many systems, e.g. e-mail and NNTP, git and even fossil itself, where trying to make in "consistent" will be not only just waste of effort, but also severely restrict their usability if succeed. And as decentralized systems are harder then centralized swelter, this is the opposite of kiddies, who lost ability even to federate and doing just centralized web-sites nowadays.

(5) By Marco Bambini (marcobambini) on 2023-03-18 06:51:27 in reply to 2 [link] [source]

Thanks Simon,

SDK(s) source codes and the SQLite bridge source code will be released next week.

(8) By Marco Bambini (marcobambini) on 2023-03-21 16:08:24 in reply to 5 [link] [source]

Repo with SDK(s) source code is now public: https://github.com/sqlitecloud/sdk

(20) By ThanksRyan on 2024-06-12 02:24:28 in reply to 1 [link] [source]

Podcast about SQLite with Marco. I have not yet listened to it:
Why SQLite is Taking Over with Brian Holt & Marco Bambini

What does this mean on the sqlitecloud.io page:

Backed by the creator of SQLite.

Does Richard approve of the implementation or something?

(21) By punkish on 2024-06-13 14:54:28 in reply to 20 [link] [source]

From the transcript, at 31.01 mark

We have a huge endorsement because Doctor. Rip, which is the original SQLite author, is one of our investor.