AbsurderSQL: Taking SQLite on the Web Even Further

What if SQLite on the web could be even better?

A few years ago, James Long wrote about absurd-sql, a brilliant hack that made SQLite work persistently in the browser by implementing a virtual filesystem on top of IndexedDB. It was groundbreaking work that showed the web platform's potential for serious database applications.

But as impressive as absurd-sql was, it had fundamental limitations. The biggest one? Your data was permanently locked in IndexedDB. No export, no import, no backups, no portability. Once your data was in there, it was stuck.

AbsurderSQL takes the absurd even further—it's absurdly absurder than absurd-sql. Written from scratch in Rust and compiled to WebAssembly, it implements a custom VFS (Virtual File System) that treats IndexedDB like a disk, storing data in 4KB blocks with intelligent LRU caching. It basically stores a whole database into another database. Which is absurder. But it gets even more absurd from here by adding dual-mode architecture, full export/import, production-grade multi-tab coordination, native CLI support, and optional observability. Your data is never locked in.

---

Why I Built This

I started down this whole journey by refactoring a legacy VBA application into a modern Next.js single-page application. My original plan was a standard PERN stack, but there was a critical caveat: the application dealt with sensitive PII, and there was too much red tape to get any solution approved that sent or persisted data server-side or worked online.

I knew about IndexedDB from previous projects, but making a transactional, object-oriented, non-relational database fill the role of a proper RDBMS was—and even now is—no trivial thing. But I needed the application to work completely offline while maintaining a single source of truth.

Eventually, I stumbled upon absurd-sql and rearchitected the entire application to use it. As grateful as I am to James's groundbreaking work, I encountered some inescapable issues and limitations that stuck with me.

That's why I built AbsurderSQL—from inspiration and frustration.

---

The Problem That Drove Me Crazy

Here's the thing: IndexedDB was never meant to be your primary database. It's a browser storage API with some database-like features. The moment you build your app around it, you've committed to:

• Data lock-in: Can't export your database to a file
• No migration path: Can't move data between browsers or devices
• No backup strategy: Hope the browser doesn't delete your data (it can!)
• No CLI tools: Can't query or analyze data outside the browser

absurd-sql solved the performance problem by using SQLite, but it inherited IndexedDB's portability problem. Your data was still stuck. This is what frustrated me most.

So I made AbsurderSQL solve this completely.

---

Your Data, Your Rules

I built the export/import system to give you what absurd-sql couldn't: complete control over your data.

import init, { Database } from './pkg/absurder_sql.js';

await init();
const db = await Database.newDatabase('myapp.db');

// Create some data
await db.execute('CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)');
await db.execute("INSERT INTO users VALUES (1, 'Alice')");

// Export to standard SQLite format
const exportedBytes = await db.exportToFile();

// Download as .db file
const blob = new Blob([exportedBytes], { type: 'application/octet-stream' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = 'myapp.db';
a.click();

That exported file is a standard SQLite database. Not a proprietary format, not some IndexedDB dump—a real SQLite file you can:

• Open with sqlite3 CLI
• Import into DB Browser for SQLite
• Query with Python's sqlite3 module
• Process with Rust's rusqlite
• Share, backup, version control, or sync to cloud storage

Import from Anywhere

// User uploads a .db file
const fileInput = document.getElementById('dbFile');
const file = fileInput.files[0];
const arrayBuffer = await file.arrayBuffer();
const bytes = new Uint8Array(arrayBuffer);

// Import it
let db = await Database.newDatabase('myapp.db');
await db.importFromFile(bytes);

// Reopen and use
db = await Database.newDatabase('myapp.db');
const result = await db.execute('SELECT * FROM users');

This unlocks powerful workflows:

• Backup/Restore: Export before major operations, restore if something breaks
• Data Migration: Move databases between devices, browsers, or platforms
• Offline Sync: Export on device A, transfer via USB/email, import on device B
• Multi-Device: Use cloud storage to sync databases across devices
• Development: Export production data, import locally for debugging
• Testing: Create test databases in CLI, import into browser for E2E tests

---

Dual-Mode Architecture: Browser AND Native

Here's where it really gets absurder: I made AbsurderSQL run in two modes.

Browser Mode (WASM)

In the browser, your app uses IndexedDB through SQLite—just like absurd-sql, but with better performance, export/import, and multi-tab coordination built-in. But here's the kicker: the same codebase compiles to native Rust, giving you a CLI tool or server that uses traditional filesystem storage.

// Browser (JavaScript)
import init, { Database } from './pkg/absurder_sql.js';

await init();
const db = await Database.newDatabase('myapp.db');
await db.execute("INSERT INTO todos (text) VALUES ('Buy milk')");

Data is stored in IndexedDB, but you're using SQLite to query it.

Native Mode (Rust CLI/Server)

The same codebase works natively too:

// Server (Rust)
use absurder_sql::database::SqliteIndexedDB;
use absurder_sql::types::DatabaseConfig;

let config = DatabaseConfig {
    name: "myapp.db".to_string(),
    ..Default::default()
};

let mut db = SqliteIndexedDB::new(config).await?;
let result = db.execute("SELECT * FROM todos WHERE synced = 0").await?;

Data is stored in ./absurdersql_storage/myapp/ with:

• database.sqlite - Standard SQLite file
• blocks/ - BlockStorage blocks with checksums
• metadata.json - Block metadata for integrity

Same Rust API. Same semantics. Different storage.

Why This Matters

Build an offline-first web app that occasionally syncs with a server:

1. Browser: Users work with local SQLite database (via IndexedDB)
2. Export: Export database to Uint8Array
3. Upload: POST to server API
4. Server: Rust backend processes with native performance
5. Sync: Server returns updated database
6. Import: Browser imports the synced database

No need for two different database systems. One codebase, one query language, two modes.

---

Multi-Tab Coordination That Actually Works

absurd-sql had no multi-tab coordination. Open your app in two tabs and you were on your own. Hope you didn't corrupt the database!

I built in leader election and write coordination from day one.

Automatic Leader Election

const db = await Database.newDatabase('myapp.db');

// Check if this tab is the leader
const isLeader = await db.isLeader();

if (isLeader) {
  console.log('👑 I am the leader - I can write');
} else {
  console.log('📖 I am a follower - read-only');
}

Using localStorage-based leader election, tabs automatically coordinate:

• One leader: Only the leader writes to IndexedDB
• Automatic failover: When leader tab closes, a follower becomes leader
• Heartbeat: Leader maintains heartbeat; stale leaders are replaced
• Write queuing: Followers can queue writes to forward to leader

Real-Time Change Notifications

// Listen for data changes from other tabs
db.onDataChange((changeType) => {
  console.log('Another tab changed data:', changeType);
  // Refresh your UI
  refreshTodoList();
});

// In another tab
await db.execute("INSERT INTO todos VALUES ('New task')");
await db.sync(); // Triggers BroadcastChannel notification

BroadcastChannel automatically notifies all tabs when data changes. No polling, no complexity, just works.

Write Queue for Non-Leaders

Even non-leader tabs can initiate writes:

// In a follower tab
const db = await Database.newDatabase('myapp.db');

// Queue a write (forwards to leader)
await db.queueWrite("INSERT INTO todos VALUES ('Background task')");

// Leader processes queued writes automatically

This gives you the flexibility of any-tab-writes with the safety of single-writer coordination.

---

Performance: Fast Enough to Build Real Apps

Let's talk numbers. IndexedDB is slow. In Chrome (the most popular browser), simple operations take ~10ms. SQLite in AbsurderSQL? ~0.01ms for cached operations.

Benchmark Results

Testing on a 2024 MacBook Pro with 100,000 rows:

Reads (SELECT SUM(value) FROM kv - scanning all rows):

• Raw IndexedDB cursor: ~2,500ms ❌
• AbsurderSQL (cold): ~800ms ✅ (3x faster)
• AbsurderSQL (warm cache): ~50ms ✅ (50x faster)

Writes (Bulk INSERT of 10,000 rows):

• Raw IndexedDB (bulk put): ~3,200ms ❌
• AbsurderSQL: ~600ms ✅ (5x faster)

Why?

1. Batching: AbsurderSQL batches IndexedDB operations
2. LRU Caching: 128-block cache (configurable) reduces reads
3. Smart Cursors: Detects sequential reads and uses IndexedDB cursors
4. Long-lived Transactions: Keeps transactions open during operations
5. SQLite's Query Optimizer: Much smarter than rolling your own indexes

Configurable Performance

const db = await Database.newDatabase({
  name: 'myapp.db',
  cache_size: 4000,  // 4000 pages = ~16MB cache (WASM defaults to 10,000)
  page_size: 8192,   // 8KB pages instead of 4KB (default: 4096)
});

Larger caches and page sizes = fewer IndexedDB operations = better performance.

---

Built for Production from Day One

Transaction Support

await db.execute('BEGIN TRANSACTION');
try {
  await db.execute("INSERT INTO accounts (id, balance) VALUES (1, 1000)");
  await db.execute("INSERT INTO accounts (id, balance) VALUES (2, 500)");
  await db.execute("UPDATE accounts SET balance = balance - 100 WHERE id = 1");
  await db.execute("UPDATE accounts SET balance = balance + 100 WHERE id = 2");
  await db.execute('COMMIT');
} catch (error) {
  await db.execute('ROLLBACK');
  throw error;
}

I made sure transactions have real ACID guarantees. ROLLBACK actually works. No auto-commit nonsense.

Crash Consistency

BlockStorage includes checksums for every block:

pub struct BlockMetadata {
    pub block_id: u32,
    pub checksum: u32,  // CRC32 checksum
    pub size: usize,
    pub dirty: bool,
}

Every block in storage has a CRC32 checksum. If a block gets corrupted, we detect it on startup and can recover from WAL or journal files.

Optional Observability

Enable production telemetry with --features telemetry:

[dependencies]
absurder-sql = { version = "0.1", features = ["telemetry"] }

Get:

• Prometheus metrics: Query latency, block I/O, cache hit rates
• OpenTelemetry traces: Distributed tracing across your stack
• Grafana dashboards: Pre-built dashboards for monitoring
• Alert rules: Production-ready alerts with runbooks
• DevTools extension: Chrome/Firefox extension for WASM debugging

rust // Metrics are automatically collected histogram!("sqlite_query_duration_seconds", duration); counter!("sqlite_cache_hits_total").increment(1); gauge!("sqlite_active_connections").set(conn_count as f64); `

All telemetry is opt-in. Default builds have zero overhead.

---

The Technical Deep Dive

How Export Works

1. Acquire export/import lock (prevents concurrent operations)
2. Read database size from SQLite
3. Call SQLite's serialization API to dump database to memory
4. Return Uint8Array with complete SQLite file
5. Release lock

The exported file is a byte-perfect SQLite database. Same format native SQLite uses.

How Import Works

1. Acquire export/import lock
2. Validate SQLite magic number (SQLite format 30)
3. Close current database connection
4. Clear all IndexedDB storage for this database
5. Parse SQLite file and write blocks to IndexedDB
6. Initialize metadata with checksums
7. Release lock

After import, you must reopen the database—the connection is closed.

Why Rust?

absurd-sql was built in JavaScript, patching sql.js and Emscripten internals. Clever, but fragile.

I wrote AbsurderSQL in Rust from the ground up:

• Type Safety: Rust's type system prevents entire classes of bugs
• Memory Safety: No segfaults, no memory leaks, no undefined behavior
• Performance: Compiled to native (CLI) or optimized WASM (browser)
• Fearless Concurrency: Rust's ownership model makes multi-threading safe
• Modern Tooling: Cargo, clippy, rustfmt, comprehensive testing
• WASM-First: Built with wasm-bindgen for excellent JS interop

The VFS layer is ~3,000 lines of clean Rust—not hacked-together C++ patches.

Async Without Asyncify

absurd-sql achieves synchronous-like behavior in the browser with two modes: a fast path using SharedArrayBuffer in a web worker, and a fallback mode (FileOpsFallback) that works without special requirements. Both intentionally avoid Emscripten's Asyncify to prevent performance and binary size penalties. However, this approach has limitations including data lock-in to IndexedDB and lack of multi-tab coordination. I took a different approach. Using wasm-bindgen, we expose async Rust functions directly to JavaScript. The VFS layer is fully async, using Rust's async/await to coordinate with IndexedDB. This keeps the binary small (~1.6MB uncompressed, ~660KB gzipped).

File Locking via IndexedDB Transactions

AbsurderSQL implements SQLite's file locking protocol using IndexedDB's transaction semantics:

• Read lock: Open readonly transaction (multiple can run in parallel)
• Write lock: Open readwrite transaction (exclusive access, blocks all others)
• Lock upgrade: Check SQLite's change counter before writing

This provides proper coordination without requiring SharedArrayBuffer or complex message passing. Multiple tabs can read simultaneously, but writes are serialized through IndexedDB's transaction system.

---

What's Not Great (Yet)

Let's be honest about limitations:

1. Still Slower Than Native SQLite

AbsurderSQL is 50-100x slower than native SQLite because IndexedDB is slow. We've done everything we can:

• Batching operations
• LRU caching
• Smart cursor usage
• Long-lived transactions

But we're ultimately limited by IndexedDB's performance. Chrome is especially slow.

Reality check: We're doing everything possible with current browser APIs. Future improvements will require better storage primitives from browser vendors, which currently aren't on any roadmap.

2. IndexedDB Durability Isn't Guaranteed

Browsers can delete your IndexedDB data under storage pressure. This rarely happens, but it's possible.

For critical data:

• Enable persistent storage: await navigator.storage.persist()
• Regular backups: await db.exportToFile() to cloud storage
• Assume cloud backup is your source of truth

We're hoping for a better durable storage API in the future.

---

Comparison with absurd-sql

Feature	absurd-sql	AbsurderSQL
Language	JavaScript (patched C++)	Rust (clean implementation)
Binary Size	~2.5MB (with Asyncify)	~660KB gzipped
Export to File	❌ Not supported	✅ Standard SQLite format
Import from File	❌ Not supported	✅ Full validation
Multi-Tab Coordination	❌ Manual only	✅ Built-in leader election
Write Queue	❌ None	✅ Forward writes to leader
Native/CLI Mode	❌ Browser only	✅ Dual-mode architecture
Crash Consistency	⚠️ Basic	✅ Block checksums + recovery
Transaction Support	⚠️ Limited	✅ Full ACID semantics
Observability	❌ None	✅ Optional Prometheus/OTel
Change Notifications	❌ Manual	✅ BroadcastChannel
Typed API	❌ Dynamic	✅ Rust types + WASM bindings

---

Getting Started

Installation

npm install @npiesco/absurder-sql

Note: You can also build from source:

git clone https://github.com/npiesco/absurder-sql
cd absurder-sql
wasm-pack build --target web --out-dir pkg

Basic Usage

import init, { Database } from '@npiesco/absurder-sql';

// Initialize WASM
await init();

// Create database
const db = await Database.newDatabase('myapp.db');

// Create table
await db.execute(
  CREATE TABLE IF NOT EXISTS tasks (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    text TEXT NOT NULL,
    completed BOOLEAN DEFAULT 0,
    created_at TIMESTAMP DEFAULT CURRENT_TIMESTAMP
  )
);

// Insert data
await db.execute("INSERT INTO tasks (text) VALUES ('Buy groceries')");
await db.execute("INSERT INTO tasks (text) VALUES ('Walk the dog')");

// Query data
const result = await db.execute('SELECT * FROM tasks WHERE completed = 0');
console.log('Pending tasks:', result.rows);

// Export for backup
const backup = await db.exportToFile();
localStorage.setItem('db_backup', JSON.stringify(Array.from(backup)));

// Close when done
await db.close();

Multi-Tab App

import init, { Database } from '@npiesco/absurder-sql';

await init();
const db = await Database.newDatabase('myapp.db');

// Check leadership
const isLeader = await db.isLeader();
document.getElementById('status').textContent = isLeader ? '👑 Leader' : '📖 Follower';

// Listen for changes
db.onDataChange(() => {
  console.log('Data changed in another tab!');
  refreshUI();
});

// Write (works in any tab via queue)
async function addTask(text) {
  if (await db.isLeader()) {
    await db.execute(INSERT INTO tasks (text) VALUES ('${text}'));
    await db.sync(); // Notify other tabs
  } else {
    await db.queueWrite(INSERT INTO tasks (text) VALUES ('${text}'));
  }
}

---

Where I'm Taking This Next

Native Mobile

Rust compiles to iOS and Android. I could run the same AbsurderSQL code natively on mobile with real filesystem storage.

WASM Component Model

The WASM Component Model: I want to ship AbsurderSQL as a language-agnostic component. Use it from Python, Go, or any language that can load WASM components.

Better Browser Storage APIs

I designed the architecture to support pluggable backends. When better browser storage APIs arrive, we'll be ready.

---

Come Build With Us

AbsurderSQL is production-ready, but I'm actively evolving it. I'd love your help:

• Try it: Build something and tell us what breaks
• Benchmark it: Run our benchmarks on your hardware
• Improve it: The codebase is clean Rust—contributions welcome
• Break it: Find edge cases and file issues

GitHub: npiesco/absurder-sql

License: AGPL-3.0

---

Final Thoughts

James Long showed us that SQLite on the web was possible. He proved IndexedDB could be fast enough if you were clever about it.

AbsurderSQL shows it can be even better.

By rewriting in Rust, adding export/import, building dual-mode architecture, and designing for production from day one, we've created something that goes beyond "SQLite in the browser" to become a complete offline-first database solution.

Your data isn't locked in IndexedDB anymore. You can export it, analyze it with CLI tools, sync it across devices, back it up to cloud storage, and process it on servers—all while giving users a fast, offline-first web experience.

Excited to see what absurder things you build with AbsurderSQL!

---

— Nicholas G. Piesco October 2025