SQLite + IndexedDB + Custom VFS that's absurdly absurder than absurd-sql
It implements a custom SQLite Virtual File System (VFS) backend that treats IndexedDB like a disk and stores data in blocks there. Your database lives permanently in browser storage with intelligent block-level I/O—reading and writing 4KB chunks with LRU caching—avoiding the performance nightmare of serializing entire database files on every operation.
It basically stores a whole database into another database using a custom VFS. Which is absurder.
But AbsurderSQL takes it further: it's absurdly better. Unlike absurd-sql, your data isn't locked in IndexedDB forever—you can export and import standard SQLite files. Need to query from both browser and CLI? Use dual-mode persistence—same database structure, IndexedDB in the browser and real files on the server. Multiple tabs? Multi-tab coordination with automatic leader election prevents conflicts. Want production observability? Optional Prometheus + Grafana monitoring with DevTools extension for debugging WASM telemetry.
Read the blog post that explains the absurdity in detail.
A high-performance tri-mode Rust library that brings full SQLite functionality to browsers, native applications, and mobile devices:
- Browser (WASM): SQLite → IndexedDB with multi-tab coordination, Web Worker support, and full export/import
- Native/CLI: SQLite → Real filesystem with traditional
.dbfiles - Mobile (React Native): SQLite → Device filesystem via UniFFI with SQLCipher encryption for iOS and Android
Unique Advantages:
Export/import databases as standard SQLite files (absurd-sql has no export/import—data is permanently locked in IndexedDB). Build web apps that store data in IndexedDB, then query the same database structure from CLI/server using standard SQLite tools. Multi-tab coordination with automatic leader election prevents conflicts. Perfect for offline-first applications with backup/restore, data migration, and optional server synchronization.
Production Observability (Optional): When enabled with --features telemetry, includes complete monitoring stack: Prometheus metrics, OpenTelemetry tracing, pre-built Grafana dashboards, production-ready alert rules with runbooks, and a Chrome/Firefox DevTools extension for debugging WASM telemetry. All telemetry features are opt-in—default builds include zero monitoring overhead.
Enabling production-ready SQL operations with crash consistency, multi-tab coordination, complete data portability, optional observability, and the flexibility to run anywhere from web apps to server applications.
AbsurderSQL runs in three modes - Browser (WASM), Native (Rust CLI/Server), and Mobile (React Native):
graph TB
subgraph "Browser Environment (WASM)"
JS["JavaScript/TypeScript<br/>Web Application"]
WASM["WASM Bridge<br/>(wasm-bindgen)"]
end
subgraph "Native Environment (Rust)"
CLI["CLI/Server<br/>Application"]
NATIVE_DB["Native Database API<br/>(database.rs)"]
end
subgraph "Mobile Environment (React Native)"
RN["React Native App<br/>(TypeScript)"]
UNIFFI["UniFFI Bridge<br/>(Swift/Kotlin)"]
end
subgraph "AbsurderSQL Core (Rust)"
DB["Database API<br/>(lib.rs)"]
SQLITE["SQLite Engine<br/>(sqlite-wasm-rs / rusqlite)"]
VFS["Custom VFS Layer<br/>(indexeddb_vfs.rs)"]
subgraph "Storage Layer"
BS["BlockStorage<br/>(block_storage.rs)"]
SYNC["Sync Operations<br/>(sync_operations.rs)"]
META["Metadata Manager<br/>(metadata.rs)"]
CACHE["LRU Cache<br/>(128 blocks)"]
ALLOC["Allocation<br/>(allocation.rs)"]
EXPORT["Export<br/>(export.rs)"]
IMPORT["Import<br/>(import.rs)"]
end
subgraph "Multi-Tab Coordination (Browser Only)"
LEADER["Leader Election<br/>(leader_election.rs)"]
BCAST["BroadcastChannel<br/>(broadcast_notifications.rs)"]
QUEUE["Write Queue<br/>(write_queue.rs)"]
end
subgraph "Monitoring & Recovery"
OBS["Observability<br/>(observability.rs)"]
RECOVERY["Recovery<br/>(recovery.rs)"]
end
end
subgraph "Browser Persistence"
INDEXEDDB["IndexedDB<br/>(Browser Storage)"]
LOCALSTORAGE["localStorage<br/>(Coordination)"]
end
subgraph "Native Persistence"
FILESYSTEM["Filesystem<br/>(Traditional .db files)"]
BLOCKS["./absurdersql_storage/<br/>database.sqlite + blocks/"]
end
subgraph "Mobile Persistence"
DEVICE_FS["Device Filesystem<br/>(iOS/Android Storage)"]
SQLCIPHER["SQLCipher<br/>(AES-256 Encryption)"]
end
subgraph "Telemetry Stack (optional --features telemetry)"
PROM["Prometheus<br/>(Metrics)"]
OTEL["OpenTelemetry<br/>(Traces)"]
GRAFANA["Grafana Dashboards"]
ALERTS["Alert Rules"]
DEVTOOLS["DevTools Extension<br/>(Chrome/Firefox)"]
end
JS -->|execute/query| WASM
WASM -->|calls| DB
CLI -->|execute/query| NATIVE_DB
NATIVE_DB -->|SQL| SQLITE
RN -->|execute/query| UNIFFI
UNIFFI -->|calls| DB
DB -->|SQL| SQLITE
DB -->|exportToFile| EXPORT
DB -->|importFromFile| IMPORT
SQLITE -->|VFS calls| VFS
VFS -->|block I/O| BS
BS -->|read/write| CACHE
BS -->|allocate| ALLOC
BS -->|persist| SYNC
SYNC -->|metadata| META
EXPORT -->|read blocks| BS
IMPORT -->|write blocks| BS
BS -->|"WASM mode"| INDEXEDDB
BS -->|"Native mode"| FILESYSTEM
NATIVE_DB -->|"fs_persist"| BLOCKS
UNIFFI -->|"Mobile mode"| DEVICE_FS
DEVICE_FS -->|"encryption"| SQLCIPHER
BS -->|metrics| OBS
LEADER -->|atomic ops| LOCALSTORAGE
LEADER -->|notify| BCAST
QUEUE -->|forward| BCAST
OBS -->|"--features telemetry"| PROM
OBS -->|"--features telemetry"| OTEL
PROM -->|scrape| GRAFANA
PROM -->|evaluate| ALERTS
OTEL -->|spans| DEVTOOLS
style SQLITE fill:#a855f7,stroke:#333,color:#fff
style VFS fill:#3b82f6,stroke:#333,color:#fff
style BS fill:#fbbf24,stroke:#333,color:#000
style EXPORT fill:#ec4899,stroke:#333,color:#fff
style IMPORT fill:#8b5cf6,stroke:#333,color:#fff
style INDEXEDDB fill:#22c55e,stroke:#333,color:#000
style QUEUE fill:#ef4444,stroke:#333,color:#fff
style OBS fill:#92400e,stroke:#333,color:#fff
style PROM fill:#1c1c1c,stroke:#333,color:#fff
style GRAFANA fill:#f97316,stroke:#333,color:#fff
%% Default styling for uncolored blocks (light gray)
style JS fill:#d1d5db,stroke:#333,color:#000
style WASM fill:#d1d5db,stroke:#333,color:#000
style CLI fill:#d1d5db,stroke:#333,color:#000
style NATIVE_DB fill:#d1d5db,stroke:#333,color:#000
style DB fill:#d1d5db,stroke:#333,color:#000
style SYNC fill:#d1d5db,stroke:#333,color:#000
style META fill:#d1d5db,stroke:#333,color:#000
style CACHE fill:#d1d5db,stroke:#333,color:#000
style ALLOC fill:#d1d5db,stroke:#333,color:#000
style LEADER fill:#d1d5db,stroke:#333,color:#000
style BCAST fill:#d1d5db,stroke:#333,color:#000
style RECOVERY fill:#d1d5db,stroke:#333,color:#000
style LOCALSTORAGE fill:#d1d5db,stroke:#333,color:#000
style FILESYSTEM fill:#d1d5db,stroke:#333,color:#000
style BLOCKS fill:#d1d5db,stroke:#333,color:#000
style RN fill:#61dafb,stroke:#333,color:#000
style UNIFFI fill:#10b981,stroke:#333,color:#fff
style DEVICE_FS fill:#d1d5db,stroke:#333,color:#000
style SQLCIPHER fill:#f59e0b,stroke:#333,color:#000
style OTEL fill:#d1d5db,stroke:#333,color:#000
style ALERTS fill:#d1d5db,stroke:#333,color:#000
style DEVTOOLS fill:#d1d5db,stroke:#333,color:#000
Legend: 🟪 SQLite Engine • 🟦 VFS Layer • 🟨 BlockStorage • 🟩 Persistence • 🟥 Multi-Tab 🟫 Observability • ⬛ Prometheus • 🟧 Grafana • 🩵 React Native • 🟢 UniFFI
absurder-sql/
├── src/
│ ├── lib.rs # WASM entry point, Database API exports
│ ├── database.rs # Native Database implementation
│ ├── types.rs # Core types (QueryResult, ColumnValue, etc.)
│ ├── utils.rs # Utility functions
│ │
│ ├── bin/ # Binary executables
│ │ └── cli_query.rs # CLI query tool for filesystem databases
│ │
│ ├── storage/ # Storage layer implementation
│ │ ├── mod.rs
│ │ ├── block_storage.rs # Core block storage with LRU cache
│ │ ├── sync_operations.rs # Cross-platform sync logic
│ │ ├── io_operations.rs # Read/write operations
│ │ ├── allocation.rs # Block allocation/deallocation
│ │ ├── metadata.rs # Block metadata management
│ │ ├── export.rs # Database export to SQLite files
│ │ ├── import.rs # Database import from SQLite files
│ │ ├── retry_logic.rs # Retry logic for transient failures
│ │ ├── fs_persist.rs # Native filesystem persistence
│ │ ├── wasm_indexeddb.rs # WASM IndexedDB integration
│ │ ├── wasm_vfs_sync.rs # WASM VFS sync coordination
│ │ ├── recovery.rs # Crash recovery logic
│ │ ├── auto_sync.rs # Native auto-sync
│ │ ├── wasm_auto_sync.rs # WASM auto-sync
│ │ ├── leader_election.rs # Multi-tab leader election
│ │ ├── broadcast_notifications.rs # BroadcastChannel messaging
│ │ ├── write_queue.rs # Write queuing for non-leaders
│ │ ├── optimistic_updates.rs # Optimistic UI updates
│ │ ├── coordination_metrics.rs # Performance metrics tracking
│ │ ├── observability.rs # Metrics and monitoring
│ │ └── constructors.rs # BlockStorage constructors
│ │
│ └── vfs/ # SQLite VFS implementation
│ ├── mod.rs
│ └── indexeddb_vfs.rs # Custom VFS for IndexedDB
│
├── tests/ # Comprehensive test suite
│ ├── integration_tests.rs # End-to-end tests
│ ├── native_database_persistence_tests.rs # Native filesystem tests
│ ├── wasm_integration_tests.rs # WASM-specific tests (inc. export/import)
│ ├── export_import_examples_test.rs # Export/import validation tests
│ ├── export_import_lock_tests.rs # Export/import locking tests
│ ├── vfs_durability_tests.rs # VFS durability tests
│ ├── lru_cache_tests.rs # Cache tests
│ ├── e2e/ # Playwright E2E tests
│ │ ├── dual_mode_persistence.spec.js # Browser + CLI validation
│ │ ├── advanced-features.spec.js
│ │ └── multi-tab-vite.spec.js
│ └── ... # 65+ test files total
│
├── examples/ # Browser demos and documentation
│ ├── vite-app/ # Production Vite application
│ ├── export_import_demo.html # Export/import 4-step wizard demo
│ ├── export_import.js # 9 production export/import examples
│ ├── test_export_import_examples.html # Export/import test suite
│ ├── sql_demo.html # Interactive SQL demo page
│ ├── web_demo.html # Full-featured web interface
│ ├── benchmark.html # Performance comparison tool
│ ├── multi-tab-demo.html # Multi-tab coordination demo
│ ├── worker-example.html # Web Worker demo
│ ├── worker-db.js # Web Worker implementation
│ ├── devtools_demo.html # DevTools extension telemetry demo
│ └── DEMO_GUIDE.md # Demo usage guide
│
├── docs/ # Comprehensive documentation
│ ├── EXPORT_IMPORT.md # Export/import guide (DATABASE PORTABILITY)
│ ├── DUAL_MODE.md # Tri-mode persistence guide
│ ├── MULTI_TAB_GUIDE.md # Multi-tab coordination
│ ├── TRANSACTION_SUPPORT.md # Transaction handling
│ ├── BENCHMARK.md # Performance benchmarks
│ ├── ENCRYPTION.md # SQLCipher encryption guide
│ ├── CODING_STANDARDS.md # Development best practices
│ ├── REMAINING_UNWRAPS.md # Unwrap safety analysis
│ └── mobile/ # Mobile-specific documentation
│ ├── INSTRUCTIONS.md # Mobile build instructions
│ └── MOBILE_BENCHMARK.md # Mobile performance benchmarks
│
├── monitoring/ # Production observability (optional --features telemetry)
│ ├── grafana/ # Pre-built Grafana dashboards
│ │ ├── query_performance.json
│ │ ├── storage_operations.json
│ │ ├── system_health.json
│ │ └── multi_tab_coordination.json
│ ├── prometheus/ # Alert rules and recording rules
│ │ └── alert_rules.yml # Alert rules + recording rules
│ └── RUNBOOK.md # Alert runbooks and remediation procedures
│
├── browser-extension/ # Browser DevTools extension (Chrome/Firefox)
│ ├── manifest.json # Manifest V3 extension configuration
│ ├── devtools.js # Message hub (devtools page)
│ ├── content.js # Content script (page bridge)
│ ├── panel.html/css/js # DevTools panel UI
│ ├── icons/ # Extension icons (16, 48, 128)
│ ├── README.md # Extension features and architecture
│ └── INSTALLATION.md # Installation guide
│
├── absurder-sql-mobile/ # React Native bindings (iOS + Android)
│ ├── src/
│ │ ├── uniffi_api/ # UniFFI exported functions
│ │ │ ├── core.rs # 20 exported database functions
│ │ │ └── types.rs # QueryResult, DatabaseConfig, etc.
│ │ ├── AbsurderDatabase.ts # High-level TypeScript API
│ │ └── lib.rs # Crate entry point
│ ├── android/ # Android-specific (Kotlin bindings, SQLCipher libs)
│ ├── ios/ # iOS-specific (Swift bindings)
│ ├── react-native/ # Test app for iOS/Android
│ ├── scripts/ # Build helper scripts
│ └── README.md # Mobile setup guide
│
├── pkg/ # WASM build output (generated)
├── Cargo.toml # Rust dependencies and config
├── package.json # Node.js dependencies
└── README.md # This file
The project follows a modular architecture with clear separation of concerns:
VFS Layer: Implements a custom SQLite Virtual File System that translates SQLite's file operations to IndexedDB operations. This allows SQLite to work seamlessly with browser storage without modifications to the core SQLite engine.
Storage Abstraction: Provides a unified interface for different storage backends, with IndexedDB as the primary target. The design allows for future expansion to other storage mechanisms while maintaining API compatibility.
WASM Bridge: Handles the interface between Rust code and JavaScript, managing memory allocation, type conversions, and async operation bridging. Uses sqlite-wasm-rs for stable SQLite operations without the hang issues that affected previous implementations. This ensures smooth interoperability between the WASM module and browser JavaScript.
Type System: Defines comprehensive data structures for SQL operations, query results, and configuration options, ensuring type safety across the Rust-JavaScript boundary.
The web demo uses vanilla JavaScript with Bootstrap for styling, demonstrating real-time SQL query execution and result visualization. The frontend architecture emphasizes simplicity and direct WASM integration without complex frameworks.
Primary Storage: IndexedDB serves as the persistent storage layer, chosen for its transaction support, large storage capacity, and widespread browser compatibility.
Memory Management: The library implements careful memory management for WASM operations, ensuring proper cleanup of allocated memory and efficient data transfer between Rust and JavaScript contexts.
Transaction Handling: Leverages SQLite's transaction capabilities while ensuring proper coordination with IndexedDB's transaction model for data consistency.
The architecture supports configurable database options including cache size, synchronization modes, and VFS-specific settings, allowing optimization for different use cases and performance requirements.
AbsurderSQL provides complete database export and import functionality - a critical feature that absurd-sql completely lacks.
With AbsurderSQL, your data is never locked in the browser. You can:
- Export databases as standard SQLite files
- Import databases from SQLite files
- Backup your data with one function call
- Migrate databases between devices and browsers
- Debug exported files with sqlite3 CLI or DB Browser
- Share databases as downloadable files
absurd-sql alternative: No export/import - data is permanently trapped in IndexedDB
// From npm package
import init, { Database } from '@npiesco/absurder-sql';
// Or from local build
// import init, { Database } from './pkg/absurder_sql.js';
await init();
const db = await Database.newDatabase('myapp.db');
// Create some data
await db.execute('CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)');
await db.execute("INSERT INTO users VALUES (1, 'Alice')");
// EXPORT: Get entire database as standard SQLite file
const exportedData = await db.exportToFile();
// Download for user
const blob = new Blob([exportedData], { type: 'application/octet-stream' });
const url = URL.createObjectURL(blob);
const a = document.createElement('a');
a.href = url;
a.download = 'myapp.db';
a.click();
// IMPORT: Load database from file
const file = document.getElementById('fileInput').files[0];
const arrayBuffer = await file.arrayBuffer();
await db.importFromFile(new Uint8Array(arrayBuffer));
// Database is immediately usable after import (no reopen needed)
const result = await db.execute('SELECT * FROM users');- Export/Import Demo - 4-step wizard with progress tracking
- Automated Tests - 5 validation tests
- Code Examples - 9 production-ready examples
See docs/EXPORT_IMPORT.md for:
- Complete API reference
- Architecture details
- Best practices & patterns
- Troubleshooting guide
- Size limits & performance
npm install @npiesco/absurder-sqlThen use in your project:
import init, { Database } from '@npiesco/absurder-sql';
// Initialize WASM
await init();
// Create database
const db = await Database.newDatabase('myapp.db');
// Use SQLite
await db.execute('CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)');
await db.execute("INSERT INTO users VALUES (1, 'Alice')");
const result = await db.execute('SELECT * FROM users');
// Export for backup
const backup = await db.exportToFile();
// Close when done
await db.close();Package includes:
- Pre-built WASM binary (~1.3MB, ~595KB gzipped)
- TypeScript definitions
- All necessary JavaScript glue code
- No telemetry dependencies - minimal size, zero observability overhead
Note: The npm package is built without the
telemetryfeature for smaller size and faster load times. If you need Prometheus/OpenTelemetry support for production monitoring, build from source with--features telemetry.
Add to your Cargo.toml:
[dependencies]
absurder-sql = "0.1.23"Or use cargo:
cargo add absurder-sqlAvailable on crates.io with full Rust documentation and dual-mode support (browser WASM + native CLI).
For development or custom builds:
Prerequisites:
- Rust 1.85.0+ with the 2024 edition
- wasm-pack for building WASM packages
- Node.js 18+ for running examples
# Clone the repository
git clone https://github.com/npiesco/absurder-sql
cd absurder-sql
# Install wasm-pack if needed
curl https://rustwasm.github.io/wasm-pack/installer/init.sh -sSf | sh
# Build for web
wasm-pack build --target web --out-dir pkgThis generates the pkg/ directory containing:
absurder_sql.js- JavaScript moduleabsurder_sql_bg.wasm- WebAssembly binary (~1.3MB)- TypeScript definitions and package files
AbsurderSQL supports optional feature flags to minimize dependencies and binary size:
The npm package does NOT include telemetry for minimal size. To enable telemetry features (Prometheus metrics, OpenTelemetry tracing), you must build from source with the telemetry feature flag:
# Build with telemetry support (Prometheus + OpenTelemetry)
wasm-pack build --target web --out-dir pkg --features telemetry
# Build without telemetry (default - smaller binary, zero telemetry overhead)
wasm-pack build --target web --out-dir pkgWhat telemetry provides:
- Prometheus metrics (queries, errors, cache hits/misses, block operations)
- OpenTelemetry distributed tracing spans
- Performance monitoring and observability
Dependencies (only when telemetry feature is enabled):
prometheus- Metrics collectionopentelemetry- Distributed tracingopentelemetry_sdk- Tracing SDKopentelemetry-prometheus- Prometheus exporter
When to use:
- Production monitoring and observability
- Performance debugging and profiling
- Integration with Grafana/Prometheus stack
- Not needed for typical applications (default build excludes it)
Exposing Metrics for Prometheus (Native Applications):
AbsurderSQL collects metrics in-memory but does NOT include an HTTP server. For Prometheus scraping, add a /metrics endpoint to your application's HTTP server:
// Example with axum
use absurder_sql::Database;
use prometheus::Encoder;
use axum::{Router, routing::get, extract::State};
async fn metrics_handler(State(db): State<Database>) -> String {
#[cfg(feature = "telemetry")]
if let Some(metrics) = db.metrics() {
let encoder = prometheus::TextEncoder::new();
let metric_families = metrics.registry().gather();
return encoder.encode_to_string(&metric_families).unwrap();
}
"Telemetry not enabled".to_string()
}
#[tokio::main]
async fn main() {
let db = Database::new("myapp.db").await.unwrap();
let app = Router::new()
.route("/metrics", get(metrics_handler))
.with_state(db);
axum::Server::bind(&"0.0.0.0:9090".parse().unwrap())
.serve(app.into_make_service())
.await
.unwrap();
}Then configure Prometheus to scrape http://localhost:9090/metrics.
Additional actix-web example:
use absurder_sql::Database;
use actix_web::{web, App, HttpServer, HttpResponse};
use prometheus::Encoder;
async fn metrics(db: web::Data<Database>) -> HttpResponse {
#[cfg(feature = "telemetry")]
if let Some(metrics) = db.metrics() {
let encoder = prometheus::TextEncoder::new();
let metric_families = metrics.registry().gather();
let body = encoder.encode_to_string(&metric_families).unwrap();
return HttpResponse::Ok()
.content_type("text/plain; version=0.0.4")
.body(body);
}
HttpResponse::Ok().body("Telemetry not enabled")
}When --features telemetry is enabled, AbsurderSQL provides a complete production observability stack.
Complete Monitoring Setup Guide | Alert Runbook | DevTools Extension
Pre-built Grafana dashboards provide real-time visibility into database operations:
-
Query Performance Dashboard (
monitoring/grafana/query_performance.json)- Query execution times (p50, p90, p99)
- Query rate and error rate
- Slow query detection
- 7 panels with drill-down capabilities
-
Storage Operations Dashboard (
monitoring/grafana/storage_operations.json)- Block read/write rates
- Cache hit rates and effectiveness
- Storage layer latency
- 6 panels for storage health monitoring
-
System Health Dashboard (
monitoring/grafana/system_health.json)- Error rates by type
- Transaction success rates
- Resource utilization
- 8 panels for overall system health
-
Multi-Tab Coordination Dashboard (
monitoring/grafana/multi_tab_coordination.json)- Leader election status
- Write queue depth
- Sync operations
- 7 panels for multi-tab debugging
Import dashboards: Load the JSON files directly into Grafana. All dashboards include variable templates for filtering by database instance.
Production-ready Prometheus alert rules with runbooks:
Critical Alerts:
- High error rate (>5% over 5 minutes)
- Extreme query latency (p99 > 1 second)
- Zero query throughput (potential deadlock)
- Storage failures (>3 failures per minute)
Warning Alerts:
- Elevated error rates, slow queries, cache inefficiency
- Multi-tab coordination issues
Info Alerts:
- New leader elected, first query executed
Recording Rules:
- Pre-aggregated metrics for faster dashboard queries
- Query rate calculations, error ratios, latency percentiles
All alerts include:
- Severity labels (critical/warning/info)
- Team routing (database-team/platform-team)
- Detailed annotations explaining the issue
- Links to runbooks with remediation steps
Location: monitoring/prometheus/alert_rules.yml
Runbooks: monitoring/RUNBOOK.md provides step-by-step debugging and remediation procedures for every alert type.
Chrome/Firefox extension for debugging WASM telemetry in the browser:
Features:
- Real-time span visualization with filtering
- Export statistics (success/failure rates)
- Manual flush trigger
- Buffer inspection
- OTLP endpoint configuration
Architecture: Manifest V3 compliant with proper message passing (page → content script → devtools hub → panel)
Installation:
- Chrome: Load unpacked from
browser-extension/ - Firefox: Load temporary add-on from
browser-extension/manifest.json
See browser-extension/README.md and browser-extension/INSTALLATION.md for complete setup instructions.
Demo Page: examples/devtools_demo.html generates sample telemetry for testing the extension.
Enable native filesystem persistence for CLI/server applications:
# Build with filesystem persistence
cargo build --features fs_persist
# Run tests with filesystem persistence
cargo test --features fs_persistNote: All telemetry code is properly feature-gated - when the telemetry feature is disabled, zero telemetry code is compiled into your binary. This ensures minimal binary size and zero runtime overhead for applications that don't need observability features.
// From npm package
import init, { Database } from '@npiesco/absurder-sql';
// Or from local build
// import init, { Database } from './pkg/absurder_sql.js';
// Initialize WASM
await init();
// Create database - persists to IndexedDB
const db = await Database.newDatabase('myapp');
// Execute SQL
await db.execute('CREATE TABLE users (id INT, name TEXT)');
await db.execute("INSERT INTO users VALUES (1, 'Alice')");
const result = await db.execute('SELECT * FROM users');
// Persist to IndexedDB
await db.sync();
// Close
await db.close();AbsurderSQL fully supports Web Workers for off-thread database operations, keeping your main thread responsive during heavy database work:
// worker.js
import init, { Database } from '@npiesco/absurder-sql';
self.onmessage = async function(e) {
// Initialize WASM in worker
await init();
const db = await Database.newDatabase('worker_db.db');
// Workers don't have localStorage for leader election
// Enable non-leader writes for worker context
db.allowNonLeaderWrites(true);
// Perform database operations
await db.execute('CREATE TABLE data (id INTEGER, value TEXT)');
await db.execute("INSERT INTO data VALUES (1, 'processed in worker')");
// Sync to IndexedDB
await db.sync();
const result = await db.execute('SELECT * FROM data');
await db.close();
// Send results back to main thread
self.postMessage({ success: true, rows: result.rows });
};// main.js
const worker = new Worker('/worker.js', { type: 'module' });
worker.onmessage = (e) => {
console.log('Worker result:', e.data);
};
worker.postMessage({ type: 'processData' });Key Points:
- IndexedDB access works in both Window and Worker contexts
- Workers don't have
localStorage, so usedb.allowNonLeaderWrites(true)for single-worker scenarios - All database operations run off the main thread
- Perfect for heavy data processing, imports, or background sync
Example: See examples/worker-example.html for a complete working demo
# Build the CLI tool
cargo build --bin cli_query --features fs_persist --release
# Create table
cargo run --bin cli_query --features fs_persist -- \
"CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT, email TEXT)"
# Insert data
cargo run --bin cli_query --features fs_persist -- \
"INSERT INTO users (name, email) VALUES ('Alice', '[email protected]')"
# Query data
cargo run --bin cli_query --features fs_persist -- \
"SELECT * FROM users"
# Special commands
cargo run --bin cli_query --features fs_persist -- ".tables"
cargo run --bin cli_query --features fs_persist -- ".schema"Data Location: ./absurdersql_storage/<db_name>/database.sqlite
See docs/DUAL_MODE.md for complete tri-mode guide.
AbsurderSQL Mobile provides native SQLite for iOS and Android via UniFFI-generated bindings:
import { AbsurderDatabase, openDatabase } from 'absurder-sql-mobile';
// Simple usage with high-level API
const db = await openDatabase({
name: 'myapp.db',
encryption: { key: 'my-secret-key' }, // Optional SQLCipher encryption
cacheSize: 20000, // 20K pages (~80MB cache)
journalMode: 'WAL', // Write-ahead logging
});
// Execute queries
await db.execute('CREATE TABLE users (id INTEGER PRIMARY KEY, name TEXT)');
await db.execute("INSERT INTO users VALUES (1, 'Alice')");
// Query with parameters
const result = await db.executeWithParams(
'SELECT * FROM users WHERE id = ?',
[1]
);
console.log(result.rows); // [{ id: 1, name: 'Alice' }]
console.log(result.lastInsertId); // Last inserted row ID
console.log(result.executionTimeMs); // Query timing
// Transactions
await db.transaction(async () => {
await db.execute("INSERT INTO users VALUES (2, 'Bob')");
await db.execute("INSERT INTO users VALUES (3, 'Charlie')");
});
// Streaming for large datasets
for await (const row of db.executeStream('SELECT * FROM large_table')) {
console.log(row);
}
// Export/Import
await db.exportToFile('/path/to/backup.db');
await db.importFromFile('/path/to/restore.db');
await db.close();Features:
- SQLCipher AES-256 encryption (iOS uses CommonCrypto, Android uses pre-built OpenSSL)
- Prepared statements, batch operations, streaming queries
- Full export/import for backup/restore
- Type-safe from Rust to TypeScript via UniFFI
Setup: See absurder-sql-mobile/README.md for build instructions.
Building mobile apps requires native toolchains and Rust cross-compilation targets. Follow these steps to set up your development environment:
Prerequisites:
- Rust 1.85.0+ with iOS/Android targets
- Xcode (iOS) with Command Line Tools
- Android Studio (Android) with NDK
1. Install Rust iOS targets:
rustup target add aarch64-apple-ios aarch64-apple-ios-sim x86_64-apple-ios2. Build iOS bindings with UniFFI:
cd absurder-sql-mobile
npx uniffi-bindgen-react-native build ios --and-generate3. Install CocoaPods dependencies:
cd your-react-native-app/ios && pod install && cd ..4. Run on iOS Simulator:
npx react-native run-ios --simulator="iPhone 16"Android Setup:
# Install Rust Android targets
rustup target add aarch64-linux-android armv7-linux-androideabi x86_64-linux-android i686-linux-android
# Build Android bindings
cd absurder-sql-mobile
npx uniffi-bindgen-react-native build android --and-generate
# Run on Android Emulator
cd your-react-native-app && npx react-native run-androidImportant: The
uniffi-bindgen-react-native buildstep compiles the Rust native library for the target platform and generates the TypeScript/Swift/Kotlin bindings. This must be run beforepod install(iOS) or Gradle build (Android).
AbsurderSQL includes several performance optimizations for high-throughput applications:
Filesystem sync operations are automatically deferred during transactions for massive performance improvements:
// Without transactions: ~2883ms for 1000 inserts (sync after each)
for (let i = 0; i < 1000; i++) {
await db.execute(`INSERT INTO data VALUES (${i}, 'value${i}')`);
}
// With transactions: <1ms for 1000 inserts (sync only on COMMIT)
await db.execute('BEGIN TRANSACTION');
for (let i = 0; i < 1000; i++) {
await db.execute(`INSERT INTO data VALUES (${i}, 'value${i}')`);
}
await db.execute('COMMIT'); // Single sync herePerformance: 2278x speedup for bulk inserts (1034ms → 0.45ms for 100 inserts)
Execute multiple SQL statements in one call to reduce bridge overhead (especially important for React Native):
// Native API (browser/WASM)
const statements = [
"INSERT INTO users VALUES (1, 'Alice')",
"INSERT INTO users VALUES (2, 'Bob')",
"INSERT INTO users VALUES (3, 'Charlie')"
];
await db.executeBatch(statements);Performance: Reduces N bridge calls to 1 call. For 5000 statements:
- Individual calls: ~170ms bridge overhead (0.034ms × 5000)
- Batch call: ~12ms total execution time
Eliminate SQL re-parsing overhead for repeated queries:
// Native/CLI usage only (not available in WASM)
use absurder_sql::{SqliteIndexedDB, DatabaseConfig, ColumnValue};
let mut db = SqliteIndexedDB::new(config).await?;
// Prepare once
let mut stmt = db.prepare("INSERT INTO users VALUES (?, ?, ?)")?;
// Execute many times
for i in 1..=1000 {
stmt.execute(&[
ColumnValue::Integer(i),
ColumnValue::Text(format!("User{}", i)),
ColumnValue::Integer(25 + (i % 50)),
]).await?;
}
// Cleanup
stmt.finalize()?;Supported parameter styles:
- Positional:
? - Numbered:
?1,?2(can reuse same parameter) - Named:
:name,:id
Performance: 1.5-2x faster for repeated queries (eliminates SQL parsing on each execution)
Note: PreparedStatement is currently available for native/CLI applications only. Browser/WASM support will be added in a future release.
The library provides a robust SQLite implementation for WebAssembly environments using the sqlite-wasm-rs crate with precompiled features. This ensures stable, production-ready SQLite functionality without the hang issues that plagued earlier custom implementations.
- Full SQLite C API Support: Complete implementation of
sqlite3_prepare_v2,sqlite3_step,sqlite3_finalize, and parameter binding - Memory Safety: Proper Rust
Droptrait implementation for automatic cleanup of SQLite resources - Async Operations: All database operations are async-compatible for seamless integration with browser event loops
- Type Safety: Comprehensive
ColumnValueenum supporting all SQLite data types (NULL, INTEGER, REAL, TEXT, BLOB, BIGINT, DATE) - JavaScript Interop: Complete
wasm-bindgenexports withWasmColumnValuewrapper for seamless JS integration - Database Encryption (Native): Optional SQLCipher integration with AES-256 encryption for secure data at rest (
--features encryption)
AbsurderSQL includes comprehensive multi-tab coordination for browser applications, ensuring data consistency across multiple tabs without conflicts.
- Automatic Leader Election: First tab becomes leader using localStorage coordination
- Write Guard: Only the leader tab can execute write operations (INSERT, UPDATE, DELETE)
- BroadcastChannel Sync: Automatic change notifications to all tabs
- Failover Support: Automatic re-election when leader tab closes
- Zero Configuration: Works out of the box, no setup required
- Write Queuing: Non-leaders can queue writes that forward to leader automatically
- Optimistic Updates: Track pending writes for immediate UI feedback
- Coordination Metrics: Monitor performance and coordination events
// From npm package
import init, { Database } from '@npiesco/absurder-sql';
// Or from local build
// import init, { Database } from './pkg/absurder_sql.js';
import { MultiTabDatabase } from './examples/multi-tab-wrapper.js';
await init();
// Create multi-tab database
const db = new MultiTabDatabase(Database, 'myapp.db', {
autoSync: true // Auto-sync after writes
});
await db.init();
// Check leader status
if (await db.isLeader()) {
// Only leader can write
await db.write("INSERT INTO users VALUES (1, 'Alice')");
}
// All tabs can read
const result = await db.query("SELECT * FROM users");
// Listen for changes from other tabs
db.onRefresh(() => {
console.log('Data changed in another tab!');
// Refresh UI
});// Write Queuing - Queue from any tab
await db.queueWrite("INSERT INTO logs VALUES (1, 'event')");
await db.queueWriteWithTimeout("UPDATE data SET processed = 1", 10000);
// Optimistic Updates - Track pending writes
await db.enableOptimisticUpdates(true);
const writeId = await db.trackOptimisticWrite("INSERT INTO users...");
const pendingCount = await db.getPendingWritesCount();
// Coordination Metrics - Monitor performance
await db.enableCoordinationMetrics(true);
await db.recordLeadershipChange(true);
await db.recordNotificationLatency(15.5);
const metrics = JSON.parse(await db.getCoordinationMetrics());
// Helper Methods
await db.waitForLeadership(); // Wait to become leader
await db.requestLeadership(); // Request leadership
const info = await db.getLeaderInfo(); // Get leader info
await db.allowNonLeaderWrites(true); // Override for single-tab apps- Multi-Tab Demo - Interactive task list with multi-tab sync
- Vite App - Production-ready multi-tab example
- Complete Guide - Full documentation and patterns
Open the demo in multiple browser tabs to see coordination in action!
Modern web app example with multi-tab coordination:
- ES modules with hot reload
- Multi-tab leader election
- Real-time sync across tabs
- Leader/follower UI indicators
- Production-ready build
cd examples/vite-app
npm install
npm run dev
# Open in multiple tabs!Comprehensive SQL operations demo:
- Table creation with foreign keys
- INSERT operations with transactions
- Complex SELECT queries with JOINs and aggregations
- UPDATE and DELETE operations
- Automatic IndexedDB persistence via
sync()calls
node examples/sql_demo.jsFull-featured interactive SQL interface:
- Visual query editor
- Real-time query execution and result display
- Console output for debugging
- Quick action buttons for common operations
- Automatic sync after write operations
npm run serve
# Open http://localhost:8080/examples/web_demo.htmlAbsurderSQL consistently outperforms absurd-sql and raw IndexedDB across all operations.
Full benchmark results and analysis
| Implementation | Insert | Read | Update | Delete |
|---|---|---|---|---|
| AbsurderSQL 🏆 | 3.2ms | 1.2ms | 400μs | 400μs |
| absurd-sql | 3.8ms | 2.1ms | 800μs | 700μs |
| Raw IndexedDB | 24.1ms | 1.4ms | 14.1ms | 6.3ms |
npm run serve
# Open http://localhost:8080/examples/benchmark.htmlAbsurderSQL is inspired by and builds upon the excellent work of absurd-sql by James Long, which pioneered SQLite-in-IndexedDB. Here's how they compare:
Both projects share core concepts:
- IndexedDB as persistent storage backend
- Block/page-based storage (not single-file)
- Full SQLite functionality in browser
- Significantly better performance than raw IndexedDB
| Feature | absurd-sql | AbsurderSQL |
|---|---|---|
| Engine | sql.js (Emscripten) | sqlite-wasm-rs (Rust C API) |
| Language | JavaScript | Rust/WASM/UniFFI |
| Platform | Browser only | Browser + Native + Mobile |
| Mobile Support | Not supported | iOS + Android via UniFFI |
| Storage | Variable SQLite pages (8KB suggested) | Fixed 4KB blocks |
| Worker | Optional (fallback mode works on main thread) | Optional (works on main thread) |
| SharedArrayBuffer | Optional (faster with SAB, fallback without) | Not used |
| CORS Headers | Optional (only if using SAB mode) | Not required |
| Feature | absurd-sql | AbsurderSQL |
|---|---|---|
| Export to File | Not supported | Full SQLite file export |
| Import from File | Not supported | Standard SQLite import |
| Database Backup | Locked in IndexedDB | Download as .db files |
| Data Migration | Manual only | Automated export/import |
| File Portability | No file access | Use with sqlite3, DB Browser |
| Multi-Device Sync | Not possible | Export -> share -> import |
| Disaster Recovery | No backups | Full backup/restore |
| Feature | absurd-sql | AbsurderSQL |
|---|---|---|
| Coordination | Throws errors | Coordinated with write queuing |
| Leadership | No concept | Automatic election with failover |
| Follower Writes | Not supported | Supported via queueWrite() |
absurd-sql:
- sql.js VFS interception for file operations
- Two modes: SAB mode (fast) or FileOpsFallback (compatible)
- SAB mode: Worker + SharedArrayBuffer for synchronous ops
- Fallback mode: Works anywhere, "one writer at a time"
AbsurderSQL:
- Custom Rust IndexedDB VFS implementation
- localStorage atomic coordination primitives
- Block-level checksums and versioning (MVCC-style)
- LRU cache (128 blocks default)
- Full multi-tab write coordination (no errors)
- Works everywhere (localStorage-based, no SAB)
Choose AbsurderSQL if you:
[CRITICAL] Need database export/import (DATA PORTABILITY)
- Export databases as standard SQLite files users can download
- Import databases from file uploads or shared files
- Backup/restore workflows for disaster recovery
- Migrate data between devices, browsers, or users
- Open exported files with sqlite3 CLI, DB Browser, or any SQLite tool
- Share databases as files (email, cloud storage, USB drives)
- Why this matters: absurd-sql has NO export/import - your data is permanently locked in IndexedDB
[✓] Need dual-mode persistence (Browser + Native)
- Build web apps with IndexedDB storage
- Query the same data from CLI/server using traditional
.dbfiles - Offline-first apps with optional server sync
- Debug production data locally using standard SQLite tools
- Why this matters: absurd-sql is browser-only - no CLI/server access to your data
[✓] Want zero deployment friction
- Deploy to GitHub Pages, Netlify, Vercel, or any CDN instantly
- No server configuration or CORS header setup required
- Works in iframes and embedded contexts
- Why this matters: absurd-sql's SAB mode requires special HTTP headers (COEP/COOP) for best performance, though fallback mode works without them
[✓] Want flexible architecture
- Can run on main thread OR in Web Worker (your choice)
- Simpler integration - no mandatory worker setup
- Easy to add to existing apps without refactoring
- Why this matters: absurd-sql's SAB mode requires a Web Worker, though fallback mode can run on main thread with performance trade-offs
[✓] Need multi-tab applications
- Multiple tabs can write data without coordination errors
- Automatic conflict resolution with leader election
- User can have multiple tabs open without issues (e.g., documentation in one tab, app in another)
- Why this matters: absurd-sql throws errors if multiple tabs try to write simultaneously
[✓] Value data integrity
- Built-in checksums detect corruption
- Crash consistency guarantees (committed data survives browser crashes)
- MVCC-style versioning prevents race conditions
- Why this matters: Protects against data loss from browser crashes, bugs, or unexpected shutdowns
[✓] Want better performance
- 16-50% faster than absurd-sql across all operations
- LRU caching optimizes hot data access
- Efficient 4KB block size balances memory and I/O
- Why this matters: Faster queries = better user experience, especially on mobile devices
[✓] Need production-ready tooling
- Comprehensive test suite (WASM + Native + E2E tests)
- Full TypeScript definitions
- Active development and maintenance
- Why this matters: Confidence in reliability, easier debugging, better IDE support
Choose absurd-sql if you:
[!] Already invested in sql.js
- Have existing sql.js code you want to keep
- Need to support very old browsers without WASM support (pre-2017)
- Trade-off: Miss out on Rust's memory safety and performance
[!] Prefer pure JavaScript stack
- Don't want to deal with Rust/WASM compilation (though wasm-pack makes this trivial)
- Want to read/modify source code in JavaScript
- Trade-off: Slower performance, more deployment complexity
[!] Don't need multi-tab
- Single-tab application only
- Users never have multiple tabs open
- Trade-off: Limited scalability if requirements change later
Bottom Line:
- AbsurderSQL = Modern, fast, works everywhere, multi-tab ready, export/import support
- absurd-sql = Proven, JavaScript-only, two modes (SAB or fallback), single-tab focus
Detailed technical comparison in BENCHMARK.md
Issue: When running native tests with cargo test --features fs_persist, you may encounter a transient failure in test_database_operations_across_threads:
test_database_operations_across_threads ... FAILED
thread 'test_database_operations_across_threads' panicked at tests/send_safety_test.rs:74:10:
Failed to insert: DatabaseError {
code: "SQLITE_ERROR",
message: "attempt to write a readonly database",
sql: Some("INSERT INTO test (value) VALUES ('test_value')")
}
Root Cause: Stale database files from previous test runs can persist in multiple storage directories. When tests attempt to reuse these files, SQLite may open them in read-only mode due to file permissions or lock state inconsistencies from interrupted test runs.
Storage Directories Created by Tests:
.absurdersql_fs- Primary test storage directorytest_storage- Additional test database storagedatasync_storage- Multi-tab sync test storageabsurdersql_storage- Native persistence test storage
Solution: Remove all stale storage directories before running tests:
# Clean up all stale test databases
rm -rf .absurdersql_fs test_storage datasync_storage absurdersql_storage
# Run tests fresh
cargo test --features fs_persistPrevention: The test suite creates unique run directories (e.g., run_<pid>) to isolate test runs, but interrupted tests (Ctrl+C, IDE stop) may not clean up properly. Consider adding this cleanup to your test scripts or CI/CD pipelines:
#!/bin/bash
# test-with-cleanup.sh
rm -rf .absurdersql_fs test_storage datasync_storage absurdersql_storage
cargo test --features fs_persist "$@"Note: This issue does not affect production usage - only test environments where multiple test runs accumulate stale databases. Production applications using fs_persist with proper database lifecycle management (open → use → close) are not affected.
- Export/Import Guide - DATABASE PORTABILITY - Complete export/import reference (HUGE advantage over absurd-sql)
- Tri-Mode Persistence Guide - Browser + Native + Mobile filesystem support
- Multi-Tab Coordination Guide - Complete guide for multi-tab coordination
- Transaction Support - Transaction handling and multi-tab transactions
- Encryption Guide - SQLCipher AES-256 encryption setup
- Benchmark Results - Performance comparisons and metrics
- Demo Guide - How to run the interactive demos
- Vite App Example - Production-ready multi-tab application
- Mobile Setup Guide - Complete iOS and Android build instructions
- Mobile Build Instructions - Detailed mobile build process
- Mobile Benchmarks - Performance comparisons on iOS/Android
- Coding Standards - Best practices, error handling, and testing requirements
- Unwrap Safety Analysis - Comprehensive analysis of remaining unwraps
- Alert Runbook - Complete debugging and remediation procedures for all alerts
- DevTools Extension - Browser extension features and architecture guide
- DevTools Installation - Step-by-step installation for Chrome/Firefox
- sqlite-wasm-rs: Production-ready SQLite WASM bindings with precompiled features
- rusqlite: Primary SQLite interface for native Rust builds, providing safe bindings to SQLite C library
- wasm-bindgen: Facilitates communication between Rust and JavaScript in WASM context
- js-sys: Provides bindings to JavaScript's built-in objects and functions
- web-sys: Offers bindings to Web APIs including IndexedDB
- serde: Handles serialization/deserialization for data exchange
- tokio: Provides async runtime support for handling asynchronous operations
Telemetry (enabled with --features telemetry):
- prometheus: Metrics collection and exposition
- opentelemetry: Distributed tracing framework
- opentelemetry_sdk: OpenTelemetry SDK implementation
- opentelemetry-prometheus: Prometheus exporter for OpenTelemetry
- opentelemetry-otlp (native only): OTLP exporter for sending traces to collectors
These dependencies are completely optional - when the telemetry feature is not enabled, zero telemetry code is compiled and none of these crates are included in your binary.
- Bootstrap 5.1.3: UI framework for responsive design and component styling
- Feather Icons: Icon library for user interface elements
- IndexedDB: Primary storage API for persistent data storage
- WebAssembly: Runtime environment for executing the compiled Rust code
- Fetch API: Used for loading WASM modules and handling HTTP requests
- wasm-pack: Build tool for generating WASM packages with JavaScript bindings
- Node.js 18+: Required for development tooling and testing infrastructure
- Rust 1.85.0+: Compiler targeting the 2024 edition for latest language features
The library is designed to work entirely in the browser environment without requiring any server-side components, making it suitable for offline-first applications and client-side data processing scenarios.
AbsurderSQL is licensed under the GNU Affero General Public License v3.0 (AGPL-3.0).
This is a strong copyleft license that requires:
- Source code disclosure: Any modifications must be shared under AGPL-3.0
- Network copyleft: If you run modified code as a web service, you must provide source to users accessing it over the network
- Patent protection: Includes patent grant provisions
- Freedom to share: Users can redistribute and modify the software
See LICENSE.md for the full license text.
Why AGPL-3.0? This license ensures that improvements to AbsurderSQL remain open source and benefit the entire community, even when used in cloud/SaaS environments.
