VibeDB

VibeDB is an embedded JSON database for Go applications. It gives you a small native API, durable local storage, immutable snapshots, exact JSON indexes, and an optional SQL layer—all inside your process.

Warning

VibeDB is under active development and has not been released. APIs and the on-disk format may change without a compatibility migration. Do not use it as the only copy of important data yet.

Why VibeDB?

• Store JSON documents by application-defined keys.
• Choose in-memory, buffered, or synchronous durability per database.
• Query with either the native typed Go API or a documented SQL subset.
• Connect through database/sql, pgx, lib/pq, or psql when SQL is useful.
• Use exact single-column or compound JSON-path indexes.
• Read immutable snapshots while writes publish new generations atomically.
• Put explicit limits on query, join, transaction, and intermediate memory.
• Run without background compaction or an external database service.

Install

go get github.com/thesyncim/vibedb

VibeDB currently follows the Go version declared in go.mod.

Native API quick start

package main

import (
	"fmt"
	"log"

	"github.com/thesyncim/vibedb"
)

func main() {
	db, err := vibedb.Open(
		"data/app.vdb",
		vibedb.WithDurability(vibedb.Durable),
	)
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	users := db.Collection("users")
	_, err = users.Put("user:1", []byte(
		`{"name":"Ada","active":true}`,
	))
	if err != nil {
		log.Fatal(err)
	}

	document, found, err := users.Get("user:1")
	if err != nil {
		log.Fatal(err)
	}
	if found {
		fmt.Println(string(document))
	}
}

Collection handles are lazy: asking for one does not touch the filesystem; the first mutation creates it. Get returns caller-owned bytes.

For repeated typed queries, import github.com/thesyncim/vibedb/query, compile a query.Query once, and reuse a collection session:

compiled := query.Select(query.Path("name")).Where(
	query.Cmp("active", query.Eq, true),
)

session := users.NewSession()
defer session.Release()

result, err := session.Run(compiled)
if err != nil {
	log.Fatal(err)
}
for row := 0; row < result.RowCount; row++ {
	name, _ := result.Columns[0].Cells[row].Text()
	fmt.Println(name)
}

See the store guide for updates, deletes, batches, indexes, snapshots, and lower-level storage configuration.

Pick the interface that fits

Interface	Best for	Package
Native document API	Embedded key/document access with the smallest surface	github.com/thesyncim/vibedb
Typed query builder	Compiled, reusable Go queries and tight execution control	github.com/thesyncim/vibedb/query
database/sql	Schemas, SQL, prepared statements, and transactions in Go	github.com/thesyncim/vibedb/sql/driver
PostgreSQL protocol	Existing pgx/lib/pq clients, direct psql, or non-Go clients	github.com/thesyncim/vibedb/pgwire

All four interfaces use the same JSON storage and query engine. SQL is a query language over JSON documents; it is not a separate relational storage engine.

SQL quick start

package main

import (
	"database/sql"
	"fmt"
	"log"

	_ "github.com/thesyncim/vibedb/sql/driver"
)

func main() {
	db, err := sql.Open("vibedb", "data/app-sql.vdb")
	if err != nil {
		log.Fatal(err)
	}
	defer db.Close()

	_, err = db.Exec(`
		CREATE TABLE users (
			id STRING PRIMARY KEY,
			name STRING NOT NULL,
			active BOOL NOT NULL
		)
	`)
	if err != nil {
		log.Fatal(err)
	}

	_, err = db.Exec(
		`INSERT INTO users VALUES (?)`,
		`{"id":"user:1","name":"Ada","active":true}`,
	)
	if err != nil {
		log.Fatal(err)
	}

	var name []byte
	err = db.QueryRow(
		`SELECT name FROM users WHERE id = ?`,
		"user:1",
	).Scan(&name)
	if err != nil {
		log.Fatal(err)
	}
	fmt.Println(string(name))
}

The supported surface includes schema-checked tables, exact indexes, DML with RETURNING, joins, derived tables, CTEs, set operations, a bounded recursive CTE subset, a documented window-function subset, views, predicate subqueries, and snapshot transactions. Unsupported shapes return explicit positioned errors instead of silently changing semantics.

VibeDB SQL is intentionally a subset, not PostgreSQL compatibility. Read the SQL surface before choosing an ORM or generating queries dynamically.

PostgreSQL clients

The pgwire package exposes the same SQL runtime over PostgreSQL protocol v3:

package main

import (
	"log"
	"net"

	"github.com/thesyncim/vibedb/pgwire"
	vibedriver "github.com/thesyncim/vibedb/sql/driver"
)

func main() {
	catalog, err := vibedriver.Open("data/app-sql.vdb")
	if err != nil {
		log.Fatal(err)
	}
	defer catalog.Close()

	server, err := pgwire.NewServer(catalog, pgwire.Options{
		Auth: pgwire.Trust(), // local development only
	})
	if err != nil {
		log.Fatal(err)
	}

	listener, err := net.Listen("tcp", "127.0.0.1:5433")
	if err != nil {
		log.Fatal(err)
	}
	log.Fatal(server.Serve(listener))
}

Then connect with a PostgreSQL client:

psql -X "host=127.0.0.1 port=5433 user=demo dbname=demo sslmode=disable"

The server supports the simple and extended protocols, prepared statements, SCRAM-SHA-256, transaction state, cancellation, text and binary results, and a small compatibility layer for basic psql introspection commands. It does not provide a queryable PostgreSQL catalog, general ORM/BI discovery, or TLS. Bind it to a trusted local interface or place it behind a TLS-terminating proxy.

See the pgwire contract for the exact protocol and client compatibility boundary.

Durability profiles

The zero-value/default profile is Durable.

Profile	A successful mutation means	Persistence boundary
vibedb.Memory	Visible in process memory	None; the path is ignored
vibedb.Buffered	Visible from bounded memory	Flush or Close
vibedb.Durable	Persisted to the recovery journal before it becomes visible	Every successful mutation

Durable and buffered databases use the path passed to Open as a database directory. Each successful write publishes a new database state while existing snapshots keep seeing the state they opened. Maintenance runs as bounded foreground work, so there is no background compactor to tune or wait for.

Read the durability contract before selecting buffered or advanced durability modes. In particular, a storage error can report an unknown commit outcome; callers must close, reopen, and reconcile instead of blindly retrying.

Important limitations

• The project is unreleased; APIs and storage format version 0 are unstable.
• SQL transactions may read multiple tables but write exactly one table.
• DDL is atomic per statement but is not transactional.
• Savepoints, two-table writes, arbitrary pg_catalog queries, and general ORM schema discovery are not supported.
• Pgwire does not implement TLS, replication, COPY, LISTEN/NOTIFY, or the full PostgreSQL type and function systems.
• Materialized views are not implemented; ordinary durable views are read-only.
• Memory and work are explicitly bounded. Exceeding a limit returns an error without publishing a partial query result or mutation.

The capability matrix is the executable authority for which combinations of operations, indexes, transactions, and durability are supported.

Distributed tier (experimental, server-only)

VibeDB is embedded-first: none of the four embedded interfaces above require a cluster, and the default embedded behavior is unchanged whether or not a placement configuration is supplied. The repository also carries an early, server-only distributed tier plus the routing types it shares with one opt-in embedded facade:

• a leader-only shard service (shardservice) that executes admitted SQL locally through a borrowed sql/driver session;
• a stateless routing gateway (gateway) that pins one immutable catalog generation and dispatches bounded, leader-only reads to the shards;
• the frozen placement scalar and tuple codec (distribution) used as cross-shard routing identity; and
• the cmd/vibedb-shard and cmd/vibedb-gateway binaries that run the server tier.

The shard service, gateway, and their binaries are server-only and not part of the embedded API. The one embedded touch point is opt-in and carries no network: the sql/driver local-cluster facade (OpenCluster / OpenClusterConnector) runs the same placement and write preflight over the shared distribution types against a single embedded store as a degenerate single-shard local cluster.

This tier is leader-only: it has no replication, failover, or online resharding. It is unreleased and unstable like the rest of VibeDB. The capability matrix covers the embedded surface only.

Read the design before relying on any of it:

• Distributed sharding
• Vitess-compatible routing
• Placement tuple format v1

Performance

VibeDB is designed for compiled queries, reusable execution storage, bounded foreground work, and allocation-free warm paths. Competitive benchmarks, corpus definitions, raw results, and reproduction commands are checked into the repository rather than summarized as context-free headline numbers here:

• Performance guide
• Competitive benchmark results
• Benchmark coverage matrix

Documentation

Topic	Document
Start using the storage API	Store guide
Exact supported combinations	Capability matrix
SQL syntax and limitations	SQL surface
Crash, recovery, and acknowledgement guarantees	Durability
Storage and snapshot design	Architecture
On-disk format	Format
Benchmarks and methodology	Performance
Development workflow	Contributing

Contributing

VibeDB favors measured behavior and explicit failure over undocumented fallbacks. Changes should include correctness tests, resource-bound tests where applicable, and benchmarks for hot-path work. See CONTRIBUTING.md.