Storage design, Rust implementation, indexing, distributed systems, APIs, benchmarks, and failure testing.
Project case study
Beta · Active developmentRust · HNSW · OpenRaft · Axum · CRC32C
FocalVector
A low-latency vector database that treats search quality, crash recovery, and operational boundaries as one system—from an exact correctness engine to persisted HNSW indexes and replicated shards.
ClientHTTP / local IPCCollection router
Query executorExact + HNSW + metadata
Write coordinatorCRC32C WAL + memtable
Exact and approximate search, typed filters, WAL recovery, immutable segments, backups, sharding, and Raft replication.
Designed first as FocalDesk's local semantic-search sidecar, with distributed operation available for further hardening.
The challenge
Make approximate search fast without making durability approximate.
Vector search systems have competing responsibilities. Queries need low latency and useful recall, writes need a clear acknowledgement boundary, filters must not silently discard valid neighbors, and a restart must recover the same committed state.
FocalVector starts with an exact-search correctness engine, then layers deterministic HNSW, full-precision reranking, indexed metadata, checksummed persistence, and replication around that reference path. Optimizations are measured against correctness instead of replacing it.
System design
An LSM-like engine with explicit read and write paths.
The write coordinator validates a complete mutation batch, appends a checksummed WAL frame, synchronizes according to the selected durability mode, and only then publishes the mutation to memory. Queries merge immutable graph segments with an exact delta of newer changes.
Keep exact search as the oracle
Exact top-k validates ANN recall, handles selective filters, and remains the best plan for small candidate sets.
Persist deterministic HNSW
Graph construction uses stable ordering and tie-breaking, while per-query ef_search exposes the recall and latency tradeoff.
Publish immutable state atomically
CRC32C-protected WAL and segment files fail closed on corruption, and manifests change only after replacement data is durable.
Engineering decisions
Own the invariants that determine search quality and recovery.
FocalVector owns its graph representation, storage formats, visibility rules, and merge semantics so upgrades and optimizations cannot quietly weaken the external contract.
WAL before memtable
- Constraint
- An acknowledged mutation must survive a process restart or power loss under synchronous durability.
- Decision
- Frame complete batches with CRC32C, append them to the WAL, call
sync_data, then apply them atomically in memory. - Tradeoff
- Durability adds I/O latency, but the acknowledgement boundary is explicit and a torn final frame can be safely discarded.
Quantized traversal, precise results
- Constraint
- Large full-precision graph arenas increase memory use, rebuild cost, and query latency.
- Decision
- Traverse scalar-int8 HNSW, oversample candidates, and rerank against authoritative full-precision vectors.
- Tradeoff
- Quantization can change graph traversal recall, so both representations remain benchmarkable and returned scores stay full precision.
Plan from metadata selectivity
- Constraint
- Post-filtering a small ANN result set can return too few neighbors even when qualifying points exist.
- Decision
- Use equality and numeric-range indexes to estimate candidates and choose exact, filtered HNSW, or expanded traversal.
- Tradeoff
- The planner is more involved than blind post-filtering, but filtered results preserve the top-k contract.
Separate sharding from replication
- Constraint
- Partitioning data and surviving node failure are different problems with different consistency costs.
- Decision
- Use stable FNV-1a sharding, then replicate each shard as its own OpenRaft group with majority-committed writes.
- Tradeoff
- Cross-shard writes are idempotent batches, not transactions, and distributed operation requires deployment-specific failure testing.
Engineering deep dive
Incremental persistence keeps fresh writes searchable.
Committed mutations remain in a small exact-search delta and merge with candidates from immutable HNSW segments. Writes therefore do not disable the graph, and routine flushes only rebuild changed live IDs.
Search fans out across a bounded number of immutable segments, suppresses stale versions and tombstones against the authoritative snapshot, and compacts after the segment limit. Replacement files are retired only after their manifest is durable.
Synchronize one validated WAL batch.
Sequence allocation and atomic batch framing make replay and read-your-writes behavior explicit.
Merge stable graphs with the exact delta.
Recent inserts, updates, and deletes stay visible before the next graph publication.
Write checksummed immutable segments off-lock.
A short publication lock swaps the manifest only if the collection sequence is still current.
Map validated files and replay the remaining WAL.
Search and reranking score vectors directly from read-only mappings instead of copying the full collection to heap arenas.
Repository evidence · August 2026
Benchmarks report the tradeoffs, not just the fastest number.
These repository measurements are regression evidence from a specific development host and synthetic corpora. Production embeddings still require their own recall, latency, concurrency, and cold-cache tests.
Coverage includes search, persistence, corruption handling, sharding, replication, failover, TLS, and local IPC.
Scalar-int8, 100k × 128 corpus at ef=384, with 95.1% recall@10 before durable API reranking.
13.6 MB scalar-int8 versus 51.2 MB f32 for the same 100k × 128 benchmark corpus.
Updating 1,000 points in a 100k × 128 corpus versus an 83.6 MB full segment in the documented run.
Where it stands
Useful as a local sidecar, candid about distributed limits.
The intended first deployment is a private FocalDesk user service over a permission-restricted Unix socket. Text and relational metadata remain authoritative elsewhere; vectors are durable but rebuildable search data.
Implemented
- Cosine, dot-product, and Euclidean search
- Exact and persisted HNSW query paths
- Metadata equality and numeric-range indexes
- Checksummed WAL, snapshots, backup, and restore
- Static sharding and deterministic global top-k
- OpenRaft replication, snapshots, membership changes, and failover
- HTTP, TLS/mTLS, metrics, and private local IPC
Operational hardening
- Long-running mixed read/write soak tests
- Disk-full and real power-loss fault injection
- Multi-million-vector cold-cache benchmarks
- Deployment-specific partition and restore exercises
- Byte-aware compaction policies and further memory reduction
- No claim of validated 100-million-vector operation yet
Go deeper
Read the storage and distributed-system documentation.
The repository documents the implemented system, benchmark methodology, durability contract, architecture, and remaining operational work.