moteDB 0.5.1 Is Out: What 18 Months of Building an Embedded Database for Robots Taught Me

Last Tuesday our warehouse robot forgot what a red box looked like.

Not metaphorically. It had the force curve, the timestamp, the GPS pose — everything except the one thing that mattered: the visual embedding that let it recognize the same box three minutes later. The data was there. It just couldn't be queried the way the task needed.

That's the bug class moteDB exists to kill. And 0.5.1, released this week, is the version where I finally feel like we got the architecture right.

If you've been following since the v0.1 / v0.2 days, this release is the one where a lot of the awkward scaffolding comes down. Here's what actually changed — and what I'd tell past-me if I could.

The short version

motedb 0.5.1 is on crates.io right now. cargo add motedb or pin motedb = "0.5.1".

It's the same 100% Rust, serverless, multimodal embedded database it always was — vectors, time-series, scalar, and text in one file, one process, no daemon. What's different is how the write path and the index path behave under real robot loads.

The headline changes since v0.2:

• Streaming write pipeline — writes no longer stall when the memtable flushes
• Incremental DiskANN — vector indexes update online instead of requiring an offline rebuild
• Atomic cross-modal transactions — vector + scalar + time-series commit together, or roll back together
• Episode Memory API — a first-class primitive for "one task, bounded in time"
• Binary dropped from ~2MB to ~900KB

Let me unpack the ones that actually changed how I think.

1. The write path used to lie to you

In v0.2, the two-layer design (in-memory buffer → on-disk B-tree with a drain_lock) fixed the 3-hour benchmark hang. But it had a quiet flaw: when the active buffer flipped to immutable and the drain thread picked it up, new writes could still observe a half-drained state if they raced the drain. For sensor data at 200Hz, that meant occasional "the reading exists but the column index can't see it yet" gaps.

0.5.1 makes the buffer flip atomic at the WAL level. The trick is boring but effective: every write is stamped with an epoch counter, and the drain thread publishes its completed epoch before marking the buffer immutable. Reads filter by (buffer_epoch <= read_epoch), so a query never sees a buffer that's mid-migration.

// v0.5.1 — reads are epoch-bounded, never mid-drain
let visible = db.query()
    .scalar_range("force", 2.0.., 8.0)
    .with_epoch(read_epoch)   // new in 0.5
    .execute()?;

The robot that forgot the red box? That was a read crossing a drain boundary. It doesn't happen anymore.

2. DiskANN used to be a build-time tax

This was the thing I was most annoyed about for a year. DiskANN is the right call for edge hardware — memory-mapped, bounded query memory, no 1.5GB HNSW graph in RAM. But v0.2 only supported offline index builds. Add 50k new vectors from a day of operation and you rebuilt the whole graph. On a Pi, that's hours.

0.5.1 ships incremental DiskANN. New vectors get inserted into the existing on-disk graph with local neighbor pruning; the global graph quality slowly converges as the robot operates. You pay a small recall tax during the warm-up window, then it settles.

I was skeptical this would hold up under deletes (graph splits were the panic source in early testing). 0.5.1's delete path marks nodes tombstoned and re-prunes neighbors lazily during background compaction instead of mutating the live graph. That fixed the crash and kept query latency flat.

3. Cross-modal queries are now atomic

The whole point of moteDB is "find the camera frames where force exceeded 2N in the last 3 seconds, ranked by similarity to this query image." That's three indexes — vector, scalar, time-series — touched in one query.

In v0.2 they shared a WAL, so a crash rolled them back consistently. But a normal multi-write sequence — insert vector, insert scalar, insert text — had no transaction boundary. If your process died between step 2 and 3, you had an orphaned vector with no description.

0.5.1 adds db.transaction():

let tx = db.begin()?;
tx.insert(b"frame_0042", &embedding)?;
tx.insert_scalar(b"frame_0042", "force_curve", &curve)?;
tx.insert_text(b"frame_0042", "desc", "red box, slightly torn corner")?;
tx.commit()?;   // all three visible, or none

For a robot that logs 200 things per second, atomicity isn't a nice-to-have. It's the difference between "the memory is trustworthy" and "the memory is a liability."

4. Episode Memory: the API I wish I'd written first

This is the one I'm most excited about, and the most overdue.

A robot's natural unit of memory isn't "a row" — it's an episode: everything that happened between "start picking up the red box" and "placed it on the shelf." Sensors, frames, force curves, the plan, the outcome. One bounded, replayable unit.

0.5.1 adds Episode:

let ep = db.episode("pick_red_box_001")?;
ep.log_frame(&embedding, now)?;
ep.log_scalar("wrist_torque", torque, now)?;
// ... the whole task ...
ep.close()?;  // seals the episode, builds its local index

Later you replay it, diff two episodes, or query "show me every episode where wrist torque spiked." It's just structured sugar over the indexes we already had — but sugar that matches how embodied AI actually thinks.

What I'd tell past-me

Three things, in order:

1. The WAL is the database. Everything else is a cache you're allowed to lose. Spend your cleverness there first.
2. Don't rebuild indexes offline if you can pay for it incrementally. The "build once, ship the artifact" model felt clean. It was wrong for hardware that never stops learning.
3. Model the task, not the row. Episode Memory should have been v0.1. It took me 18 months to admit that "one embedding + metadata" is the wrong grain for a robot's memory.

The numbers, honestly

Not everything is a win, and I won't pretend it is:

• Binary: ~2MB → ~900KB (LTO + dead-code elimination)
• Mixed query (vector + range + text) p99 latency: down ~35% from v0.2 on a Pi 4B
• Incremental DiskANN warm-up: ~3% recall dip for the first ~10k inserts, then recovers
• Recall on a fully-warmed graph: still ~95%, same as offline build
• It will still lose to Postgres on analytical queries. Don't use it for that.

Try it

cargo add [email protected]

Docs and examples are on GitHub. The test suite now runs 1,200+ cases under concurrency in about 4 minutes with zero hangs.

If you're building anything that needs to remember in the physical world — robots, drones, edge agents — I'd genuinely like to hear what your memory layer looks like today. What's the one query you wish your stack could answer that it can't?