Nectenda/nectenda
1
0
Fork 0
Code Issues 7 Pull requests Projects Releases Packages Wiki Activity Actions

e2e: a fresh vault sometimes applies a note's only update and stays empty #3

New issue
Closed
opened 2026-09-14 15:06:50 +01:00 by cruelacid · 1 comment
cruelacid commented 2026-09-14 15:06:50 +01:00
Owner
Copy link

Seen 14 September 2026 while landing Phase A (issue #2). Pre-existing: the previous commit (e2dd7d2) fails the same way 1 of 2 full-suite runs on the same machine; with the change it failed 5 of 8. Load-sensitive (the machine was short of memory); a single file run alone passes most of the time.

Symptom

recovery.test.ts "reads the pre-reset note in a fresh vault" and, once, identity.test.ts "syncs with the owner": a fresh vault receives the note's file (created from the meta doc), subscribes to its content document, gets one catch-up frame, decrypts it without error, and the document is empty afterwards — the file stays 0 bytes for 60 s. Diagnostic signature (new lines added in this phase):

Document bound {path: Shared/before-the-reset.md, idbBytes: 0, alreadySynced: false}
Reconciliation decision {hasUnsentWork: false, lastSeq: 1, catchUpFrames: 1}
Synced content already on disk {path: Shared/before-the-reset.md, bytes: 0, hasSyncedOnce: true}

In a passing run the same doc shows Writing synced content to disk {diskBytes: 0, docBytes: 42}; the server holds one 90-byte update for it either way. Nothing is logged on the server side.

Hypotheses (unverified)

  • The reader integrates the update as pending (a struct dependency it does not have), so Y.Text stays empty with no error. Would mean the writer's single push is not self-contained from clock 0.
  • The meta document in failing runs sometimes reports lastSeq one above catchUpFrames (4 vs 3, 5 vs 4) — a frame missing from catch-up, or a snapshot counted differently. Not the note doc, but the same run.
  • Timing: Document bound → Reconciliation decision is ~1 ms apart in the failing runs.

How to reproduce

packages/e2e/run-detached.sh pnpm test:e2e:multi repeatedly under memory pressure; recovery.test.ts now writes both vaults' diagnostic logs to .e2e-tmp-recovery/diag-{A,B}.log and leaves recovery.db there, so a failing run can be inspected (doc_updates sizes per doc).

Why it matters

"Present, named correctly, and 0 bytes" is the silent failure content-sync.ts already documents guarding against; this is a path where the guard does not help. Real users would see an empty note until something else touches the document.

Seen 14 September 2026 while landing Phase A (issue #2). **Pre-existing**: the previous commit (`e2dd7d2`) fails the same way 1 of 2 full-suite runs on the same machine; with the change it failed 5 of 8. Load-sensitive (the machine was short of memory); a single file run alone passes most of the time. ## Symptom `recovery.test.ts` "reads the pre-reset note in a fresh vault" and, once, `identity.test.ts` "syncs with the owner": a fresh vault receives the note's file (created from the meta doc), subscribes to its content document, gets **one catch-up frame, decrypts it without error**, and the document is **empty** afterwards — the file stays 0 bytes for 60 s. Diagnostic signature (new lines added in this phase): ``` Document bound {path: Shared/before-the-reset.md, idbBytes: 0, alreadySynced: false} Reconciliation decision {hasUnsentWork: false, lastSeq: 1, catchUpFrames: 1} Synced content already on disk {path: Shared/before-the-reset.md, bytes: 0, hasSyncedOnce: true} ``` In a passing run the same doc shows `Writing synced content to disk {diskBytes: 0, docBytes: 42}`; the server holds one 90-byte update for it either way. Nothing is logged on the server side. ## Hypotheses (unverified) - The reader integrates the update as *pending* (a struct dependency it does not have), so `Y.Text` stays empty with no error. Would mean the writer's single push is not self-contained from clock 0. - The meta document in failing runs sometimes reports `lastSeq` one above `catchUpFrames` (4 vs 3, 5 vs 4) — a frame missing from catch-up, or a snapshot counted differently. Not the note doc, but the same run. - Timing: `Document bound` → `Reconciliation decision` is ~1 ms apart in the failing runs. ## How to reproduce `packages/e2e/run-detached.sh pnpm test:e2e:multi` repeatedly under memory pressure; `recovery.test.ts` now writes both vaults' diagnostic logs to `.e2e-tmp-recovery/diag-{A,B}.log` and leaves `recovery.db` there, so a failing run can be inspected (`doc_updates` sizes per doc). ## Why it matters "Present, named correctly, and 0 bytes" is the silent failure `content-sync.ts` already documents guarding against; this is a path where the guard does not help. Real users would see an empty note until something else touches the document.
cruelacid commented 2026-09-15 13:22:34 +01:00
Author
Owner
Copy link

Same defect as #15, which I filed yesterday without checking this one first — my mistake, and this is the older report. Fixed on main in 8bd2113; the full write-up is on #15.

What it actually was. Not decryption and not a pending struct dependency. FileSync creates an empty placeholder file the moment the folder listing names the note. The content then arrives from the server and fills the document correctly. Obsidian's modify event for that placeholder lands after it — one millisecond after — and onLocalModify read the empty placeholder as an edit, deleted every character that had just arrived, and pushed the deletion. The vault that wrote the note applied it and blanked its own copy too, so the writing was gone from both machines.

Logging every transaction's origin and resulting length named it in a single run:

B  TXN origin=remote  local=false  len=42     the content arrives intact
B  TXN origin=null    local=true   len=0      the placeholder's modify event deletes it
A  TXN origin=remote  local=false  len=0      the writer receives that deletion
A  Writing synced content to disk {diskBytes: 42, docBytes: 0}

Your two hypotheses were both wrong, and one of your observations was the key. The reader does not integrate the update as pending — it integrates it perfectly, then throws it away. The meta-document lastSeq discrepancy is unrelated. But you noted "the server holds one 90-byte update for it either way", and that is exactly right and exactly the point: the content reaches the server in failing runs too, which rules out the whole writer-side family of explanations. I spent two wrong theories before instrumenting, and that line would have saved me one of them.

Why it looked load-sensitive. It is a race between the placeholder's modify event and the arrival of the content. Memory pressure widens the window; it is not a memory bug.

Fixes. An empty file no longer empties a document until the vault has seen the two agree at least once. Blanking a file now copies it to .nectenda-backups/ first, so this class of loss can never again be silent. And seeding a document from disk completes before the document is marked confirmed, which was a second, real race.

Failed about one run in three before, passed twelve of twelve after. Both rules mutation-checked in empty-document-guard.test.ts; docs/sync-limitations.md records the case.

Closing as the same issue.

Same defect as #15, which I filed yesterday without checking this one first — my mistake, and this is the older report. Fixed on main in 8bd2113; the full write-up is on #15. **What it actually was.** Not decryption and not a pending struct dependency. FileSync creates an empty placeholder file the moment the folder listing names the note. The content then arrives from the server and fills the document correctly. Obsidian's modify event for that placeholder lands *after* it — one millisecond after — and `onLocalModify` read the empty placeholder as an edit, deleted every character that had just arrived, and pushed the deletion. The vault that wrote the note applied it and blanked its own copy too, so the writing was gone from both machines. Logging every transaction's origin and resulting length named it in a single run: ``` B TXN origin=remote local=false len=42 the content arrives intact B TXN origin=null local=true len=0 the placeholder's modify event deletes it A TXN origin=remote local=false len=0 the writer receives that deletion A Writing synced content to disk {diskBytes: 42, docBytes: 0} ``` **Your two hypotheses were both wrong, and one of your observations was the key.** The reader does not integrate the update as pending — it integrates it perfectly, then throws it away. The meta-document `lastSeq` discrepancy is unrelated. But you noted "the server holds one 90-byte update for it either way", and that is exactly right and exactly the point: the content reaches the server in failing runs too, which rules out the whole writer-side family of explanations. I spent two wrong theories before instrumenting, and that line would have saved me one of them. **Why it looked load-sensitive.** It is a race between the placeholder's modify event and the arrival of the content. Memory pressure widens the window; it is not a memory bug. **Fixes.** An empty file no longer empties a document until the vault has seen the two agree at least once. Blanking a file now copies it to `.nectenda-backups/` first, so this class of loss can never again be silent. And seeding a document from disk completes before the document is marked confirmed, which was a second, real race. Failed about one run in three before, passed twelve of twelve after. Both rules mutation-checked in `empty-document-guard.test.ts`; `docs/sync-limitations.md` records the case. Closing as the same issue.
Sign in to join this conversation.
No Branch/Tag specified
Branches Tags
main
fix-recovery-empty-overwrite
cd-phase-3
No results found.
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
Nectenda/nectenda#3
No description provided.