NEC-312: Excalidraw hardness: a tab crash at ~677 shapes on a drawing switch (FLAKE-20) #318

Merged
nectenda-agent merged 15 commits from worktree-nec-312-excalidraw-hardness-a-tab-crash-at-677-s into main 2026-10-26 00:17:18 +00:00
Collaborator

Task: NEC-312 — https://projectron.nerchure.com/tasks/332

FLAKE-20, a renderer crash on a drawing switch at about 677 shapes, left no evidence: the hang watch went blind after a vault's first relaunch, and a crash's logs were read through the renderer that had died. This makes a crash report itself. It records Chromium's termination status and signal, with a host-side report (heartbeats, cgroup memory with oom_kill, processes, diag log from disk), re-arms the watch after every relaunch, samples every Obsidian process's memory once a second, and traces each vault's memory after every hardness round. A probe measures memory across drawing switches.

The seed was replayed three times with the reporting in place, once on the tree that crashed. The crash did not come back. One renderer reaches about 4 GB on a switch at the crash's size, and the container gets to within about a gigabyte of the VM's limit, which fits an OOM or a heap failure but was not observed. FLAKE-20 stays Unexplained, and now reports itself.

The replays, the probe and two fresh-context reviews found three drawing sync bugs, fixed here as ride-alongs, plus one in the container runner. The Excalidraw hardness run passed two hours on two seeds, newest against oldest, on this branch's sync path (one before NEC-340, one after); the canvas and Markdown runs had 5 minutes each to check the shared harness wiring.

Spec: docs/changes/NEC-312-excalidraw-tab-crash/spec.md

Also fixes NEC-337: A drawing a script wrote gets a spurious kept copy of every image on first open
Also fixes NEC-339: A vault's open drawing never draws a winning rival of the same version
Also fixes NEC-340: An edit made on a version a drawing's view went back to is lost, with no copy
Also fixes NEC-342: A container e2e run copies out an earlier run's dumps and report as its own

Changelog

An edit made to a shape just as an open drawing reloads an older version of it is no longer lost.

Task: NEC-312 — https://projectron.nerchure.com/tasks/332 FLAKE-20, a renderer crash on a drawing switch at about 677 shapes, left no evidence: the hang watch went blind after a vault's first relaunch, and a crash's logs were read through the renderer that had died. This makes a crash report itself. It records Chromium's termination status and signal, with a host-side report (heartbeats, cgroup memory with oom_kill, processes, diag log from disk), re-arms the watch after every relaunch, samples every Obsidian process's memory once a second, and traces each vault's memory after every hardness round. A probe measures memory across drawing switches. The seed was replayed three times with the reporting in place, once on the tree that crashed. The crash did not come back. One renderer reaches about 4 GB on a switch at the crash's size, and the container gets to within about a gigabyte of the VM's limit, which fits an OOM or a heap failure but was not observed. FLAKE-20 stays Unexplained, and now reports itself. The replays, the probe and two fresh-context reviews found three drawing sync bugs, fixed here as ride-alongs, plus one in the container runner. The Excalidraw hardness run passed two hours on two seeds, newest against oldest, on this branch's sync path (one before NEC-340, one after); the canvas and Markdown runs had 5 minutes each to check the shared harness wiring. Spec: `docs/changes/NEC-312-excalidraw-tab-crash/spec.md` Also fixes NEC-337: A drawing a script wrote gets a spurious kept copy of every image on first open Also fixes NEC-339: A vault's open drawing never draws a winning rival of the same version Also fixes NEC-340: An edit made on a version a drawing's view went back to is lost, with no copy Also fixes NEC-342: A container e2e run copies out an earlier run's dumps and report as its own ## Changelog An edit made to a shape just as an open drawing reloads an older version of it is no longer lost.
A renderer that dies shows up as `tab crashed` and nothing else. The
Excalidraw hardness run lost vault A's renderer twice at the same round
(FLAKE-20) and left no way to tell the kernel's OOM killer from V8 running
out of heap from a native crash. The hang watch that would have shown its
memory had gone blind at the vault's first relaunch, twenty relaunches
earlier, because `reloadObsidian` quits through the wrapped
`deleteSession` and nothing armed it again.

Now the watch is armed again after every relaunch. Chromium's own account
of each renderer's end, from the browser's DevTools endpoint, goes to
`<label>.crash`. The first command to find a renderer dead prints a crash
report like the hang report, with the container's cgroup memory and
`oom_kill`. A once-a-second sample of every Obsidian process's memory goes
to `memory-<worker>.jsonl`. `crash-watch.test.ts` crashes and kills real
renderers to keep it working.

Task: NEC-312
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Seed 1642605425's dump held nothing of vault A but the `tab crashed` it
failed with: every log was read through the renderer that had died, and
the first failing read stopped the dump before vault B. Each vault's part
of the Excalidraw dump now goes on past a dead renderer. Every hardness
dump copies in what the host holds: heartbeat, Chromium's log, crash
record, crash report, memory samples, and the diag logs from disk where
the renderer could not answer. The trace gets a memory line after every
round, and one more when a round fails.

Task: NEC-312
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Seed 1642605425's renderer died on a tab switch with about 677 shapes on
the board. `excalidraw-switch-memory` builds a drawing of that size and
switches one tab between it and a small one, reading the renderer's
memory after a full collection each time, with this plugin bound and then
disabled. It can also build every picture distinct, start with the plugin
disabled, and report what the view changed of each element as built.

At the crash's numbers, the heap after a full collection held flat: 89 to
96 MB over 40 switches bound, and 93 to 96 MB disabled. Without a forced
collection it had looked like +13 MB a switch.

Task: NEC-312
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
A drawing that an ExcalidrawAutomate script wrote with images, in a vault
with live sync on, got a "Kept by Nectenda: version from before live sync"
copy of every image the first time it opened. A drawing of 180 images
opened with 540 more elements and a notice that it had been changed in two
places at once.

Excalidraw sets an image's `status` itself as a view loads the picture, at
a version it raises on its own. The file had no `status`, so settle read
the fill's version as having lost writing and kept it. An image's load
state is no longer content (SAFE-A27). Its file, crop and scale still are.

Task: NEC-312
Also-fixes: NEC-337
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Two vaults made rival versions of one shape offline. B's won, A kept its
own edit as a copy, and B's was drawn into A's view. Then A's view went
back to A's own version: a reopened drawing finishing its load from a file
that still held it. That rival had been offered once already, so nothing
carried it in, and only a change to the document drew. A's view showed
its losing edit standing beside the copy of the same edit for as long as
it stayed open, and B's edit was in A's vault neither standing nor kept
(Excalidraw hardness seed 1642605425, round 170).

A capture now also draws when the view holds an element beside what
stands (a rival already offered, and so settled) or behind it. An element
being edited is still held back by reconcile (SAFE-A29).

Task: NEC-312
Also-fixes: NEC-339
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
The probe found no growth across switches, and the first replay a
lost-rival bug (NEC-339) instead of the crash. The second, with that
fixed, passed the hour; at the crash's round one renderer reached 4 GB
and the container 6.8 GB of a 7.75 GiB VM, without crashing.

Task: NEC-312
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
A late load put a shape back on an older version, or on this vault's
losing rival. Drawn over only at the next capture, 100 ms on, an edit
made in between was made on what went back. Judged against what was
last drawn, it read as older and was drawn over, or, raised past what
stands by a drag, replaced the other vault's edit. Either way it was
lost with no copy. Found by NEC-312's review; it predates NEC-339,
which only narrowed it.

A view seen going back, under or beside what the binding drew or
carried in, is now captured in a microtask after Excalidraw's change
event, before the next input. Once per going back, so a shape held by
reconcile is not drawn at every frame. docs/sync-limitations.md records
the window left: an input between Excalidraw replacing the scene and
announcing it.

Task: NEC-312
Also-fixes: NEC-340
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Between renderers nothing is armed, and a report made then says so and
names the previous pid, with the crashed vault's own files in it. A
crash met by the relaunch itself is reported, the sampler keeps going
through it, and DevTools fetches have a deadline. The crash-watch test
now checks the debugger is armed again, samples up to the crash, and
the cgroup figures wherever their files exist. A backup that cannot be
copied no longer stops a hardness dump.

Task: NEC-312
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Task: NEC-312
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
A shape held back while it is resized cannot be drawn over after a late
load, at once or later, so the resize goes on from the version it went
back to. Judged against what was last drawn, it was lost the same two
ways NEC-340 found: drawn over as older, or carried in as made on a
version it never held, replacing the other vault's edit. The binding
now keeps the version each shape was seen going back to, and judges an
edit against it until a draw or a carry-in names another.

The once-only draw is keyed by every shape that went back, so one held
there no longer hides another going back after it. The e2e test waits
past a pending capture first, so an unfixed build cannot pass by luck;
its record of Excalidraw's order (scene replaced in updateScene, change
announced later) now backs the window docs/sync-limitations.md leaves.
Found by NEC-312's second review.

Task: NEC-312
Also-fixes: NEC-340
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Between renderers a crash report was keyed by the previous pid, so a
relaunch after a reported crash said nothing of its own; it is keyed by
the relaunch now. The sampler keeps going until the new renderer is
watched, not only until Obsidian is back. Found by the second review.

Task: NEC-312
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Task: NEC-312
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Task: NEC-312

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Clear a container run's .results first, so an earlier run's dumps never pass as its own
Some checks failed
CI / promote (pull_request) Has been cancelled
CI / build (pull_request) Has been cancelled
CI / e2e (pull_request) Has been cancelled
Release note / release-note (pull_request) Successful in 24s
bfbfa09ea0
The container's work volume outlives the run, and rsync --delete skips
the excluded .results, so every run copied out what earlier runs had
left there, freshly dated. NEC-312's passing hour came out holding a
round-170 failure dump from five hours before, and after a hardness run
e2e-ledger.mjs could take the last multi report as the newest. CI's
action already clears it. CLAUDE.md's line that .results is kept relied
on the accident, so it now says to copy a dump out before the next run.

Task: NEC-312
Also-fixes: NEC-342

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Add NEC-312's release notes, one line per change
All checks were successful
Release note / release-note (pull_request) Successful in 19s
CI / build (pull_request) Successful in 5m35s
CI / e2e (pull_request) Successful in 14m19s
CI / promote (pull_request) Has been skipped
CI / build (push) Successful in 6m3s
CI / e2e (push) Successful in 14m18s
CI / promote (push) Successful in 33s
05887b3acd
Task: NEC-312

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_016kxqF3V5NbscqR19L5uL6f
Sign in to join this conversation.
No reviewers
No milestone
No project
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
Nectenda/nectenda!318
No description provided.