enh(#2875): materialization primitives — durable user-artifact staging and descriptor-authoritative agents (#3600)
* fix(#2875): stage user artifacts durably across install wipes (#1874-F19) preserveUserArtifacts held user files only in an in-memory Map across the wipe, so any process death between preserve and restore lost them outright. Seven call sites, not the four the issue records. Three of them never called the helper at all - they open-coded the same read/wipe/write - so searching for callers under-counted by construction; the extra sites were found by sweeping for the pattern instead. The worst is the mainline install path, where the crash window spans the entire gsd-core tree copy rather than a single rmSync. Adds src/user-artifact-staging.cts: durable on-disk staging with a record written after the copies land as the commit point, plus recovery of orphaned batches on the next run - without recovery the staged bytes survive but the user's file is still gone, which would pass its own test while delivering nothing. Routes copyPreservingSymlink through installFs() so staging cannot bypass the install fs seam, and reunites its symlink-safety docblock with the function it documents. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#2875): amend ADR-3574 with four claims disproved by implementation Implementing Phase 6 disproved four statements the ADR rests on. The central decision - no single materializer - is unaffected and stands. Corrected: decision 3 was already satisfied, so nothing was extracted; the agents-bypass runtime set omitted claude, kilo and opencode, and closing it needed three new pieces of descriptor contract rather than proceeding on its own terms; three of the four blockers the layout comment names were already stale; and F19 is seven call sites, not four. Records the generalizable lesson: the defect is the pattern of holding user data in memory across a wipe, not the helper, so searching for callers of the helper under-counts by construction. Also resolves the ADR's open question on USER_OWNED_ARTIFACTS membership, and notes that copyPreservingSymlink needed routing through the install fs seam before it could be reused. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#2875): close dangling-symlink blind spot and harden staging recovery An adversarial review found the F19 staging work shipped red and unsafe. Root cause, shared by two arbitrary-write findings: hasExistingSymlinkBetween missed dangling symlinks in both its root check and its per-segment walk, because it probed with existsSync, which is false for a link whose target does not exist. Fixing only the new module would have reused a guard that was itself blind. This guard protects the whole install tree. Recovery no longer throws: it degrades per entry and per file, so one bad batch cannot block the others. Previously an unrecoverable entry propagated out of the first statement of install and uninstall, before the cleanup that would have removed it - wedging the installer permanently. Partial fs adapters now throw on any omitted method instead of silently reaching the real filesystem, closing the trap that let a test poison list pass while real IO happened. Staged names must be flat, recovery refuses a dangling destination symlink, and a batch whose recovery genuinely failed is no longer swept - it was discarding the only durable copy of the file it had just failed to restore. Replaces three tests that could not fail, including the one labelled negative proof. Known limitation, documented not closed: concurrent installs sharing a staging key can still lose a batch. A real fix needs a cross-process lock. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * enh(#2875): make the descriptor authoritative for the agents kind Deletes the inline agent-staging loop in bin/install.js and the _DESCRIPTOR_AGENTS_RUNTIMES set, so every runtime materializes agents from its capability descriptor instead of an inline hostBehaviors dispatch. Closing it needed three pieces of contract the descriptor pipeline never had, all reducible to one missing input - per-agent resolution context: a frontmatter-extensions step for claude's effort and disallowedTools, per-agent model-override resolution for kilo and opencode, and a named branding converter for hermes, whose rewrite data was already declared. Seven runtimes were on the loop, not the six the design recorded - kimi-code was found by a golden fixture, not by analysis. claude-local and kimi-code both silently lost their agents mid-change; the fixtures caught both and the cause was fixed rather than the fixtures regenerated. A parity harness gates the migration: both pipelines over identical inputs, byte-identical output including filenames, per runtime. It is demonstrated red before being trusted. Surface and install paths converge for all seven, which also fixes surface previously writing no agents for these runtimes. Codex's config.toml strip stays put - it mutates host config, which no descriptor kind models. Also routes install-model-override-resolver and install-effort-resolver through the install fs seam. Both leaked real filesystem IO from the install call tree; the stricter adapter is what exposed them. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#2875): record the agents-descriptor migration and correct the ADR count The _DESCRIPTOR_AGENTS_RUNTIMES allow-list no longer exists, so the host integration guide told readers to join a set that is gone. Replaces that with what is now true - declare an agents entry and it installs, on the surface path as well as install - and points anyone needing a per-agent transform at the three extension points rather than at a new inline branch. Corrects the ADR amendment: seven runtimes were on the inline loop, not six. kimi-code was found by a golden fixture going red, not by reading. That is the third short count this phase, all from enumerating by symbol or set membership when the thing that matters is a behavior. Adds the Changed changeset for the surface-path convergence. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#2875): amend ADR-2866 - claude global always wrote agents on disk The claude row's global=[skills] described what capability.json declared, not what the installer wrote. bin/install.js's inline agent-staging loop was never scope-gated and never consulted the descriptor, so a claude --global install has always written agents/gsd-*.md. Phase 6 closes the gap by deleting that loop and declaring agents on claude's descriptor at global scope. On-disk bytes are unchanged - the golden fixtures did not move, which is the evidence that the descriptor, not the installer, was incomplete. #2218 is unaffected: agents are not trigger-bearing, so the wider row does not introduce a new shadowing case. Records the warning that an incomplete descriptor is invisible while a second code path silently does its work, and only surfaces when the two are forced into agreement. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#2875): close review findings across staging, agents and the parity harness Two independent reviews of this branch found defects the local gates missed. Security: a dangling symlink at a migration destination allowed writing outside configDir - the same class this change claimed to close, missed at the terminal write of the flow being added. The staging-root resolver threw as the first statement of install and uninstall, so a hostile symlink bricked both, and symlinked-configDir users lost uninstall as well as install; it now degrades instead of aborting. Recovery gained a source-side symlink check and now refuses a relative destDir, which resolved against cwd. Converter dispatch gained a runtime allowlist - lint-time validation stopped mattering once this branch promoted that dispatch from the surface path to real installs. Correctness: claude --local --minimal exited 1 because the minimal profile legitimately yields zero agents and the new path treated that as a failure. cline --local silently lost its agents - its descriptor declared none while the deleted loop wrote them unconditionally. The agents prune was widened to any gsd-* entry and destroyed user files it never owned. The parity harness, on which the migration's safety argument rested, drove a synthetic registry and never byte-compared the shipped descriptors; two of its trap rows could not fail. It now drives the real registry across 13 runtime-scope rows including kimi-code and cline-local, and its red-proof is demonstrated by corrupting a live capability.json. Three goldens that had encoded the cline regression as expected behavior were corrected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#2875): close findings from both mandated review engines /security-review found the staging source-side walk honouring GSD_ALLOW_SYMLINKED_DEST, an opt-in documented as relaxing only the write destination. A symlinked files/ component dereferenced because copyPreservingSymlink lstats the leaf only, so an intermediate link is followed. The source walk no longer honours the opt-in; the destination check still does. /code-review spec axis found this branch had reintroduced its own bug: migrateLegacyDevPreferencesToSkill's new symlink refusal threw unguarded after the legacy dir was wiped and before the staged batch was restored, so a planted symlink bricked uninstall permanently and orphaned the batch. Refusal kept, abort removed. kimi-code local silently lost its agents, the same class as the cline bug, and the parity harness recorded that exclusion as intentional - the third test in this branch to pin a regression as correct. --minimal now creates an empty agents/ dir that never existed. Behaviour restored rather than softening the changeset, so its byte-identical claim stays true. Standards axis: try/finally removed from twelve test bodies, fast-check properties added for parseOwnerPid, boundary coverage at the grace window and the ancestor-probe depth, a parity assertion for the staging-root helper duplicated across two files, and the 8-deep config walk deduplicated. Records 60-review.json with every finding and disposition from five passes, including the smells left unfixed and why. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#2875): prune stale agents unconditionally in minimal mode The previous round stopped an empty agents/ directory being created when the resolved profile yields no agents. That was implemented by skipping the agents kind entirely, which also skipped its stale-agent prune - so a full to minimal downgrade left stale gsd-* agents behind. The deleted inline loop pruned unconditionally and only skipped writing. Those are three separate conditions, not one: prune always, write only when there is something to write, create the directory only when writing. Both call sites now run _removeGsdEntries before the empty-staged early exit. The symlink-escape guard moved with it, since the prune also touches dest. Codex .toml agents and the config.toml stanzas are cleaned again, and user-owned agents are still preserved. The agents/ directory is left in place after a prune empties it, matching every sibling kind - none of them remove the destination directory itself. Golden fixtures confirmed byte-identical: the prune is a no-op on a fresh install, so fixture generation is unaffected. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#2875): document interrupted-install recovery for user-owned files The durable-staging fix is invisible to the user it protects. Someone whose install died mid-flight has no way to know USER-PROFILE.md was staged before the delete, that the next run restores it, or that recovery happens at the start of that run rather than in the background. Written as the task the user has - finish the interrupted command - rather than as a description of the mechanism, and states what it will not do: overwrite a file already present, or touch staging belonging to another install still running. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(#2875): backfill changeset pr number Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * test(#2875): assert the J8 model override without building a regex CodeQL flagged incomplete string escaping: the assertion interpolated the override value into a RegExp while escaping only forward slashes, which is meaningless in a constructor, leaving real metacharacters unescaped. The failure direction was the dangerous one - a metacharacter would have made the match more permissive, so the row would pass when it should fail. That matters here because J8 exists precisely because an earlier revision was a tautology; the rewrite reintroduced a different way for the same assertion to stop discriminating. Replaced with a line-wise exact match, so no regex is constructed at all. Swept the other test files this branch adds; no sibling instances. lint:ci passed on the original - lint-no-adhoc-regex-escape matches a full metachar-escape copy, so a single slash replace slipped under it. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -44,6 +44,38 @@ This correction is also why `windsurf`'s row above reads correctly without amend
|
||||
one), which is exactly the case Phase 2's test suite locks in as "windsurf must not report a shadow
|
||||
it does not have."
|
||||
|
||||
## Amendment (2026-08-17, #2875): `global=[skills]` described the descriptor, not the disk
|
||||
|
||||
Phase 6 found that the claude row is wrong even as the *placement* fact the amendment above
|
||||
preserved. **A `claude --global` install has always written `agents/gsd-*.md`.** It did so through
|
||||
the inline agent-staging loop in `bin/install.js`, which was never scope-gated and never consulted
|
||||
the descriptor — claude was not a member of the `_DESCRIPTOR_AGENTS_RUNTIMES` allow-list, so it fell
|
||||
through to that loop on every install, at both scopes.
|
||||
|
||||
So `global=[skills]` accurately described what claude's `capability.json` *declared*, and did not
|
||||
describe what the installer actually wrote. Those two things had silently diverged, and this ADR —
|
||||
along with the layout module's own golden tests — recorded the declaration as though it were the
|
||||
outcome.
|
||||
|
||||
Phase 6 closes the gap from the other side: the inline loop is deleted, the descriptor is
|
||||
authoritative for `agents` on every runtime, and claude's descriptor now declares `agents` at global
|
||||
scope. The row's true shape is therefore **`global=[skills, agents]`, `local=[commands, agents]`**.
|
||||
On-disk bytes are unchanged — the golden install-tree fixtures did not move, which is the evidence
|
||||
that the descriptor, not the installer, was the thing that was incomplete.
|
||||
|
||||
**#2218 is again unaffected, for the reason the previous amendment established.** `agents` is not
|
||||
trigger-bearing, so widening claude's global row to include it changes nothing about the
|
||||
`commands`-versus-`skills` collision that #2218 *is*. Anyone re-reading this ADR after Phase 6
|
||||
should not infer a new shadowing case from the wider row.
|
||||
|
||||
**The generalizable warning.** A descriptor that is merely *incomplete* is invisible: nothing fails,
|
||||
because a separate code path is quietly doing the work the descriptor should have declared. It
|
||||
surfaces only when something forces the two into agreement — here, deleting the code path. Reviews
|
||||
that check "does the descriptor say the right thing?" cannot catch this; only "does anything install
|
||||
artifacts this descriptor does not declare?" can. Three separate enumerations in Phase 6 were short
|
||||
for a related reason, all recorded in [ADR-3574](3574-install-materialization-primitives.md)'s
|
||||
amendment.
|
||||
|
||||
No decision in this ADR changes as a result — Phase 2's `resolveTriggerSurface` signature, the
|
||||
`triggerPrecedence` axis, and the phase map above were all designed against the corrected model.
|
||||
See `.gsd/phase/feat-2871-trigger-resolution/40-design.md` for the full analysis.
|
||||
|
||||
@@ -165,3 +165,103 @@ the same name. The decision above rests on read code, not on those numbers.
|
||||
- Placement seam: [ADR-3660](3660-runtime-artifact-layout-module.md) · content seam: [ADR-1508](1508-runtime-artifact-conversion-module.md) · policy/adapter split: [ADR-58](58-runtime-install-policy-module.md)
|
||||
- The epic's own frame: [ADR-2866](2866-install-surface-resolution.md), which mandated that this module owe its own ADR
|
||||
- User-directory preservation this ADR protects: #2973, #3664
|
||||
|
||||
## Amendment (2026-08-17, #2875): four factual claims corrected by implementation
|
||||
|
||||
Implementing this ADR as Phase 6 disproved four of the statements it rests on. **The central
|
||||
decision — §1, no single materializer — is unaffected and stands**; the divergence table that
|
||||
justified it was measured correctly. What follows corrects the surrounding claims, because a reader
|
||||
who acts on them will be misled.
|
||||
|
||||
This is the same failure mode the ADR itself warns about in "A note on the evidence": conclusions
|
||||
reached by reading code without executing it. Three of the four corrections below are cases where
|
||||
inspection produced a confident, wrong answer.
|
||||
|
||||
### 1. §Decision 3 is void — the retired-kind prune already had a single owner
|
||||
|
||||
The ADR says the prune "is extracted" and is "already called by both `installRuntimeArtifacts` and
|
||||
`applySurface`". Measured: `pruneRetiredRuntimeArtifacts` already lives alone in
|
||||
`src/retired-artifact-cleanup.cts`, already exports a single function, already routes every fs call
|
||||
through `installFs()`, and has **three** callers — `installRuntimeArtifacts`,
|
||||
`uninstallRuntimeArtifacts` and `applySurface`.
|
||||
|
||||
There was nothing to extract. **No refactor was invented to satisfy this decision.** A future reader
|
||||
should treat §3 as already-satisfied, not as outstanding work.
|
||||
|
||||
### 2. The `agents`-bypass runtime set was wrong, and §Decision 4 was the *hardest* part, not the easiest
|
||||
|
||||
The ADR states the inline dispatch "survives only for codex, cline, hermes and generic runtimes",
|
||||
and calls closing it "the part of Phase 6 whose evidence survived scrutiny intact".
|
||||
|
||||
Both are wrong. `_DESCRIPTOR_AGENTS_RUNTIMES` (`bin/install.js`) held ten runtimes; every other
|
||||
runtime reached the inline loop — **seven** of them: claude (the flagship), cline, codex, hermes,
|
||||
kilo, opencode and kimi-code. (`pi` is excluded separately by its `pluginOnlyInstall` branch.)
|
||||
|
||||
> **Even this correction undercounted.** It originally said six. `kimi-code` was found only when a
|
||||
> golden install-tree fixture went red mid-implementation — not by any amount of reading. That is
|
||||
> the third time this phase's enumeration was short (four call sites → seven; six runtimes →
|
||||
> seven), and every miss shares one cause: counting by *symbol* or *set membership* when the thing
|
||||
> that matters is a *behavior*. Fixtures and executed tests found what inspection did not.
|
||||
|
||||
The set and the loop are both **gone** as of this phase; the descriptor is authoritative for
|
||||
`agents` on every runtime, so there is no longer an allow-list to join.
|
||||
|
||||
Worse, closing the bypass could not be done "on its own terms". It required **three new pieces of
|
||||
descriptor contract**, because the descriptor pipeline had no per-agent resolution context:
|
||||
|
||||
| gap | consumer |
|
||||
|---|---|
|
||||
| a frontmatter-extensions step (`effort`, `disallowedTools`) | claude |
|
||||
| per-agent model-override resolution threaded to the converter | kilo, opencode |
|
||||
| a named branding converter (the *data* was already declared; the converter was not) | hermes |
|
||||
|
||||
Every one of those failed **silently** if migrated without the contract work — wrong bytes, nothing
|
||||
thrown. §Decision 4's framing as independent and low-risk should not be relied on.
|
||||
|
||||
### 3. Three of the four blockers in `runtime-artifact-layout.cts` were already stale
|
||||
|
||||
The ADR treats that comment's blocker list as current. Measured, only one was:
|
||||
|
||||
| blocker | status |
|
||||
|---|---|
|
||||
| Copilot's `.agent.md` filename rename | stale — #2099 dropped the ternary; the suffix comes from `hostBehaviors.agentFileExtension` |
|
||||
| cross-cutting path-prefix rewrite + attribution | stale — `stageAgentsForRuntimeWithConverter` already does both when `agentCtx` is present |
|
||||
| stale-file cleanup | stale — `_removeGsdEntries` prunes more broadly than the loop's extension-gated check |
|
||||
| config-reading steps | **real** — and it was the entire remaining gap (see §2 above) |
|
||||
|
||||
### 4. F19 is seven call sites, not four — and the helper was the wrong thing to search for
|
||||
|
||||
The ADR names four call sites, found by locating callers of `preserveUserArtifacts`. There are
|
||||
**seven**. Three of them never call the helper at all; they open-code the same
|
||||
`readFileSync` → wipe → `writeFileSync`.
|
||||
|
||||
**The generalizable lesson: the defect is the pattern "user data held only in memory across a
|
||||
wipe", not the helper.** Searching for callers of the helper under-counts by construction. The three
|
||||
extra sites were found by sweeping for the pattern — a read shortly before a wipe and a write
|
||||
shortly after.
|
||||
|
||||
The ADR also understates the severity. The worst site is the mainline install path, where the
|
||||
window spans the **entire `gsd-core` tree rebuild** inside `copyWithPathReplacement`, not a single
|
||||
`rmSync`. Any interruption of a normal install destroys the file.
|
||||
|
||||
### 5. Resolved: `USER_OWNED_ARTIFACTS`
|
||||
|
||||
"What this ADR does not decide" records its membership as unconfirmable. It is confirmed:
|
||||
`src/install-engine.cts` defines it as exactly **`['USER-PROFILE.md']`**, with a docblock recording
|
||||
the invariant that a file is either manifest-tracked distribution or a preserved user artifact,
|
||||
never both (#2771). `dev-preferences.md` is preserved at other sites by explicit name. That open
|
||||
question is closed.
|
||||
|
||||
### 6. `copyPreservingSymlink` could not be reused verbatim
|
||||
|
||||
§Decision 2 directs reusing it, and that is still the right primitive for the reason given (it never
|
||||
dereferences a symlink). But it used raw `fs` for all five of its calls, while its new caller sits on
|
||||
the install path Phase 5 routed through an injectable seam. Verbatim reuse would have punched a hole
|
||||
through that seam — the partial-adapter trap `install-fs-adapter.cts` documents. It was routed
|
||||
through `installFs()` as part of the extraction; its existing migration caller is unaffected, since
|
||||
the ambient default resolves to real fs.
|
||||
|
||||
**A caution for anyone extending this module:** that same fall-through is a live hazard. A missing
|
||||
method on an injected adapter does not fail loudly — it silently reaches the real filesystem. Adding
|
||||
a new `installFs()` call to a routed path without extending every adapter is a real-IO bug that
|
||||
passes typecheck.
|
||||
|
||||
Reference in New Issue
Block a user