f0bb0787c9f550c19f7ccfbaf2cfb4db98218db3
3 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
Tom Boucher
|
1c93df04db |
fix(#2711): propagate the #2517 omit-on-inherit rule to all 15 unguarded workflows (#2713)
* test(#2711): derive the omit-rule guarded set from the corpus instead of a hand list The GUARDED array was a Goodhart metric: it reported green across 15 non-compliant workflows for no better reason than that nobody had added them to it. The guard now derives its set — every workflow emitting a model="{…}" dispatch site must state the omit-on-inherit/empty rule — and asserts the derivation is non-empty so a broken scan fails rather than passes. Rule detection stays a PROPERTY check, not a template match: plan-phase.md and execute-phase.md state it in different words and both are correct. RED expected on 15 workflows: audit-milestone, code-review, code-review-fix, debug, discuss-phase-assumptions, docs-update, map-codebase, new-milestone, new-project, quick, secure-phase, ui-phase, ui-review, validate-phase, verify-work. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso * fix(#2711): propagate the #2517 omit-on-inherit rule to all 15 unguarded workflows 15 of the 19 model=-dispatching workflows carried no omit-on-inherit/empty guidance — 43 unguarded dispatch sites. Each would emit model="" whenever the bound *_model resolved empty, which is the DEFAULT state on non-Claude runtimes: the installer writes resolve_model_ids:"omit" into ~/.gsd/defaults.json for every one of them (references/model-profiles.md:101), and resolveModelInternal returns "" for that case (src/model-resolver.cts:383-386) and "inherit" for opus-tier agents and the inherit profile (:395). Both 404 on runtimes without native tier aliases — the failure #2517 documented and fixed in one file. Each file now carries a `<!-- #2517 model-omit-on-inherit -->` blockquote naming its own bound placeholders and linking the canonical statement in references/model-profile-resolution.md, mirroring the `<!-- #2508 runtime-aware-dispatch -->` block already present in all 15. The rule text lives in the reference; the workflows carry a pointer plus the one-line instruction, so the next revision edits one file rather than fifteen. plan-phase.md and execute-phase.md are deliberately untouched — they already state the rule in their own wording, and the guard checks the property rather than a template string. No dispatch site is edited and no placeholder renamed: the #2684 binding guard reports the same 19 files / 60 placeholders / 0 findings before and after, which is the independence proof that this change is additive prose only. There is no Hyrum's-Law routing change to disclose. Placement is span-aware. An initial pass anchored to the #2508 marker, but in six files that marker sits INSIDE the Agent(prompt="…") string, so the new paragraph's literal model= landed in a dispatch call span and tripped the #2284 fail-closed Hermes projection guard (bin/install.js:3704), refusing the install. Blocks are now anchored before the opening Agent( of the span owning the first dispatch, and verified to fall inside no span. gen:golden exits 0 across all 19 runtimes. Fixes #2711 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso * fix(#2711): cite the issue number in the changeset body and tidy block placement Review findings from the two orthogonal passes: - The changeset body ended (#0). Repo convention across every prior fragment (e.g. #2617/#2693, #2608, #2605) is that the trailing (#NNN) is the ISSUE number, known at authoring time; only the frontmatter pr: field carries the 0 placeholder pending backfill. (#0) would have rendered a dead link in the published release notes. - new-milestone.md glued the inserted block directly under the preceding paragraph with no blank line, inconsistent with the other 14 insertions. - The derived-guard non-vacuity floor was >=17 against an actual derived count of 19, tolerating a silent two-file regression. Tightened to >=19. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso * fix(#2711): reword the omit block so it survives Hermes projection, and exempt quick.md by size The first block wording regressed two suites on the full matrix (4 failures on both linux-node22 and linux-node24). gen:golden passing was not sufficient evidence — it exercises the installer's own fail-closed guard, which is narrower than the dedicated tests. 1. tests/fix-2284-hermes-agent-delegate-task-projection.test.cjs asserts that the INSTALLED code-review-fix.md contains no `model=` anywhere outside a string literal — masked whole-file, not merely inside call spans. The block's backticked `model=` survived the mask. The assertion is right: on Hermes the projection strips the parameter because delegate_task has no per-call model at all, so instructing the orchestrator to "omit the model= parameter" is advice about a parameter that does not exist there. The block now says "the `model` parameter" and carries no bare `model=` token. 2. tests/prompt-injection-scan.security.test.cjs flagged quick.md at 50,164 normalized chars against a 50,000 prompt-stuffing threshold. quick.md sits just under the line on next, so any insertion trips it — the situation review.md is already documented for in SIZE_ONLY_WORKFLOWS ("sat at 49,971 chars — 29 below the threshold — so it was going to trip on whatever was added to it next"). quick.md joins it with the same justification. This is a size-finding exemption only: the file is still fully injection scanned, and every other security check still runs on it. Because the canonical block can no longer carry a literal `model=`, the guard's detector now accepts the `<!-- #2517 model-omit-on-inherit -->` marker as the canonical signal, falling back to the inline-prose property for the four files that predate it (plan-phase, execute-phase, scan, ship — all four match the legacy branch). That is strictly stronger than word-proximity matching, and it keeps the guard a property check rather than a template match. Verified: derived guard 19/19 with 0 missing; the #2684 binding guard unchanged at 19 files / 60 placeholders / 0 findings; no inserted block contains a bare model= token; the masked-projection assertion passes for code-review-fix.md; gen:golden exits 0 across all 19 runtimes; lint:ci exits 0. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso * chore(#2711): backfill changeset PR number Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tom Boucher
|
6fb832a93d |
fix(#2684): bind scan.md and ship.md model dispatch to fields their workflows resolve (#2710)
* test(#2684): failing-first guard for unbound model= dispatch placeholders Extends the #2517 omit-on-inherit bucket with a behavioral binding guard: every model="{X}" in a workflow must name a field that workflow actually binds — an init-payload key (queried for real), a shell assignment, or a declared parse field. scan.md ({resolved_model}) and ship.md ({balanced_model}) substitute names nothing emits, so the orchestrator invents the value (ADR-1411). Also pins the shipped reference that seeded the placeholder and instructs the #2517-forbidden model="inherit". RED expected on scan.md, ship.md, and references/model-profile-resolution.md. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso * fix(#2684): bind scan.md and ship.md model dispatch to fields their workflows resolve scan.md:85 passed model="{resolved_model}" and ship.md:486 passed model="{balanced_model}" — names neither workflow's init payload emits. init.map-codebase emits mapper_model; init.phase-op emits no model field at all. With no source for the substitution the orchestrator invents a value, so model_overrides/model_policy are silently inert at both sites — the invisible partial application ADR-1411 prohibits. - scan.md: declare the init fields in a Parse JSON line and substitute {mapper_model}, matching its sibling map-codebase.md. - ship.md: ref.agent is only known at runtime, so resolve it per hook via query resolve-model and dispatch with {HOOK_AGENT_MODEL}, following the same NAME=$(...) → model="{NAME}" convention every other shell-resolved dispatch in the corpus uses (code-review.md, secure-phase.md, ui-phase.md). - Both sites now carry the #2517 rule: omit model= entirely when the resolved value is "inherit" or empty. A bare rename would have traded a dangling placeholder for model="", which 404s on non-Claude runtimes — an agent type absent from the profile table resolves to the empty string, which is exactly ship.md's case. - references/model-profile-resolution.md was the seam: it shipped the copy-pasteable {resolved_model} snippet scan.md inherited, used the stale Task( spelling, and instructed passing model="inherit" outright. Rewritten to teach the real binding convention and the omit rule. Two defects surfaced while fixing this and fixed inline rather than deferred: 1. ref.agent originates in a capability manifest, which may be third-party. Resolving it at runtime made this the first place that value reaches a shell command, so ship.md now validates its shape before interpolating and skips the hook when it fails — matching code-review.md's existing defense-in-depth pattern. Covered by a test that runs the shipped regex against real agent names and injection payloads. 2. The reference doc's omit example first placed a literal model= inside an Agent(...) comment, which tripped the #2284 fail-closed Hermes projection guard (bin/install.js:3704) and refused the install outright. Moved out of the call span; gen:golden is green across all 19 runtimes. Guard tests extend the existing #2517 bucket: every model="{X}" in a workflow must name a field that workflow binds — an init-payload key queried for real, a shell assignment, or a declared parse field. Behavior change (Hyrum's Law): the scan mapper now runs on the catalog-resolved model rather than the session model. The omit path is unchanged. Disclosed in the changeset body. Fixes #2684 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso * fix(#2684): validate capability-supplied ref.agent in-context, not in the shell The first cut of the ref.agent guard was placed after the injection point it was meant to close. It instructed the orchestrator to substitute the untrusted manifest value into a shell assignment and test it there: HOOK_AGENT="<the ref.agent value>" if [[ "$HOOK_AGENT" =~ ^[A-Za-z0-9][A-Za-z0-9._-]*$ ]]; then … Substitution happens before bash parses anything, so a manifest supplying `x"; touch /tmp/pwned; echo "` yields three statements and runs the middle one unconditionally — the regex fires afterwards and protects nothing. ship.md now requires the check to run in-context, the same way the workflow already reads activeHooks ("do NOT pipe it through a shell parser"), and to skip the hook outright on a mismatch. Only a value that has already matched ^[A-Za-z0-9][A-Za-z0-9._-]*$ ever reaches a command line. The guard test asserts the ordering — the in-context requirement and the absence of any raw shell assignment — not just that the pattern rejects metacharacters, since a pattern alone was exactly what gave false assurance here. Also corrects the empty-string explanation in references/model-profile- resolution.md. model_profile:"inherit" resolves to the literal "inherit", not "" (model-resolver.cts:395), and an unknown agent takes the empty-string path via resolve_model_ids:"omit" rather than by absence alone — the doc claimed all three produced "". The workflow instructions were already correct (omit on "inherit" OR empty); only the rationale in the citable reference was wrong. Both found by the isolated adversarial review pass for #2684. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso * chore(#2684): backfill changeset PR number Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
Tom Boucher
|
b97b3dad21 |
fix(#2517): plan-phase omits model= when *_model is inherit/empty (port execute-phase fix) (#2634)
* test(#2517): guard plan/execute-phase omit model= when *_model inherit/empty * fix(#2517): plan-phase omits model= when *_model is inherit/empty (port execute-phase fix) * chore(#2517): backfill changeset pr to 2634 |