Commit Graph

5 Commits

Author SHA1 Message Date
Tom Boucher
1c93df04db fix(#2711): propagate the #2517 omit-on-inherit rule to all 15 unguarded workflows (#2713)
* test(#2711): derive the omit-rule guarded set from the corpus instead of a hand list

The GUARDED array was a Goodhart metric: it reported green across 15
non-compliant workflows for no better reason than that nobody had added them to
it. The guard now derives its set — every workflow emitting a model="{…}"
dispatch site must state the omit-on-inherit/empty rule — and asserts the
derivation is non-empty so a broken scan fails rather than passes.

Rule detection stays a PROPERTY check, not a template match: plan-phase.md and
execute-phase.md state it in different words and both are correct.

RED expected on 15 workflows: audit-milestone, code-review, code-review-fix,
debug, discuss-phase-assumptions, docs-update, map-codebase, new-milestone,
new-project, quick, secure-phase, ui-phase, ui-review, validate-phase,
verify-work.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

* fix(#2711): propagate the #2517 omit-on-inherit rule to all 15 unguarded workflows

15 of the 19 model=-dispatching workflows carried no omit-on-inherit/empty
guidance — 43 unguarded dispatch sites. Each would emit model="" whenever the
bound *_model resolved empty, which is the DEFAULT state on non-Claude runtimes:
the installer writes resolve_model_ids:"omit" into ~/.gsd/defaults.json for every
one of them (references/model-profiles.md:101), and resolveModelInternal returns
"" for that case (src/model-resolver.cts:383-386) and "inherit" for opus-tier
agents and the inherit profile (:395). Both 404 on runtimes without native tier
aliases — the failure #2517 documented and fixed in one file.

Each file now carries a `<!-- #2517 model-omit-on-inherit -->` blockquote naming
its own bound placeholders and linking the canonical statement in
references/model-profile-resolution.md, mirroring the `<!-- #2508
runtime-aware-dispatch -->` block already present in all 15. The rule text lives
in the reference; the workflows carry a pointer plus the one-line instruction, so
the next revision edits one file rather than fifteen.

plan-phase.md and execute-phase.md are deliberately untouched — they already
state the rule in their own wording, and the guard checks the property rather
than a template string.

No dispatch site is edited and no placeholder renamed: the #2684 binding guard
reports the same 19 files / 60 placeholders / 0 findings before and after, which
is the independence proof that this change is additive prose only. There is no
Hyrum's-Law routing change to disclose.

Placement is span-aware. An initial pass anchored to the #2508 marker, but in six
files that marker sits INSIDE the Agent(prompt="…") string, so the new paragraph's
literal model= landed in a dispatch call span and tripped the #2284 fail-closed
Hermes projection guard (bin/install.js:3704), refusing the install. Blocks are
now anchored before the opening Agent( of the span owning the first dispatch, and
verified to fall inside no span. gen:golden exits 0 across all 19 runtimes.

Fixes #2711

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

* fix(#2711): cite the issue number in the changeset body and tidy block placement

Review findings from the two orthogonal passes:

- The changeset body ended (#0). Repo convention across every prior fragment
  (e.g. #2617/#2693, #2608, #2605) is that the trailing (#NNN) is the ISSUE
  number, known at authoring time; only the frontmatter pr: field carries the 0
  placeholder pending backfill. (#0) would have rendered a dead link in the
  published release notes.
- new-milestone.md glued the inserted block directly under the preceding
  paragraph with no blank line, inconsistent with the other 14 insertions.
- The derived-guard non-vacuity floor was >=17 against an actual derived count
  of 19, tolerating a silent two-file regression. Tightened to >=19.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

* fix(#2711): reword the omit block so it survives Hermes projection, and exempt quick.md by size

The first block wording regressed two suites on the full matrix (4 failures on
both linux-node22 and linux-node24). gen:golden passing was not sufficient
evidence — it exercises the installer's own fail-closed guard, which is
narrower than the dedicated tests.

1. tests/fix-2284-hermes-agent-delegate-task-projection.test.cjs asserts that
   the INSTALLED code-review-fix.md contains no `model=` anywhere outside a
   string literal — masked whole-file, not merely inside call spans. The block's
   backticked `model=` survived the mask. The assertion is right: on Hermes the
   projection strips the parameter because delegate_task has no per-call model
   at all, so instructing the orchestrator to "omit the model= parameter" is
   advice about a parameter that does not exist there. The block now says "the
   `model` parameter" and carries no bare `model=` token.

2. tests/prompt-injection-scan.security.test.cjs flagged quick.md at 50,164
   normalized chars against a 50,000 prompt-stuffing threshold. quick.md sits
   just under the line on next, so any insertion trips it — the situation
   review.md is already documented for in SIZE_ONLY_WORKFLOWS ("sat at 49,971
   chars — 29 below the threshold — so it was going to trip on whatever was
   added to it next"). quick.md joins it with the same justification. This is a
   size-finding exemption only: the file is still fully injection scanned, and
   every other security check still runs on it.

Because the canonical block can no longer carry a literal `model=`, the guard's
detector now accepts the `<!-- #2517 model-omit-on-inherit -->` marker as the
canonical signal, falling back to the inline-prose property for the four files
that predate it (plan-phase, execute-phase, scan, ship — all four match the
legacy branch). That is strictly stronger than word-proximity matching, and it
keeps the guard a property check rather than a template match.

Verified: derived guard 19/19 with 0 missing; the #2684 binding guard unchanged
at 19 files / 60 placeholders / 0 findings; no inserted block contains a bare
model= token; the masked-projection assertion passes for code-review-fix.md;
gen:golden exits 0 across all 19 runtimes; lint:ci exits 0.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

* chore(#2711): backfill changeset PR number

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DPq9ovaovP2UvSVLjD4Lso

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-07-27 15:02:26 -04:00
Tom Boucher
09b535ac00 feat(#2481): add a negotiated effortSurface axis and wire invocation-time effort
ADR-1239 gains a ninth negotiated axis, effortSurface (argv | none), declaring how
a host accepts reasoning effort. ADR-443 is amended in the same change because its
recorded deferral is what the axis resolves: its Unblock condition offered paths
(a) and (b) and stated the choice was 'a maintainer call this file records but does
not make'. Path (a) is selected and satisfied here.

Before this, effort reached a runtime only through install-time channels
(EFFORT_RENDERING's frontmatter/api), so reviewer CLIs spawned as subprocesses
silently inherited whatever effort sat in the user's own global CLI config. The
review lane now resolves one universal effort through the ADR-443 cascade and
renders it per host through the negotiated descriptor.

Every per-host value is documentation-sourced, never inferred:
- claude   argv  -- verified via 'claude --help' (--effort <level>)
- opencode argv  -- verified via 'opencode run --help' (--variant)
- codex    argv  -- codex-rs/exec/src/cli.rs: model_reasoning_effort is NOT a CLI
                    flag (config.toml key only), so the global -c override is the
                    only argv route
- 15 hosts undocumented -- their docs state no reasoning setting; the sentinel
                    fails closed rather than inheriting a profile baseline

No config-file vocabulary member: the only host that ever had one (Gemini CLI's
thinkingConfig) was removed as a sunset runtime by 8f2ebbe9b (#1928, PR #1996),
and neither Antigravity CLI nor ZCode documents a reasoning setting.

Closes #2481

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-21 19:10:22 -04:00
Tom Boucher
f08b177215 feat(#1726): G1-G6 portability AST rules; fix all offenders; delete the ratchet (Phase 4) (#1731)
Phase 4 of epic #1702. Closes #1726.
2026-06-25 18:24:39 -04:00
Colin
cd5db1f8db test(suites): seed security/slow/integration suites via measured retags
Renames (git mv) with all references updated (ci-test-scope RULES,
windows-parity allowlist, test-file-count allowlist, docs in 6 locales):

- 5 scanner tests -> *.security.test.cjs — the 'Run security tests' CI step
  ran zero files since the suite taxonomy landed; it is now honest.
- graphify-auto-update -> *.slow.test.cjs (36s, slowest file in the suite;
  e2e gsd-tools spawns) — runs on full-matrix lanes and push to next.
- installer-migration-install-integration -> *.integration.test.cjs
  (13s; an integration test by its own name).

Coverage gate measured after retags: 88.55% lines (gate 70%).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 23:50:41 -04:00
Colin
a647053dcf ci(scope): narrow #494 invariant — changed tests join windows lane, not full matrix
full_matrix fired on 15/15 sampled PRs because any tests/** change forced it,
costing ~25 runner-minutes each. A changed test file now always joins the
scoped windows lane (covering the #482 OS-specific failure class per-file)
and still runs on ubuntu 22/24 via targeted_tests; the residual macOS /
windows-node-22 cross-product is covered on every push to next.

Also narrows WINDOWS_HINTS from 6 substrings (102/633 files, a ~10-minute
scoped lane) to windows/win32/shell/path — the dropped hints (workflow,
install, hook) are either platform-independent lint tests or already covered
by fullMatrix rules.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-09 23:50:41 -04:00