9faacc0c153a88f939ef07ced74d56dc468f0708
5 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
sim
|
6128f73003 |
test(#3090): normalize the whole reason line, not just the category token
The allow-test-rule gate keys on identity, and the identity it records is everything after the colon on the annotation line — not the category token. Ten annotations carried the canonical category plus a trailing justification on the same line, so the recorded identity was a prose blob, and where the prose wrapped it was a sentence fragment: `source-text-is-the-product — the workflow .md content IS`. Seven of those ten are ones this branch already rewrote. That pass renamed the token and left the prose, which is the same error this wave exists to correct, one level down: the label was fixed without checking what the machine reads. Justifications move to the following comment line, which the scanner ignores because it lacks the token. No annotation gains or loses an issue reference, so no exemption changes compliance status; the allowlist goes 161 to 159 as two files' duplicate identities collapse. git-base-branch.test.cjs carried the token twice — once as the real annotation, once echoed in docblock prose that the line scanner parsed as a second exemption with a truncated identity. The echo is reworded to drop the literal token. intel.test.cjs:1360 was cut off mid-clause with an issue ref appended after the break; its sentence is restored and the ref kept on the annotation line so it stays compliant. Every remaining non-canonical identity is an ESLint RuleTester fixture inside a `code:` template literal, which the line scanner cannot tell apart from an annotation. Those two stay grandfathered. Refs #3057 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
sim
|
7dd9e59f6b |
test(#3090): stop exempting violations under categories that do not fit
An allow-test-rule annotation citing a category that does not apply is worse than no annotation, because it reads as reviewed. Eight were confirmed by reading the assertions each one covered, and auditing the rest found five more plus one refutation — a converter test whose wording described the wrong mechanism while the covered assertion genuinely was deployed-text. The instructive one used the CANONICAL string for the same mistake: STATE.md command output labelled as a deployed artifact. A canonical string is not evidence the category fits, which is why normalising strings alone would have laundered the problem rather than fixed it. Every mapping the audit had inferred rather than code-verified was spot-checked before rewriting, and the ones that turned out not to fit were re-annotated rather than relabelled. Fourteen STATE.md assertions had a typed extractor available all along and now use it; their annotations came out because nothing needs exempting. Eight assertions genuinely need a production change first — CLI stdout and stderr with no structured mode — and are tagged pending-migration-to-typed-ir citing #3090, which is what that category is for. It had zero real uses before this, while one file carried a real citation to migration issue #2974 under a non-canonical tag. Six annotations covered assertions that do no text matching at all. An exemption for a violation that does not exist is noise that makes the real ones harder to audit; those are removed. atomic-write-coverage gains the annotation it always warranted — its own docstring describes a structural-regression-guard while the file carried none. Fifty-nine non-canonical strings across roughly thirty files are normalised, and the allow-test-rule allowlist is regenerated to match. 472 annotations became 463: every one now uses a canonical category, and the two remaining non-canonical strings are ESLint RuleTester fixtures, not annotations. Refs #3057 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> |
||
|
0xdhx
|
3274db2757 |
fix(#2526): remove gsd-ui-auditor's uncallable Playwright-MCP block (#2594)
* fix(#2526): drop gsd-ui-auditor's uncallable Playwright-MCP block The agent declares `tools: Read, Write, Bash, Grep, Glob, Skill` — no `mcp__*` grant of any kind — while its body presented a `<playwright_mcp_approach>` block as the *preferred* capture path. That branch was unreachable by construction: the availability check had a fixed answer, the three `mcp__playwright__*` calls could never dispatch, and the "when Playwright-MCP is NOT available" fallback was the only branch that ever ran — 39 lines of instruction loaded on every /gsd-ui-review spawn that also invited the model to claim a capture path it could not take. Remove the dead block, leaving the CLI screenshot path as the sole documented approach. Guard the class in tests/mcp-tool-inheritance.test.cjs, which already owns agent MCP-grant parity: the new block generalizes the #1284 researcher check from two agents and one dispatch table to every agents/*.md and its whole body — no agent may document an `mcp__<server>__*` namespace absent from its own `tools:` declaration. Frontmatter is read through the canonical parser (gsd-core/bin/lib/frontmatter.cjs) rather than a hand-rolled scan, so inline CSV, block sequences, flow arrays, quoted scalars and full-line comments are handled by construction; inline comments inside a scalar survive that parser, so they are stripped explicitly. The check is server-level by design, ignores prose metavariables like `mcp__X__*`, matches hyphenated server ids, and carries a discovery guard plus negative controls for every documented boundary so it cannot decay into a vacuous pass. The session-level Playwright-MCP pass in gsd-core/workflows/ui-review.md is deliberately untouched — workflow files carry no fixed allowlist, so their availability check is genuinely runtime-detected and honest. Fixes #2526 * chore(#2526): set changeset fragment pr to 2594 The fragment shipped with the documented `pr: 0` placeholder because the PR number does not exist until the PR is opened, and scripts/changeset/parse.cjs rejects `pr <= 0`. Now that the PR is open, set the real number so changeset-lint passes. * test(#2526): cover the two-char server-id boundary of the metavariable exclusion The length-1 "prose metavariable" exclusion was tested at length=1 and at real ids (>=3 chars), but never at length=2 — the limit+1 boundary where a server id starts being recognized. Review finding on #2594: `mcp__ab__foo` in a body with no grant must flag `['ab']`. * fix(#2526): treat a bare mcp__* grant as covering every server `grantedServers()` stripped `mcp__*` to the empty string and dropped it via `if (server)`, so an allowlist that grants every MCP server read as granting none — and the guard then fired against a body the grant plainly covered. That is the one input shape that inverts the check, turning it against a correct agent rather than merely missing a bad one. A `/^mcp__\*+$/` token now sets a GRANT_ALL sentinel that short-circuits `ungrantedServers()`. The sentinel `*` is outside REFERENCE_RE's character class, so no body reference can collide with it. A bare `mcp__` with no wildcard stays a typo rather than a grant and keeps failing closed. No agent uses the `mcp__*` spelling today, so this was latent rather than live. Two negative controls pin both halves. * fix(#2526): scan the frontmatter description for MCP references too `ungrantedServers()` scanned `stripFrontmatter(content)` only, so an `mcp__foo__bar` reference in the `description` field escaped the check. That field ships with the agent and the dispatcher reads it, which makes a dead reference there exactly as dead as one in the body. Only `description` is added to the scanned surface, never the whole frontmatter: `tools:` is the grant list itself, so scanning it would let every allowlist satisfy itself and turn the guard vacuous. A negative control pins that boundary alongside the new positive case. All 34 per-agent tests still pass with the wider surface, so no live agent verdict changes — this was latent. * test(#2526): give multi-character placeholders a convention the checker knows The metavariable exclusion is `length === 1`, so the natural placeholders `mcp__SRV__*` and `mcp__SERVER__*` were flagged as real references — and the failure message then offered an author two remedies ("grant the namespace or drop the block") that both misread what they wrote. Adopts the angle-bracket half of the suggested fix: `mcp__<SERVER>__*` is the sanctioned multi-character placeholder, exempt by construction because `<` is outside the reference pattern's character class. This pins an existing property rather than adding a special case. Declines the all-caps half. An all-caps exemption would be a false NEGATIVE for any real server spelled in caps, and a guard that misses a dead reference fails in exactly the direction this check exists to prevent. The bare-caps form keeps firing; the message now names the convention as a third remedy. Also corrects "grants neither" in that message, which was wrong for any count other than two. * test(#2526): pin the zero-length server id, completing the boundary triple `mcp____foo` yields `[]`, but for a different reason than the length-1 case: it is unrepresentable by `/mcp__([A-Za-z0-9_-]+?)__/g` since `+?` requires at least one character, so the pattern skips it before the metavariable exclusion is ever consulted. Pinning limit-1 completes the 0/1/2 boundary rule on its own terms and records which mechanism owns the case. * docs(#2526): correct every drifted AGENTS.md Tools row, not just the one The review asked for the one-line `gsd-ui-auditor` correction (missing `Skill`). Sweeping the defect class first — every `**Tools**` row in docs/AGENTS.md against its agent's `tools:` frontmatter — found it was 26 of 34 rows, so the one-line framing was the reviewer's premise rather than the population. Breakdown of the 26: * 21 omitted `Skill`, 6 omitted `Edit` (overlapping) — under-promises, the same drift class as #2526 but in the harmless direction. * 8 wrote `mcp (context7)` as shorthand while frontmatter granted up to 8 servers (firecrawl, exa, tavily, ref, jina, perplexity, both context7s). * 1 was actively wrong: gsd-debug-session-manager documented `Task`, a tool name that no longer exists — the #2526 shape at the doc layer, naming a capability that cannot dispatch. Every row is now the frontmatter `tools:` value verbatim, which is also what makes the parity guard in the following commit non-brittle. The diff is 26 insertions / 26 deletions, all Tools rows. * test(#2526): guard AGENTS.md Tools rows against agent frontmatter The 26 corrected rows in the previous commit were free to drift because nothing asserted the role card and the frontmatter agreed — the same reason the #2526 block itself survived. Correcting them without an invariant just resets the clock. Lands in agent-classification-parity.test.cjs rather than a new file: that suite already owns docs/AGENTS.md as a contract surface, already carries the `allow-test-rule` exemption for treating the doc as the product, and file count is the unit of CI overhead (docs/TESTING-SUITES.md). Compares the row to the frontmatter value VERBATIM, not as a set — a set comparison would keep accepting the "mcp (context7)" shorthand that hid eight grants behind one, which is the under-documentation half of the drift. Carries the same discovery guard #2526's own check uses: a section with a granted `tools:` but no **Tools** row fails loudly, so deleting a row cannot silently retire its assertion. Both halves are negative-controlled — against the pre-fix doc it fails naming 26 rows (gsd-ui-auditor:339 among them), and with a row deleted it fails on the missing-row assertion. * chore(#2526): note the AGENTS.md drift correction in the changeset The role cards are user-visible, and 26 of them documented a tool set the agent did not have. Type, `pr: 2594`, and the trailing `(#2526)` are unchanged. * fix(#2526): use a CRLF-safe split in the AGENTS.md Tools-row guard `lint-tests` (npm run lint:ci) rejected `rawAgentsMd.split('\n')` under the repo's local/no-crlf-fragile-split rule: Windows autocrlf yields CRLF, so a trailing \r rides into the parsed line. Switched to `/\r?\n/`. Caught by CI on the round-3 push before the response comment went out. * docs(#2526): correct the drift tallies stated in c5a9607e Re-derived the census programmatically from the pre-fix doc instead of by eye. The 26-of-34 headline was right; the breakdown was not. Skill omitted 21 -> 22 Edit omitted 6 -> 7 "mcp (context7)" shorthand 8 rows -> 7 rows The 8 was conflating two things: 8 rows omitted MCP grants entirely, but only 7 of them used the "mcp (context7)" shorthand — gsd-executor listed no MCP at all. Also names the one `Agent` omission (gsd-debug-session-manager, the row that still read `Task`). c5a9607e's message keeps the wrong numbers rather than rewriting a pushed branch mid-review; the test comment and changeset are the durable statements and both are corrected here. --------- Co-authored-by: Tom Boucher <trekkie@nomorestars.com> |
||
|
Behruz Nassre Esfahani
|
fc5ca178a2 |
test(#1178): consolidate duplicated agent-roster helper into tests/helpers (#1420)
* test(#1178): consolidate duplicated agent-roster helper into tests/helpers The "list gsd-*.md agent files, strip .md, sort" derivation was hand-duplicated across the suite (two listAgentFiles(), an identical agentFilesOnDisk(), and inline readdir blocks). Add tests/helpers/agent-roster.cjs exporting listAgentFiles(agentsDir?) and route the genuinely-identical source-roster sites through it. Semantically-different sites (installed-dest dirs, absolute-path returns, .toml-inclusive Codex rosters, full-.md-filename readers, the uniform multi-family inventory table) are left intact, each with a one-line comment. Test-only; no production code touched. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * test(#1178): note AGENTS_DIR export is for future call sites Review nit: clarify that the currently-unused AGENTS_DIR export is intentional — available for future tests needing the canonical source agents path. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Tom Boucher <trekkie@nomorestars.com> |
||
|
Tom Boucher
|
a19a709e62 |
test(#1171): add agent-classification parity guard (#1176)
Makes docs/AGENTS.md section structure the single source of truth for the
primary-vs-advanced agent classification and fails when docs/INVENTORY.md's
"Primary doc" column or the AGENTS.md prose counts drift from it.
The classification ("primary" = full role card, "advanced stub" = concise
stub) was hand-duplicated across three doc surfaces with no enforcement.
This drift guard derives the expected class from AGENTS.md section placement
and cross-checks the INVENTORY.md column, the prose counts, the parenthetical
advanced-agent list, and full roster completeness (21 primary / 12 advanced
/ 0 inventory-only today).
No agent frontmatter field is added (avoids the capability-registry `tier`
collision and the research-profiles.cjs ripple); the classification stays
documentary, enforced from where it is defined.
Closes #1171
Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
|