* test(#3897): failing-first coverage for §8.3 rungs 2-4 ADR-3473 §8.3 has four rungs; #3883/PR #3896 shipped the first. This pins the other three RED before any fix. Rung 2 — the install marker has four readers and resolveRuntime is not one. resolveRuntime resolves GSD_RUNTIME > config.runtime > 'claude' and reads no marker at all, while bin/install.js writes one (#2297) and FOUR hand-rolled readInstallRuntimeMarker copies exist: src/model-resolver.cts:65 (cached, with test seams), hooks/gsd-agent-isolation-guard.js:112, and TWICE in hooks/gsd-cursor-subagent-start.js at :346 and :355. Four copies of one rule. Fixtures and seam names mined from PR #3382 rather than re-derived; it implemented this rung and was closed "not on the merits". Rung 3 — the sandbox map, and the fallback that was the real defect. Measured across all 35 files in agents/, deriving workspace-write iff tools: declares Write or Edit: - all 11 CODEX_AGENT_SANDBOX entries derive to their mapped value exactly, zero disagreements — the map carries nothing the contract does not - 24 roles fall through `|| 'read-only'`, of which 16 declare Write or Edit So the map is redundant and the silent fallback is the defect. The maintainer chose to derive but hold those 16 at read-only pending the question of whether Codex enforces sandbox_mode or merely advises; HALT.md records it. T20 asserts the emitted sandbox_mode PER ROLE against a captured baseline, not in aggregate — an aggregate passes while one role silently widens, which is the proxy-instead-of-identity shape this repo names. T24 and T25 fail on a stale hold, so the hold list cannot rot into the subset map being deleted. Rung 4 — shortFormToId, recovered rather than invented. I nearly reported this as another wrong §8.3 claim: `git log -S shortFormToId` returns only documentation commits. That was the wrong instrument. Direct inspection of sdk/src/query/phase.ts at 11918dcc3^ shows five occurrences, and the tests match that code rather than a guess at its semantics — including first-write-wins on a duplicate short form. T43 asserts at the consumer's output: the emitted `waves` map from the real CLI, which pre-fix collapses to {"1":[...]} because every short-form edge is dropped. A unit assertion on resolveDependencyId would have passed throughout this defect's life. Observed RED, this tree: rung 2 11/11 fail — no marker rung, no seams rung 3 T23,T24,T25,T26,T30 fail; T28 fails (validate agents passes a TOML whose sandbox_mode disagrees — it checks presence only) rung 4 T42,T44 fail; T43,T49 fail with waves collapsed to a single wave 1 Green and staying green: T20/T21/T22/T27 as captured baselines, #3885's unresolvable-token warning and wave-verdict suppression, and #3785's display-mapping passthrough. If the third tier over-reaches, those go red — that is their job. Disclosed weakness: T45 (a canonical id with no dash is not short-form indexed) cannot be isolated behaviorally, because planMap always masks it. It is a non-crash boundary pin, weaker than the other rows, and is recorded as such rather than presented as equivalent. Design: .gsd/phase/feat-3897-adr3473-83-rungs/40-design.md Test matrix: .gsd/phase/feat-3897-adr3473-83-rungs/50-test-matrix.md Decision: .gsd/phase/feat-3897-adr3473-83-rungs/HALT.md Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * enhance(#3897): §8.3 rungs 2-4 — one marker reader, a derived sandbox, the third depends_on tier ADR-3473 §8.3 has four rungs. #3883/PR #3896 shipped the first. These are the other three. Rung 2 — the install marker had four readers, and resolveRuntime was not one. resolveRuntime resolved GSD_RUNTIME > config.runtime > 'claude' and read no marker, while bin/install.js writes one (#2297) and four hand-rolled readInstallRuntimeMarker copies existed: src/model-resolver.cts (cached, with seams), hooks/gsd-agent-isolation-guard.js, and twice in hooks/gsd-cursor-subagent-start.js. model-resolver's was already the house idiom, so it was promoted rather than replaced: src/runtime-slash.cts now owns it, and model-resolver plus both hooks delegate. The hooks reach it through ensureRuntimeBuild(), the seam lint-hooks-runtime-build-seam enforces. No import cycle existed - checked both directions before moving anything. The marker is the THIRD rung: env > project config > marker > 'claude'. N1 was checked rather than assumed, and my first reading of it was wrong. A marker holding an unknown name comes back essentially verbatim, which looked like a validation gap. Measured against the env rung with the same inputs - including "../../etc/passwd" and "claude;rm -rf /" - the two are identical, because they share resolveRuntimeNameFromCandidates. N1 asks for exactly that, and it is met. The residual (the shared normalizer normalizes shape, it does not validate against the known-runtime set) is pre-existing on the env rung and plausibly deliberate, since a new runtime should not need a code change. The marker also does not widen the trust boundary in any real sense: it lives inside the install tree beside the code, so anyone who can write it can write runtime-slash.cjs itself. Rung 3 — the map was redundant; the silent fallback was the defect. Measured across all 35 files in agents/, deriving workspace-write iff tools: declares Write or Edit: all 11 CODEX_AGENT_SANDBOX entries derive to their mapped value exactly, zero disagreements. The map carried nothing the contract did not already have, so it is DELETED rather than reconciled. What was actually broken is `|| 'read-only'`, which silently under-granted 24 of 35 roles. 16 of those 24 declare Write or Edit and would widen under derivation. Per the maintainer's decision (HALT.md), they are held at read-only pending the question of whether Codex enforces sandbox_mode or merely advises. Emitted TOML is therefore byte-identical for all 35 roles - asserted per role, not in aggregate, because an aggregate passes while one role silently widens. The hold list self-invalidates. A hold whose role no longer derives broader fails, and so does a hold naming a role with no agents/<name>.md. Without that it would rot into exactly the hand-maintained subset map being deleted, and this commit's own ledger claim would become false over time. Both cases were proved by injecting them and watching them throw. Two committed tests asserted the deleted map's existence and contents. They were pinning the thing being removed, so the tests moved rather than the production code: the 11 role-value pairs survive as a test-local PRE_3897_CODEX_AGENT_SANDBOX baseline, and the assertions now drive the real derivation against real agents/*.md. The coverage is preserved; only its source moved out of production code. validate agents gains checkCodexSandboxPosture, mirroring the existing checkCodexModelPosture: each installed TOML's sandbox_mode must equal the role's expected value, failing with role, expected and found. It previously checked file presence and manifest completeness only, so a TOML whose sandbox_mode disagreed passed. Rung 4 — shortFormToId, recovered rather than invented. I nearly reported this as another wrong §8.3 claim: git log -S returns only documentation commits. Wrong instrument. sdk/src/query/phase.ts at 11918dcc3^ carries five occurrences, and the implementation here matches that code rather than a guess at its semantics - including first-write-wins on a duplicate short form, deterministic from the sorted plan order. It resolves the bare plan number: depends_on: ["01"] now reaches 26-01-auth-hardening. That is a control-flow change, not a diagnostic one - plans that silently collapsed into a single wave 1 now execute in their declared waves, and execute-phase.md consumes those wave values. In-phase only, by construction: the map is built from this phase's rawPlans, so a same-named short form in another phase does not resolve. #3785's display-mapping passthrough and #3885's unresolvable-token warning and wave-verdict suppression are untouched and stay green. If the third tier had over-reached, those are what would have caught it. Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#3897): close a fail-open I introduced, and wire the posture check to its command Two blockers from review. Both are mine, and one is a security regression my own change created. 1. A held role could escape its hold by editing its own frontmatter. The Codex install loop set the sandbox identity from the agent's frontmatter `name:` field rather than from its filename, so the hold lookup keyed off a value the file itself declares: deriveCodexSandboxMode('gsd-doc-writer', <real file>) -> read-only deriveCodexSandboxMode('gsd-doc-writer-x', <same file, name: edited>) -> workspace-write deriveCodexSandboxMode('GSD-Doc-Writer', <same file, name: recased>) -> workspace-write What makes this a blocker rather than a nit is the DIRECTION. The deleted CODEX_AGENT_SANDBOX map had the identical lookup-key quirk, but it was an allowlist: an unmatched key fell back to read-only, which is safe. The new scheme derives workspace-write from the tool contract and uses the hold as a subtraction, so the same mismatch fails OPEN. I converted a fail-closed quirk into a fail-open one and did not notice; the isolated reviewer proved it by execution. Neither safety net caught it. validateCodexSandboxHolds only checks that <key>.md exists, never that a file's derived identity matches its key. checkCodexSandboxPosture looks the canonical source up by the installed TOML's filename, finds nothing for a renamed agent, and treats it as a custom non-roster agent — silently no violation. The identity is now the FILENAME STEM, which is what validateCodexSandboxHolds already validates and what an attacker editing frontmatter cannot change without renaming the file — at which point the existing validator catches it. The lookup is case-insensitive so a recase does not slip past either. The frontmatter name still drives the TOML body and filename, unchanged; only the sandbox identity moved. All 35 roster files were checked: name matches filename stem everywhere, so a stricter "they must agree or throw" invariant would have been safe against real content. It is deliberately NOT added — it would abort an install on a tampered file where emitting a correctly-derived read-only TOML is the safer outcome. Recorded as a fork rather than decided silently. 2. checkCodexSandboxPosture was exported and never called. cmdValidateAgents (src/verify.cts) called checkAgentsInstalled and checkCodexModelPosture only; grep for the sandbox check in that file returned nothing. So criterion 3 — "validate agents fails on semantic drift, not only on missing files" — was unmet, and `validate agents` behaved exactly as before. That is ADR-3473 Decision 2's named shape: a declared policy with no executor. It also meant the T28 test asserted at the helper's return value while the COMMAND stayed broken — the ADR-3180 Decision 4(b) failure this epic exists to close, committed by me while enforcing it elsewhere in the same epic. Now wired as an additive `sandbox_posture` field beside `codex_posture`, following the sibling precedent exactly. Drift is report-only, not a non-zero exit, because that is what checkCodexModelPosture does — two sibling posture checks disagreeing about whether a violation is fatal would be its own defect. The choice is recorded in a comment rather than left implicit. A consumer-output test now drives the real CLI and asserts on the emitted JSON, and was shown failing before the wiring and passing after. Also corrected a stale artifact: the design's Known limit L1 still claimed rung 3 was not in this deliverable, written while it was halted and false once the maintainer unblocked it. Verified after both fixes: the three bypass probes all return read-only, the per-role table is 35/35 byte-identical, and both hold self-invalidation cases still throw. Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#3897): the marker rung, the derived sandbox, and the bare plan-number depends_on Reference: the runtime precedence ladder in docs/CLI-TOOLS.md gains the install marker rung; docs/COMMANDS.md documents validate agents' new sandbox_posture field; docs/reference/plan-md.md documents that depends_on accepts the bare plan number. Explanation: a docs/features fragment keyed id 3897, so it cannot collide with a concurrent PR hand-allocating a section number, regenerated into FEATURES.md. ADR-3473 §8.3 gains an ANSWER blockquote in the document's own correction style, recording what was measured and built against the section's 2026-08-26 correction - including the qualification that checkAgentsInstalled itself still checks presence only, and the semantic assertion lives in a sibling wired into validate agents rather than folded into it. No how-to. Both user-visible changes are zero-step: a non-Claude install resolving its own runtime, and plans executing in their declared waves, both happen without the user doing anything. docs/how-to/control-the-reported-host-runtime.md covers a DIFFERENT ladder (resolveReportedRuntime / agent_runtime) that this change does not touch, and was deliberately left alone rather than edited by association. No tutorial - nothing multi-step to walk through. docs/AGENTS.md unchanged: it documents Claude-side tools frontmatter, never Codex sandbox_mode, and the emitted tools contract did not change. The prompt layer documents depends_on only by example, not by schema, so nothing there needed editing - and few-shot-examples/plan-checker.md already showed depends_on: ['01'], which now actually resolves. Translated copies of plan-md.md are untouched; the project treats translations as community-maintained. Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#3897): move the sandbox derivation out of the installer, off the install path, and off a third parser The full suite came back with 26 failures across four files. Three distinct causes, mapped individually rather than assuming the first explained the rest. A. Requiring bin/install.js printed the GSD banner to stdout and corrupted `validate agents` JSON. Unexpected token '', "[36m ██"... is not valid JSON checkCodexSandboxPosture reached deriveCodexSandboxMode by lazily requiring bin/install.js, whose module load prints the ASCII banner. So the command emitted banner bytes before its JSON and every JSON consumer broke, including ten tests that predate this branch. src/ reaching into bin/ was backwards layering that happened to also be loud. The derivation now lives in src/codex-agent-toml.cts - the existing Codex TOML domain module, no new module and no six-gate ripple - and both bin/install.js and src/agent-install-check.cts import it. One owner, which is §8.3's rule applied to the fix for §8.3. B. The stale-hold throw fired on a legitimate partial source dir, and masked a security assertion. validateCodexSandboxHolds treated "this hold's .md is absent from the install SOURCE dir" as a stale hold and threw. A test fixture, or any partial install source, legitimately contains a couple of agents. Worse, it threw BEFORE the path-escape check, so a test asserting that a `../../evil` frontmatter name is rejected got my unrelated error instead of the traversal rejection it was written for. A fail-closed check of mine was hiding a real security check. The "no stale holds, shrink-only" invariant is a property of the repo's canonical agents/ roster, not of whatever directory an install happens to read. It is off the runtime path and enforced where it belongs, in the tests that already existed for it. A partial source dir now installs cleanly, and the evil-name case throws with its own escapes-configHome message again. C. T8 depended on ambient process.env state. The marker/env parity assertion round-tripped through live process.env. It now compares against resolveExplicitRuntime's already-exported dependency-injection parameter - deterministic and hermetic, same claim. Proven still falsifiable rather than assumed: with the marker rung's normalization temporarily bypassed the two rungs diverge ("codex\n../../etc/passwd" vs "codex-../../etc/passwd") and the assertion fails, then passes again once reverted. One correction folded in along the way. The first version of the move added private _extractFrontmatterAndBody/_extractFrontmatterField helpers to codex-agent-toml.cts - a THIRD copy of frontmatter extraction, where the graph already shows two (bin/install.js:2348, runtime-artifact-conversion.cts:893). Adding a third inside the epic whose thesis is one implementation per rule is not defensible. deriveCodexSandboxMode no longer parses anything: it takes (identity, toolsValue) and each caller supplies the tools value using the extractor it already has. Both helpers are deleted. The identity argument is still the filename stem, so the fail-open fix is untouched. Verified after all three: `validate agents --raw` emits parseable JSON with no banner and both posture fields; the four hold-bypass probes still return read-only; the per-role table is 35/35 byte-identical at 26 read-only / 9 workspace-write; the hold list is still 16. Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#3897): drop a dev-only transitive dep, make the derivation total, retire a stale fallback test Suite down to 7 failures from 26. Three more causes, mapped individually. A. My extractor import dragged in a script that does not exist in an installed tree. Cannot find module '../../../scripts/fix-slash-commands.cjs' Chain: src/agent-install-check.cts imported runtime-artifact-conversion.cjs, which requires command-roster.cjs, whose line 36 requires ../../../scripts/fix-slash-commands.cjs. That path exists in the repo and not in an install, so every test exercising a synthetic install dir died at module load. I picked that extractor for convenience without checking what it pulls in - the same mistake that produced the banner bug, one layer further out. agent-install-check now uses a single-purpose extractToolsLine on codex-agent-toml.cts. That is deliberately NOT a general frontmatter parser: we deleted those helpers a commit ago for good reason, and this reads one line. Verified from outside the repo root that requiring either module prints nothing and does not throw. B. A test pinned the deleted name-based fallback. 'defaults unknown agents to read-only' called generateCodexAgentToml with a fixture declaring tools: Read, Write, Edit. Under derivation an unknown agent with a writing contract correctly derives workspace-write - design row S6, a new writing role gets the contract, not the pin. The behavior it asserted was the silent fallback this rung deleted; identity no longer decides the sandbox. Replaced with two rows rather than a flipped string: no tools declared -> read-only (absence is not a grant), and Write/Edit declared -> workspace-write. Strictly more coverage than the row it replaces. C. The stale-hold check still threw per derivation call. Last commit took the roster-existence check off the install path, but deriveCodexSandboxMode itself still threw when a hold's role did not derive broader FOR THE CONTENT IT WAS HANDED - so it fired on any synthetic fixture for a held role. The throw is gone, and it cost nothing: if a held role's content does not derive broader, the hold pins read-only and derivation returns read-only anyway, so the hold is a no-op and there is nothing to fail about. The staleness invariant is a property of the real agents/ roster, and validateCodexSandboxHolds still enforces it there - confirmed against the real roster after the change, not assumed. deriveCodexSandboxMode is now total: every (identity, toolsValue) including undefined and null returns read-only or workspace-write, never throws. Verified: validate agents emits parseable JSON; the four hold-bypass probes return read-only; the per-role table is 35/35 at 26 read-only / 9 workspace-write. Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#3897): put the rung-3 decision in the shipped docs instead of pointing at an ignored path The ADR entry and the feature fragment both ended their rung-3 explanation with "see .gsd/phase/feat-3897-adr3473-83-rungs/45-decision-rung3-sandbox.md". That directory is gitignored (.gitignore:55), so the rationale for holding 16 roles at read-only was reachable only from the machine that produced it. A reader of the ADR got a pointer to nothing. Both now carry the reasoning inline: the criterion asks both that the sandbox derive from the declared tool contract and that no role gain a broader sandbox, and those cannot both hold, because a faithful derivation widens 16 roles the deleted map never listed and that fell through its silent read-only default. The resolution is derive-and-hold - the derivation owns the rule now, each hold is released as its enforcement question is answered, and a hold is reversible where a widened sandbox that turns out to be enforced is not. Checked before assuming this was a defect class: CONTEXT.md cites .gsd/phase/<slug>/40-design.md as its standard Design: provenance line in eight module entries, and four other shipped docs do the same. Citing a phase artifact is an established convention here, so those are left alone. What was wrong was specific to these two: they put load-bearing rationale behind the pointer instead of provenance. docs/FEATURES.md regenerated from the fragment via scripts/gen-features.cjs rather than hand-edited. Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * fix(#3897): close a fail-open, stop a silent mis-resolution, and read a declaration as a declaration Two orthogonal reviews on the shipped sha. Three of the findings are the same failure class this epic exists to close, committed inside it. 1. BLOCKER - the sandbox was decided for one identity and applied to another. bin/install.js derived sandbox_mode for the filename stem and then wrote the result to `${name}.toml`, where name comes from the file's own frontmatter. Make the two disagree and a HELD role's artifact goes wide: rename gsd-doc-writer.md -> gsd-doc-writer-v2.md, keep name: gsd-doc-writer -> stem is unheld, derives workspace-write, lands on gsd-doc-writer.toml add any gsd-*.md whose frontmatter name: is a held role -> clobbers that role's toml with workspace-write Both emit read-only on origin/next, because the deleted map was an allowlist and a miss fell back safe. This is a regression my change introduced. The previous review round moved the HOLD KEY off frontmatter to the filename stem and left the OUTPUT PATH on frontmatter; my own comment at install.js:6985 calls that value attacker-editable, four lines above the line that uses it as the filename. The decision is now made over BOTH candidate identities, most-restrictive wins: if either the stem or the emitted name is held, the mode is read-only. 2. MAJOR - hold matching was toLowerCase() only, so confusables escaped. Turkish dotted/dotless i, fullwidth, NFD, trailing space/NBSP/dot/newline, ./ and ../agents/ all slipped the hold and emitted workspace-write. Identities are now basenamed, trimmed of NBSP/zero-width/control characters, NFKC-normalized and lowercased - and anything still carrying a character outside [a-z0-9._-] is treated as suspicious and derives read-only. We do not enumerate confusables; every shipped roster file is ASCII, so refusing to widen on an identity we cannot recognize is fail-closed with no false positives on real content. 3. MAJOR - the short-form depends_on tier mis-resolved SILENTLY. shortFormToId keyed on the last dash-segment of any canonical id with no constraint that it is a plan number, so a phase holding 09-FIX-auth-PLAN.md made depends_on: ["auth"] bind at wave 2 with zero warnings. This is the worst shape in the epic: the unresolvable-token warning fires on a DROPPED token, so a MIS-RESOLVED one is invisible and the tool reports a confident wave assignment built from a wrong edge. A wrong edge is worse than a missing one. The segment must now match /^\d+$/, which is exactly the contract docs/reference/plan-md.md already documents. This tier was recovered verbatim from the retired SDK lineage, which carried the same defect; we are deliberately NOT preserving it bug-for-bug, and the comment says so, so the next reader does not "restore" it. 4. MAJOR - the derivation was reading a declaration as an absence. extractToolsLine read one line, so a YAML list-form tools: block returned only its first item. Two roster files use list form, and gsd-nyquist-auditor declares Write and Edit there - parsed as "- Read", found no write tool, and emitted read-only. Rung 3's headline claim is that sandbox_mode derives from the declared tool contract; that claim was false for 2 of 35 roles and materially wrong for 1. Reading a declaration as an absence is the silent-drop class this epic exists to close. Renamed extractToolsValue and taught it both shapes. gsd-nyquist-auditor now derives workspace-write and joins CODEX_SANDBOX_HOLDS as its 17th entry, per the standing derive-and-hold decision - so emitted TOML stays byte-identical at 26 read-only / 9 workspace-write while the hold list finally records every role that would widen. A previous pass declined this fix because it moved the count; that inverts the priority. Byte-identity is preserved THROUGH the hold, not by leaving a parser broken. Divergence check, because this is where that bug hides: both paths feeding sandbox derivation - install.js's emitter and checkCodexSandboxPosture - now route through the one extractor. The tools readers in runtime-artifact-conversion and install.js's other frontmatter call sites serve Claude-side emission and do not feed sandbox derivation. Also fixed, each real: the posture check's `found` used a naive whole-file regex where its own sibling uses the block-aware scanner, so prose inside developer_instructions produced a false violation; `found` skipped truncatePostureValue and leaked a 300-char value into validate agents output; deriveCodexSandboxMode's absolute never-throws claim was false for an object with a throwing toString; T49 could not falsify cross-phase leakage (its target phase had its own 01, so a globally-scoped map passed too); T20/N6 iterated a hardcoded table and pinned the FIXTURE size, so a 36th agent would be silently unchecked; three tests reimplemented the code they were testing instead of importing it; and T2-T4 deleted GSD_RUNTIME without restoring it. Verified: hold list 17, gsd-nyquist-auditor derives workspace-write unheld and emits read-only held, roster 35/35 at 26/9, depends_on ["auth"] no longer resolves while ["01"] still does, both identity-bypass cases and every confusable vector emit read-only. Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * docs(#3897): the hold list is 17, and the reason the 17th was missing The count read 16 because the derivation could not read the declaration it claimed to derive from: the tools reader was single-line, so a YAML list-form tools: block returned only its first item and gsd-nyquist-auditor's declared Write and Edit were read as an absence. Both the ADR entry and the feature fragment now carry the corrected count and the reason for it, rather than a silently updated number. Deriving from a declaration you cannot parse is not deriving, and a flattering count is worse than a wrong one because it looks settled. docs/FEATURES.md regenerated from the fragment. Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> * chore(#3897): backfill changeset pr number Refs #3897 Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> --------- Co-authored-by: sim <sim@local> Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
1520 lines
68 KiB
JavaScript
1520 lines
68 KiB
JavaScript
'use strict';
|
|
|
|
/**
|
|
* Agent Install Check Module — behaviour tests (#1268 T0, T1 #1277)
|
|
*
|
|
* Seam: gsd-core/bin/lib/agent-install-check.cjs
|
|
* Interface: getAgentsDir, checkAgentsInstalled
|
|
*
|
|
* Verifies:
|
|
* 1. getAgentsDir behaviour: GSD_AGENTS_DIR override, claude path, non-claude path
|
|
* 2. checkAgentsInstalled behaviour against temp dirs via GSD_AGENTS_DIR:
|
|
* - missing dir → agents_installed:false, missing_agents = all expected
|
|
* - existing-but-empty dir → installed_agents:[], agents_installed:false
|
|
* - no manifest → completeness skipped (incomplete_agents empty)
|
|
* - partial manifest (agent.toml absent, agent.md present) → incomplete_agents includes agent
|
|
* - malformed manifest → no throw, completeness skipped
|
|
*/
|
|
|
|
const { describe, test, beforeEach, afterEach } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const fs = require('node:fs');
|
|
const path = require('node:path');
|
|
const { createTempDir, cleanup } = require('./helpers.cjs');
|
|
|
|
const AGENT_INSTALL_CHECK_PATH = path.join(
|
|
__dirname, '..', 'gsd-core', 'bin', 'lib', 'agent-install-check.cjs'
|
|
);
|
|
const RUNTIME_HOMES_PATH = path.join(
|
|
__dirname, '..', 'gsd-core', 'bin', 'lib', 'runtime-homes.cjs'
|
|
);
|
|
|
|
const agentInstallCheck = require(AGENT_INSTALL_CHECK_PATH);
|
|
const { getGlobalConfigDir } = require(RUNTIME_HOMES_PATH);
|
|
const { getDirName } = require(path.join(__dirname, '..', 'gsd-core', 'bin', 'lib', 'runtime-name-policy.cjs'));
|
|
|
|
// Get EXPECTED_AGENTS from model-profiles (same source of truth)
|
|
const MODEL_PROFILES = require(path.join(__dirname, '..', 'gsd-core', 'bin', 'lib', 'model-profiles.cjs')).MODEL_PROFILES;
|
|
const EXPECTED_AGENTS = Object.keys(MODEL_PROFILES);
|
|
|
|
// #3242 — single source of truth for the Anthropic-flavored alias/id set (Phase 1
|
|
// moved the predicate here), so the posture tests below can't silently drift from
|
|
// what the posture check is actually supposed to reject.
|
|
const { CLAUDE_AGENT_ALIASES } = require(
|
|
path.join(__dirname, '..', 'gsd-core', 'bin', 'lib', 'model-catalog.cjs'),
|
|
);
|
|
|
|
// ─── Environment isolation ────────────────────────────────────────────────────
|
|
|
|
let savedAgentsDir;
|
|
let savedRuntime;
|
|
let savedCodexHome;
|
|
|
|
beforeEach(() => {
|
|
savedAgentsDir = process.env['GSD_AGENTS_DIR'];
|
|
savedRuntime = process.env['GSD_RUNTIME'];
|
|
savedCodexHome = process.env['CODEX_HOME'];
|
|
delete process.env['GSD_AGENTS_DIR'];
|
|
delete process.env['GSD_RUNTIME'];
|
|
delete process.env['CODEX_HOME'];
|
|
});
|
|
|
|
afterEach(() => {
|
|
if (savedAgentsDir === undefined) {
|
|
delete process.env['GSD_AGENTS_DIR'];
|
|
} else {
|
|
process.env['GSD_AGENTS_DIR'] = savedAgentsDir;
|
|
}
|
|
if (savedRuntime === undefined) {
|
|
delete process.env['GSD_RUNTIME'];
|
|
} else {
|
|
process.env['GSD_RUNTIME'] = savedRuntime;
|
|
}
|
|
if (savedCodexHome === undefined) {
|
|
delete process.env['CODEX_HOME'];
|
|
} else {
|
|
process.env['CODEX_HOME'] = savedCodexHome;
|
|
}
|
|
});
|
|
|
|
function createCompleteAgents(agentsDir) {
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
fs.writeFileSync(path.join(agentsDir, `${agent}.toml`), `name = "${agent}"\n`);
|
|
}
|
|
}
|
|
|
|
function markLocalGsdInstall(configDir) {
|
|
fs.writeFileSync(
|
|
path.join(configDir, 'gsd-file-manifest.json'),
|
|
JSON.stringify({ files: {} }),
|
|
);
|
|
}
|
|
|
|
function createCompleteLocalGsdInstall(configDir) {
|
|
const agentsDir = path.join(configDir, 'agents');
|
|
createCompleteAgents(agentsDir);
|
|
markLocalGsdInstall(configDir);
|
|
return agentsDir;
|
|
}
|
|
|
|
// ─── 1. getAgentsDir behaviour ────────────────────────────────────────────────
|
|
|
|
describe('getAgentsDir', () => {
|
|
test('GSD_AGENTS_DIR override takes priority', () => {
|
|
process.env['GSD_AGENTS_DIR'] = '/tmp/x';
|
|
assert.strictEqual(agentInstallCheck.getAgentsDir(), '/tmp/x');
|
|
assert.strictEqual(agentInstallCheck.getAgentsDir('cursor'), '/tmp/x');
|
|
});
|
|
|
|
test('claude runtime outside node_modules returns the install-relative path', () => {
|
|
// Repo runs and runtime-config-dir installs: the sibling agents/ IS the
|
|
// user's agents dir, so install-relative resolution is correct there.
|
|
const expected = path.resolve(
|
|
path.dirname(AGENT_INSTALL_CHECK_PATH), '..', '..', '..', 'agents'
|
|
);
|
|
assert.strictEqual(agentInstallCheck.getAgentsDir('claude'), expected);
|
|
});
|
|
|
|
test('npm-global install resolves the config dir, never the bundled agents (#3203)', (t) => {
|
|
// Mirror the published npm-global layout: the package sits inside a
|
|
// node_modules tree and ships its own agents/. Pre-fix, getAgentsDir
|
|
// resolved that bundled copy, so checkAgentsInstalled validated the
|
|
// package against itself and agents_installed could never be false.
|
|
const tmp = createTempDir('gsd-npm-global-');
|
|
const savedClaudeDir = process.env['CLAUDE_CONFIG_DIR'];
|
|
t.after(() => cleanup(tmp));
|
|
t.after(() => {
|
|
if (savedClaudeDir === undefined) {
|
|
delete process.env['CLAUDE_CONFIG_DIR'];
|
|
} else {
|
|
process.env['CLAUDE_CONFIG_DIR'] = savedClaudeDir;
|
|
}
|
|
});
|
|
|
|
const pkgRoot = path.join(tmp, 'node_modules', '@opengsd', 'gsd-core');
|
|
fs.cpSync(
|
|
path.join(__dirname, '..', 'gsd-core', 'bin'),
|
|
path.join(pkgRoot, 'gsd-core', 'bin'),
|
|
{ recursive: true }
|
|
);
|
|
// Bundled agents/ is always complete — that is exactly why the pre-fix
|
|
// self-validation could never report a missing agent.
|
|
const bundledAgents = path.join(pkgRoot, 'agents');
|
|
fs.mkdirSync(bundledAgents, { recursive: true });
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
fs.writeFileSync(path.join(bundledAgents, `${agent}.md`), `# ${agent}\n`);
|
|
}
|
|
|
|
// Config dir carries every expected agent except the first — the issue
|
|
// repro's negative control (gsd-verifier removed from ~/.claude/agents).
|
|
const configDir = path.join(tmp, 'claude-config');
|
|
const agentsDir = path.join(configDir, 'agents');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
const [removedAgent, ...presentAgents] = EXPECTED_AGENTS;
|
|
for (const agent of presentAgents) {
|
|
fs.writeFileSync(path.join(agentsDir, `${agent}.md`), `# ${agent}\n`);
|
|
}
|
|
process.env['CLAUDE_CONFIG_DIR'] = configDir;
|
|
|
|
const globalInstallCheck = require(
|
|
path.join(pkgRoot, 'gsd-core', 'bin', 'lib', 'agent-install-check.cjs')
|
|
);
|
|
// Pin the resolved directory, not just an /agents suffix — the pre-fix
|
|
// resolver also ended with /agents, which is how the bug survived.
|
|
assert.strictEqual(globalInstallCheck.getAgentsDir('claude'), agentsDir);
|
|
|
|
const result = globalInstallCheck.checkAgentsInstalled('claude');
|
|
assert.strictEqual(result.agents_dir, agentsDir);
|
|
assert.strictEqual(result.agents_installed, false);
|
|
assert.deepStrictEqual(result.missing_agents, [removedAgent]);
|
|
});
|
|
|
|
test('non-claude runtime returns getGlobalConfigDir(runtime)/agents', () => {
|
|
const runtime = 'cursor';
|
|
const expected = path.join(getGlobalConfigDir(runtime), 'agents');
|
|
assert.strictEqual(agentInstallCheck.getAgentsDir(runtime), expected);
|
|
});
|
|
|
|
test('GSD_RUNTIME env var is respected when no argument provided', () => {
|
|
process.env['GSD_RUNTIME'] = 'codex';
|
|
const expected = path.join(getGlobalConfigDir('codex'), 'agents');
|
|
assert.strictEqual(agentInstallCheck.getAgentsDir(), expected);
|
|
});
|
|
|
|
test('defaults to claude when no arg and no GSD_RUNTIME', () => {
|
|
const fromModule = agentInstallCheck.getAgentsDir();
|
|
const fromClaude = agentInstallCheck.getAgentsDir('claude');
|
|
assert.strictEqual(fromModule, fromClaude);
|
|
});
|
|
|
|
test('a manifest-backed local runtime installation wins over global agents', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
const globalHome = createTempDir('gsd-global-codex-');
|
|
const localAgentsDir = createCompleteLocalGsdInstall(path.join(projectRoot, '.codex'));
|
|
t.after(() => cleanup(projectRoot));
|
|
t.after(() => cleanup(globalHome));
|
|
createCompleteAgents(path.join(globalHome, 'agents'));
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
|
|
assert.strictEqual(agentInstallCheck.getAgentsDir('codex', projectRoot), localAgentsDir);
|
|
assert.strictEqual(agentInstallCheck.checkAgentsInstalled('codex', projectRoot).agents_installed, true);
|
|
});
|
|
|
|
test('GSD_AGENTS_DIR remains terminal when a local Codex installation exists', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
const overrideDir = path.join(projectRoot, 'override-agents');
|
|
createCompleteLocalGsdInstall(path.join(projectRoot, '.codex'));
|
|
t.after(() => cleanup(projectRoot));
|
|
fs.mkdirSync(overrideDir, { recursive: true });
|
|
process.env['GSD_AGENTS_DIR'] = overrideDir;
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled('codex', projectRoot);
|
|
assert.strictEqual(result.agents_dir, overrideDir);
|
|
assert.strictEqual(result.agents_installed, false);
|
|
assert.deepStrictEqual(result.missing_agents, EXPECTED_AGENTS);
|
|
});
|
|
|
|
test('a manifest-backed empty local directory is authoritative over complete global agents', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
const globalHome = createTempDir('gsd-global-codex-');
|
|
const localAgentsDir = path.join(projectRoot, '.codex', 'agents');
|
|
t.after(() => cleanup(projectRoot));
|
|
t.after(() => cleanup(globalHome));
|
|
fs.mkdirSync(localAgentsDir, { recursive: true });
|
|
markLocalGsdInstall(path.dirname(localAgentsDir));
|
|
createCompleteAgents(path.join(globalHome, 'agents'));
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled('codex', projectRoot);
|
|
assert.strictEqual(result.agents_dir, localAgentsDir);
|
|
assert.strictEqual(result.agents_installed, false);
|
|
assert.deepStrictEqual(result.missing_agents, EXPECTED_AGENTS);
|
|
});
|
|
|
|
test('Codex falls back to global agents when no local directory exists', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
const globalHome = createTempDir('gsd-global-codex-');
|
|
t.after(() => cleanup(projectRoot));
|
|
t.after(() => cleanup(globalHome));
|
|
createCompleteAgents(path.join(globalHome, 'agents'));
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled('codex', projectRoot);
|
|
assert.strictEqual(result.agents_dir, path.join(globalHome, 'agents'));
|
|
assert.strictEqual(result.agents_installed, true);
|
|
});
|
|
|
|
test('Codex falls back to global agents when the local candidate is a regular file', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
const globalHome = createTempDir('gsd-global-codex-');
|
|
const localCandidate = path.join(projectRoot, '.codex', 'agents');
|
|
t.after(() => cleanup(projectRoot));
|
|
t.after(() => cleanup(globalHome));
|
|
fs.mkdirSync(path.dirname(localCandidate), { recursive: true });
|
|
fs.writeFileSync(localCandidate, 'not an agents directory\n');
|
|
createCompleteAgents(path.join(globalHome, 'agents'));
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled('codex', projectRoot);
|
|
assert.strictEqual(result.agents_dir, path.join(globalHome, 'agents'));
|
|
assert.strictEqual(result.agents_installed, true);
|
|
});
|
|
|
|
test('Codex falls back to global agents when the local candidate cannot be inspected', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
const globalHome = createTempDir('gsd-global-codex-');
|
|
const localAgentsDir = path.join(projectRoot, '.codex', 'agents');
|
|
const realLstatSync = fs.lstatSync;
|
|
t.after(() => cleanup(projectRoot));
|
|
t.after(() => cleanup(globalHome));
|
|
fs.mkdirSync(localAgentsDir, { recursive: true });
|
|
markLocalGsdInstall(path.dirname(localAgentsDir));
|
|
createCompleteAgents(path.join(globalHome, 'agents'));
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
t.mock.method(fs, 'lstatSync', function injectedLocalProbeFailure(target, ...args) {
|
|
if (target === localAgentsDir) {
|
|
throw Object.assign(new Error('injected EACCES'), { code: 'EACCES' });
|
|
}
|
|
return realLstatSync.call(fs, target, ...args);
|
|
});
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled('codex', projectRoot);
|
|
assert.strictEqual(result.agents_dir, path.join(globalHome, 'agents'));
|
|
assert.strictEqual(result.agents_installed, true);
|
|
});
|
|
|
|
test('Codex does not follow a symlinked local agents directory', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
const globalHome = createTempDir('gsd-global-codex-');
|
|
const localConfigDir = path.join(projectRoot, '.codex');
|
|
const localAgentsDir = path.join(localConfigDir, 'agents');
|
|
const symlinkTarget = path.join(projectRoot, 'shared-agents');
|
|
t.after(() => cleanup(projectRoot));
|
|
t.after(() => cleanup(globalHome));
|
|
createCompleteAgents(symlinkTarget);
|
|
fs.mkdirSync(localConfigDir, { recursive: true });
|
|
markLocalGsdInstall(localConfigDir);
|
|
createCompleteAgents(path.join(globalHome, 'agents'));
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
try {
|
|
fs.symlinkSync(symlinkTarget, localAgentsDir, process.platform === 'win32' ? 'junction' : 'dir');
|
|
} catch (error) {
|
|
if (error && ['EPERM', 'EACCES', 'ENOTSUP'].includes(error.code)) {
|
|
t.skip('symlink creation is not available on this platform');
|
|
return;
|
|
}
|
|
throw error;
|
|
}
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled('codex', projectRoot);
|
|
assert.strictEqual(result.agents_dir, path.join(globalHome, 'agents'));
|
|
assert.strictEqual(result.agents_installed, true);
|
|
});
|
|
|
|
test('a project-native agents directory without a GSD manifest does not override global agents', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
const globalHome = createTempDir('gsd-global-codex-');
|
|
const localAgentsDir = path.join(projectRoot, '.codex', 'agents');
|
|
t.after(() => cleanup(projectRoot));
|
|
t.after(() => cleanup(globalHome));
|
|
createCompleteAgents(localAgentsDir);
|
|
createCompleteAgents(path.join(globalHome, 'agents'));
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled('codex', projectRoot);
|
|
assert.strictEqual(result.agents_dir, path.join(globalHome, 'agents'));
|
|
assert.strictEqual(result.agents_installed, true);
|
|
});
|
|
|
|
test('a manifest-backed Cursor installation resolves from the project root', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-cursor-');
|
|
const localAgentsDir = createCompleteLocalGsdInstall(path.join(projectRoot, getDirName('cursor')));
|
|
t.after(() => cleanup(projectRoot));
|
|
|
|
assert.strictEqual(agentInstallCheck.getAgentsDir('cursor', projectRoot), localAgentsDir);
|
|
assert.strictEqual(agentInstallCheck.checkAgentsInstalled('cursor', projectRoot).agents_installed, true);
|
|
});
|
|
|
|
test('a manifest-backed Cline installation resolves from the project root', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-cline-');
|
|
const localAgentsDir = createCompleteLocalGsdInstall(projectRoot);
|
|
t.after(() => cleanup(projectRoot));
|
|
|
|
assert.strictEqual(agentInstallCheck.getAgentsDir('cline', projectRoot), localAgentsDir);
|
|
assert.strictEqual(agentInstallCheck.checkAgentsInstalled('cline', projectRoot).agents_installed, true);
|
|
});
|
|
|
|
test('a partial manifest-backed local installation remains selected and incomplete', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
const localConfigDir = path.join(projectRoot, '.codex');
|
|
const localAgentsDir = createCompleteLocalGsdInstall(localConfigDir);
|
|
const partialAgent = EXPECTED_AGENTS[0];
|
|
t.after(() => cleanup(projectRoot));
|
|
fs.writeFileSync(path.join(localAgentsDir, `${partialAgent}.md`), `# ${partialAgent}\n`);
|
|
fs.unlinkSync(path.join(localAgentsDir, `${partialAgent}.toml`));
|
|
fs.writeFileSync(
|
|
path.join(localConfigDir, 'gsd-file-manifest.json'),
|
|
JSON.stringify({ files: { [`agents/${partialAgent}.md`]: {}, [`agents/${partialAgent}.toml`]: {} } }),
|
|
);
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled('codex', projectRoot);
|
|
assert.strictEqual(result.agents_dir, localAgentsDir);
|
|
assert.strictEqual(result.agents_installed, false);
|
|
assert.deepStrictEqual(result.missing_agents, []);
|
|
assert.deepStrictEqual(result.incomplete_agents, [partialAgent]);
|
|
});
|
|
|
|
test('Claude and other runtimes ignore a supplied Codex-local candidate', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-');
|
|
t.after(() => cleanup(projectRoot));
|
|
createCompleteLocalGsdInstall(path.join(projectRoot, '.codex'));
|
|
|
|
assert.strictEqual(
|
|
agentInstallCheck.getAgentsDir('claude', projectRoot),
|
|
agentInstallCheck.getAgentsDir('claude'),
|
|
);
|
|
assert.strictEqual(
|
|
agentInstallCheck.getAgentsDir('cursor', projectRoot),
|
|
path.join(getGlobalConfigDir('cursor'), 'agents'),
|
|
);
|
|
});
|
|
});
|
|
|
|
// ─── 2. checkAgentsInstalled behaviour ───────────────────────────────────────
|
|
|
|
describe('checkAgentsInstalled', () => {
|
|
let tmpDir;
|
|
|
|
beforeEach(() => {
|
|
tmpDir = createTempDir('gsd-agent-check-');
|
|
// Point GSD_AGENTS_DIR at a path we control
|
|
process.env['GSD_AGENTS_DIR'] = path.join(tmpDir, 'agents');
|
|
});
|
|
|
|
afterEach(() => {
|
|
cleanup(tmpDir);
|
|
});
|
|
|
|
test('missing dir → agents_installed:false, missing_agents = all expected', () => {
|
|
// agents dir does not exist
|
|
const result = agentInstallCheck.checkAgentsInstalled();
|
|
assert.strictEqual(result.agents_installed, false);
|
|
assert.deepStrictEqual(result.missing_agents, EXPECTED_AGENTS);
|
|
assert.deepStrictEqual(result.installed_agents, []);
|
|
assert.deepStrictEqual(result.incomplete_agents, []);
|
|
});
|
|
|
|
test('existing-but-empty dir → installed_agents:[], agents_installed:false', () => {
|
|
const agentsDir = path.join(tmpDir, 'agents');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled();
|
|
assert.strictEqual(result.agents_installed, false);
|
|
assert.deepStrictEqual(result.installed_agents, []);
|
|
assert.ok(result.missing_agents.length > 0, 'missing_agents should not be empty');
|
|
// No manifest → completeness skipped
|
|
assert.deepStrictEqual(result.incomplete_agents, []);
|
|
});
|
|
|
|
test('all agents present, no manifest → agents_installed:true, incomplete_agents:[]', () => {
|
|
const agentsDir = path.join(tmpDir, 'agents');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
// Write all expected agent .md files
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
fs.writeFileSync(path.join(agentsDir, `${agent}.md`), `# ${agent}\n`);
|
|
}
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled();
|
|
assert.strictEqual(result.agents_installed, true);
|
|
assert.deepStrictEqual(result.missing_agents, []);
|
|
assert.deepStrictEqual(result.installed_agents, EXPECTED_AGENTS);
|
|
assert.deepStrictEqual(result.incomplete_agents, []);
|
|
});
|
|
|
|
test('partial manifest: agent.toml absent but agent.md present → incomplete_agents includes agent', () => {
|
|
const agentsDir = path.join(tmpDir, 'agents');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
|
|
// Write all agent .md files so presence check passes
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
fs.writeFileSync(path.join(agentsDir, `${agent}.md`), `# ${agent}\n`);
|
|
}
|
|
|
|
// Pick the first expected agent to make "incomplete" via manifest
|
|
const targetAgent = EXPECTED_AGENTS[0];
|
|
|
|
// Write manifest that tracks agent.toml for targetAgent (absent on disk)
|
|
// and tracks agent.md for all others (present)
|
|
const manifestFiles = {};
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
manifestFiles[`agents/${agent}.md`] = {};
|
|
}
|
|
// Add a .toml for targetAgent to manifest (not present on disk)
|
|
manifestFiles[`agents/${targetAgent}.toml`] = {};
|
|
|
|
const manifest = { files: manifestFiles };
|
|
fs.writeFileSync(
|
|
path.join(tmpDir, 'gsd-file-manifest.json'),
|
|
JSON.stringify(manifest)
|
|
);
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled();
|
|
assert.ok(result.incomplete_agents.includes(targetAgent),
|
|
`Expected ${targetAgent} in incomplete_agents, got: ${JSON.stringify(result.incomplete_agents)}`);
|
|
assert.strictEqual(result.agents_installed, false,
|
|
'agents_installed must be false when any agent is incomplete');
|
|
});
|
|
|
|
test('malformed manifest → no throw, completeness skipped (incomplete_agents:[])', () => {
|
|
const agentsDir = path.join(tmpDir, 'agents');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
// Write all agent files
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
fs.writeFileSync(path.join(agentsDir, `${agent}.md`), `# ${agent}\n`);
|
|
}
|
|
|
|
// Write malformed manifest
|
|
fs.writeFileSync(path.join(tmpDir, 'gsd-file-manifest.json'), '{not json"');
|
|
|
|
let result;
|
|
assert.doesNotThrow(() => {
|
|
result = agentInstallCheck.checkAgentsInstalled();
|
|
});
|
|
// Malformed → completeness skipped → incomplete_agents empty
|
|
assert.deepStrictEqual(result.incomplete_agents, []);
|
|
// But presence check still passed
|
|
assert.strictEqual(result.agents_installed, true);
|
|
});
|
|
|
|
test('agents_dir and agent_runtime are returned in result', () => {
|
|
const agentsDir = path.join(tmpDir, 'agents');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled('cursor');
|
|
// GSD_AGENTS_DIR overrides, so agents_dir = our tmp path
|
|
assert.strictEqual(result.agents_dir, agentsDir);
|
|
assert.strictEqual(result.agent_runtime, 'cursor');
|
|
});
|
|
});
|
|
|
|
// ─── 3. checkCodexModelPosture behaviour (#3242, ADR-2313 D6) ─────────────────
|
|
//
|
|
// Spec: .gsd/phase/feat-3242-codex-posture-health-check/{40-design,50-test-matrix}.md
|
|
// Interface (not yet implemented — every test below is red until it lands):
|
|
// POSTURE_REASON: frozen enum { ANTHROPIC_FLAVORED_MODEL, ORPHANED_REASONING_EFFORT,
|
|
// UNREADABLE, NOT_CODEX, AGENTS_DIR_MISSING }
|
|
// checkCodexModelPosture(runtime?, projectRoot?) => {
|
|
// ok, violations: [{ agent, file, reason, value? }], checked, agents_dir,
|
|
// agent_runtime, reason?
|
|
// }
|
|
//
|
|
// Row numbers below (# N) map 1:1 to 50-test-matrix.md. Rows 12, 13, 14, 15, 16, 25
|
|
// are the negative proofs the matrix calls out as the ones that actually discriminate
|
|
// a correct implementation from a naive whole-file `/model\s*=/` scan; each of those
|
|
// test names states the specific implementation mistake it catches.
|
|
|
|
const POSTURE_FIXTURES_DIR = path.join(__dirname, 'fixtures', 'adversarial', 'toml');
|
|
|
|
function writeAgentToml(agentsDir, agentName, content) {
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
fs.writeFileSync(path.join(agentsDir, `${agentName}.toml`), content);
|
|
}
|
|
|
|
// Loads a hand-authored fixture (tests/fixtures/adversarial/toml/, #2371 provenance)
|
|
// as raw bytes so CRLF / BOM content is copied byte-for-byte, not re-encoded through
|
|
// a JS string round-trip that could normalize either.
|
|
function copyFixtureToml(agentsDir, agentName, fixtureFile) {
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
const raw = fs.readFileSync(path.join(POSTURE_FIXTURES_DIR, fixtureFile));
|
|
fs.writeFileSync(path.join(agentsDir, `${agentName}.toml`), raw);
|
|
}
|
|
|
|
// Row 18a's CRLF fixture is derived at test runtime, never read from a committed file.
|
|
// `.gitattributes:2` is `* text=auto eol=lf`, repo-wide and deliberate, so a `\r\n`
|
|
// fixture committed to disk is normalized to LF on every commit and every checkout —
|
|
// a whole-file CRLF fixture proves nothing about CRLF handling once git has touched it.
|
|
// Authoring the LF content inline and converting it here keeps the CRLF-ness under the
|
|
// test's control instead of git's. (The BOM fixture is unaffected by eol=lf — a BOM is
|
|
// not a line ending — so it stays a committed file.)
|
|
function toCrlf(lfContent) {
|
|
return lfContent.replace(/\n/g, '\r\n');
|
|
}
|
|
|
|
// Fault injection for row 21 — monkeypatches node:fs's readFileSync and restores it
|
|
// in a `finally` INSIDE this helper (never in a test body, and never chmod 0o000,
|
|
// which root bypasses under Docker/CI and would give the test zero real coverage).
|
|
function withInjectedReadFailure(targetPath, injectedError, fn) {
|
|
const realReadFileSync = fs.readFileSync;
|
|
fs.readFileSync = function poisonedReadFileSync(target, ...args) {
|
|
const targetStr = typeof target === 'string' ? target : String(target);
|
|
if (targetStr === targetPath || path.resolve(targetStr) === path.resolve(targetPath)) {
|
|
throw injectedError;
|
|
}
|
|
return realReadFileSync.apply(fs, [target, ...args]);
|
|
};
|
|
try {
|
|
return fn();
|
|
} finally {
|
|
fs.readFileSync = realReadFileSync;
|
|
}
|
|
}
|
|
|
|
describe('checkCodexModelPosture', () => {
|
|
let tmpDir;
|
|
let agentsDir;
|
|
|
|
beforeEach(() => {
|
|
tmpDir = createTempDir('gsd-posture-check-');
|
|
agentsDir = path.join(tmpDir, 'agents');
|
|
process.env['GSD_AGENTS_DIR'] = agentsDir;
|
|
});
|
|
|
|
afterEach(() => {
|
|
cleanup(tmpDir);
|
|
});
|
|
|
|
// # 1 — runtime is not codex: a no-op, not a failure, and it must not even read
|
|
// the filesystem to get there (checkAgentsInstalled's job is presence; this
|
|
// function's job starts only once the runtime is actually codex).
|
|
test('row 1: non-codex runtime (claude) is a no-op — NOT_CODEX, no violations, no fs.readFileSync call', (t) => {
|
|
const reads = [];
|
|
t.mock.method(fs, 'readFileSync', (...args) => {
|
|
reads.push(args[0]);
|
|
throw new Error('unreachable: readFileSync must not be called for a non-codex runtime');
|
|
});
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('claude', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
assert.strictEqual(result.reason, agentInstallCheck.POSTURE_REASON.NOT_CODEX);
|
|
assert.deepStrictEqual(reads, [], 'non-codex runtime must short-circuit before any file read');
|
|
});
|
|
|
|
// # 2 — codex runtime, agents dir absent: distinct from row-3 empty-dir and
|
|
// distinct from a violation. Presence is checkAgentsInstalled's job.
|
|
test('row 2: codex + agents dir absent — AGENTS_DIR_MISSING, not a violation', () => {
|
|
// agentsDir intentionally not created
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
assert.strictEqual(result.reason, agentInstallCheck.POSTURE_REASON.AGENTS_DIR_MISSING);
|
|
});
|
|
|
|
// # 3 — codex runtime, agents dir exists but is empty.
|
|
test('row 3: codex + empty agents dir — ok:true, checked:[]', () => {
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
assert.deepStrictEqual(result.checked, []);
|
|
});
|
|
|
|
// # 4 — one clean .toml (no model, no effort key at all).
|
|
test('row 4: clean .toml (no model, no effort) — ok:true, no violations', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-clean',
|
|
'name = "gsd-clean"\ndescription = "a clean agent"\ndeveloper_instructions = \'\'\'\nDo the work.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 5 — the base case: a bare Claude tier alias pinned as `model`.
|
|
test('row 5: model = "sonnet" — ANTHROPIC_FLAVORED_MODEL naming the agent and the value', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-planner',
|
|
'name = "gsd-planner"\nmodel = "sonnet"\ndeveloper_instructions = \'\'\'\nPlan.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].agent, 'gsd-planner');
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
assert.ok(result.violations[0].file.endsWith('gsd-planner.toml'));
|
|
});
|
|
|
|
// # 6 — table-driven over the full bare-alias set, sourced from model-catalog's
|
|
// CLAUDE_AGENT_ALIASES (not hardcoded) so this can't silently drift from the
|
|
// predicate the design says the posture check consumes.
|
|
for (const alias of CLAUDE_AGENT_ALIASES) {
|
|
test(`row 6: bare Claude alias "${alias}" — ANTHROPIC_FLAVORED_MODEL`, () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-alias-agent',
|
|
`name = "gsd-alias-agent"\nmodel = "${alias}"\ndeveloper_instructions = '''\nWork.\n'''\n`,
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, alias);
|
|
});
|
|
}
|
|
|
|
// # 7 — table-driven over full Claude model ids across provider namespacings,
|
|
// plus a case-insensitivity check.
|
|
for (const modelId of ['claude-opus-4-5', 'anthropic/claude-x', 'us.anthropic.claude-x', 'CLAUDE-X']) {
|
|
test(`row 7: Claude model id "${modelId}" — ANTHROPIC_FLAVORED_MODEL`, () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-id-agent',
|
|
`name = "gsd-id-agent"\nmodel = "${modelId}"\ndeveloper_instructions = '''\nWork.\n'''\n`,
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
});
|
|
}
|
|
|
|
// # 8 — the rule is Anthropic-flavored, never an allowlist: a real Codex/OpenAI id
|
|
// must never be flagged, however unfamiliar it looks.
|
|
test('row 8: model = "gpt-5.6-sol" — no violation (not an allowlist)', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-gpt-agent',
|
|
'name = "gsd-gpt-agent"\nmodel = "gpt-5.6-sol"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 9 — must not go stale on a hypothetical future OpenAI release id.
|
|
test('row 9: model = "some-future-model-id" — no violation', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-future-agent',
|
|
'name = "gsd-future-agent"\nmodel = "some-future-model-id"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 10 — #838 coupling: a static reasoning-effort with no model pin means Codex
|
|
// is inheriting the session model while GSD's effort pin silently disagrees.
|
|
test('row 10: model_reasoning_effort with no model — ORPHANED_REASONING_EFFORT', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-orphan-agent',
|
|
'name = "gsd-orphan-agent"\nmodel_reasoning_effort = "high"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ORPHANED_REASONING_EFFORT);
|
|
assert.strictEqual(result.violations[0].agent, 'gsd-orphan-agent');
|
|
});
|
|
|
|
// # 11 — the legal pinned pair: model + matching reasoning effort is intentional.
|
|
test('row 11: model + model_reasoning_effort, model legal — no violation', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-pinned-agent',
|
|
'name = "gsd-pinned-agent"\nmodel = "gpt-5-codex"\nmodel_reasoning_effort = "high"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 12 — NEGATIVE PROOF. Catches: over-generalizing the #838 model/effort
|
|
// coupling rule to service_tier/model_verbosity too. Those are #774's
|
|
// cost/verbosity knobs and are decoupled from `model` by design — an
|
|
// implementation that treats "any knob present without model" as orphaned
|
|
// fails this row.
|
|
test('row 12 (negative proof): service_tier + model_verbosity, no model — no violation', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-light-agent',
|
|
'name = "gsd-light-agent"\nservice_tier = "flex"\nmodel_verbosity = "low"\ndeveloper_instructions = \'\'\'\nWork fast.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 13 — NEGATIVE PROOF. Catches: implementing the check as a whitelist over the
|
|
// whole TOML document (flagging any key GSD doesn't itself emit) instead of a
|
|
// predicate on exactly the two fields the posture owns (`model`,
|
|
// `model_reasoning_effort`). A hand-added `approval_policy` is legitimate and
|
|
// must never be flagged.
|
|
test('row 13 (negative proof): extra hand-added key (approval_policy) — no violation', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-custom-agent',
|
|
'name = "gsd-custom-agent"\napproval_policy = "on-request"\ndeveloper_instructions = \'\'\'\nFollow policy.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 14 — NEGATIVE PROOF, the headline trap. Catches: scanning the WHOLE file with
|
|
// a line-oriented `/^model\s*=/m` instead of only the header slice (the lines
|
|
// before `developer_instructions = '''`). The fixture below contains a literal,
|
|
// unindented `model = "sonnet"` line INSIDE the developer_instructions block —
|
|
// verified (see PR description / dispatch notes) to trip a naive whole-file
|
|
// regex scan (`/^model\s*=/m.test(wholeFile) === true`) while the correct
|
|
// header-slice scan sees nothing (`/^model\s*=/m.test(headerOnly) === false`).
|
|
// If this test passed against a whole-file scanner it would prove nothing; it
|
|
// must fail against one.
|
|
test('row 14 (negative proof, headline): model = inside developer_instructions block — no violation', () => {
|
|
copyFixtureToml(agentsDir, 'gsd-planner', 'model-in-developer-instructions.toml');
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 15 — NEGATIVE PROOF. Catches: a header-slice scan that doesn't skip comment
|
|
// lines, so a commented-out `# model = "sonnet"` still counts as a live pin.
|
|
test('row 15 (negative proof): commented-out model pin — no violation', () => {
|
|
copyFixtureToml(agentsDir, 'gsd-reviewer', 'commented-model-pin.toml');
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 16 — NEGATIVE PROOF. Catches: probing for the key by substring/prefix
|
|
// (e.g. a bare `/model/` test) instead of anchoring on the full key name, so
|
|
// `model_verbosity` gets misidentified as a `model` pin.
|
|
test('row 16 (negative proof): model_verbosity only (key-prefix collision) — no violation', () => {
|
|
copyFixtureToml(agentsDir, 'gsd-analyst', 'model-verbosity-prefix-collision.toml');
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 17 — boundary: whitespace around a REAL key must still be recognized as a pin.
|
|
test('row 17: indented / inner-spaced model pin — IS a violation', () => {
|
|
copyFixtureToml(agentsDir, 'gsd-tester', 'whitespace-indented-model-pin.toml');
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
});
|
|
|
|
// # 18 — cross-platform: CRLF and BOM must parse identically to plain LF.
|
|
test('row 18a: CRLF file with a pinned model — parsed identically to LF (still a violation)', () => {
|
|
const lfContent =
|
|
'name = "gsd-scribe"\ndescription = "Writes changelog entries from merged PRs"\n' +
|
|
'model = "sonnet"\ndeveloper_instructions = \'\'\'\n' +
|
|
'Write a changelog entry summarizing the merged pull request.\n\'\'\'\n';
|
|
writeAgentToml(agentsDir, 'gsd-scribe', toCrlf(lfContent));
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
});
|
|
|
|
test('row 18b: BOM-prefixed file with a pinned model — parsed identically to a BOM-free file', () => {
|
|
copyFixtureToml(agentsDir, 'gsd-archivist', 'bom-pinned-model.toml');
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
});
|
|
|
|
// # 19 — independence: multiple agents, some violating; deterministic order;
|
|
// clean agents are simply absent from violations (not present-but-empty).
|
|
test('row 19: several agents, some violating — one entry per offender, deterministic order, clean absent', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-alpha',
|
|
'name = "gsd-alpha"\ndeveloper_instructions = \'\'\'\nClean.\n\'\'\'\n',
|
|
);
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-bravo',
|
|
'name = "gsd-bravo"\nmodel = "opus"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-charlie',
|
|
'name = "gsd-charlie"\nmodel_reasoning_effort = "medium"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.strictEqual(result.violations.length, 2);
|
|
assert.strictEqual(result.violations[0].agent, 'gsd-bravo');
|
|
assert.strictEqual(result.violations[1].agent, 'gsd-charlie');
|
|
assert.ok(
|
|
!result.violations.some((v) => v.agent === 'gsd-alpha'),
|
|
'clean agent must not appear in violations at all',
|
|
);
|
|
assert.strictEqual(result.checked.length, 3);
|
|
});
|
|
|
|
// ─── Reviewer-found false negatives (quoted keys; block-marker truncation) ──
|
|
//
|
|
// Both defects are false negatives — checkCodexModelPosture reported ok:true
|
|
// when a real pin was present. Each test below is red against the
|
|
// implementation this PR replaces; see the PR description for exactly which
|
|
// assertion fails against each.
|
|
|
|
// Defect 1: a quoted TOML key (`"model" = ...`) is legal TOML and was invisible
|
|
// to a key regex that required a bare identifier.
|
|
test('quoted key "model" = "sonnet" (double-quoted) — still ANTHROPIC_FLAVORED_MODEL', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-quoted-double',
|
|
'name = "gsd-quoted-double"\n"model" = "sonnet"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
});
|
|
|
|
test("quoted key 'model' = \"sonnet\" (single-quoted) — still ANTHROPIC_FLAVORED_MODEL", () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-quoted-single',
|
|
'name = "gsd-quoted-single"\n\'model\' = "sonnet"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
});
|
|
|
|
// Defect 2a: the block marker used to be found by an unanchored whole-content
|
|
// search, so a `description` value that merely quotes the marker text earlier
|
|
// in the file truncated the header before a real, later `model` pin.
|
|
test('a description value quoting the marker text does not hide a real model pin before it', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-decoy-marker',
|
|
'name = "gsd-decoy-marker"\n' +
|
|
'description = "mentions developer_instructions = \'\'\' as an example string"\n' +
|
|
'model = "sonnet"\n' +
|
|
'developer_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
});
|
|
|
|
// Defect 2b: the old implementation only ever scanned the slice BEFORE the
|
|
// marker, so a hand-reordered file with `model` placed AFTER the
|
|
// developer_instructions block (still legal TOML) was never scanned at all.
|
|
test('a model pin placed AFTER the developer_instructions block is still flagged', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-reordered',
|
|
'name = "gsd-reordered"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\nmodel = "sonnet"\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
});
|
|
|
|
// Defect-2 boundary: no developer_instructions block at all — every line must
|
|
// be scanned. NOTE: a bare-key version of this fixture is already green
|
|
// against the pre-fix implementation (its no-marker fallback already scanned
|
|
// the whole file), so this uses a quoted key to keep the assertion genuinely
|
|
// red pre-fix (via defect 1) while proving the no-block path is fully scanned.
|
|
test('a file with no developer_instructions block at all is fully scanned', () => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-noblock',
|
|
'name = "gsd-noblock"\ndescription = "no prompt block on this agent"\n"model" = "sonnet"\n',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
});
|
|
|
|
// # 21 — filesystem failure: an unreadable .toml is reported, not thrown, and
|
|
// does not abort checking the rest of the agents. Injected via fs.readFileSync
|
|
// monkeypatch/restore (see withInjectedReadFailure) rather than chmod 0o000,
|
|
// which root bypasses under Docker/CI.
|
|
test('row 21: unreadable .toml (EACCES) — UNREADABLE violation naming the file, other agents still checked, no throw', () => {
|
|
writeAgentToml(agentsDir, 'gsd-bad', 'name = "gsd-bad"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n');
|
|
writeAgentToml(agentsDir, 'gsd-good', 'name = "gsd-good"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n');
|
|
const badPath = path.join(agentsDir, 'gsd-bad.toml');
|
|
const injected = Object.assign(new Error('injected EACCES'), { code: 'EACCES' });
|
|
|
|
const result = withInjectedReadFailure(badPath, injected, () =>
|
|
agentInstallCheck.checkCodexModelPosture('codex', tmpDir),
|
|
);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
const unreadable = result.violations.find((v) => v.agent === 'gsd-bad');
|
|
assert.ok(unreadable, 'unreadable file must produce a named violation, not a silent skip');
|
|
assert.strictEqual(unreadable.reason, agentInstallCheck.POSTURE_REASON.UNREADABLE);
|
|
assert.ok(unreadable.file.endsWith('gsd-bad.toml'));
|
|
assert.strictEqual(result.checked.length, 2, 'the unreadable file must still be counted as checked');
|
|
assert.ok(
|
|
!result.violations.some((v) => v.agent === 'gsd-good'),
|
|
'the still-readable sibling must be checked and found clean',
|
|
);
|
|
});
|
|
|
|
// Security review (#3242, MEDIUM): readdirSync + readFileSync followed symlinks,
|
|
// so a symlink in the agents directory pointing at an arbitrary file could have
|
|
// that file's content echoed into a violation's `value` field. Fixed by lstat-
|
|
// filtering to regular files only, matching cmdEffortSync's existing symlink
|
|
// guard in commands.cts. Symlinks are silently excluded (not reported), same
|
|
// as cmdEffortSync — see agent-install-check.cts inline comment for why.
|
|
test('symlink pointing at a file containing model = "sonnet" is never read — no violation names that value', (t) => {
|
|
const targetPath = path.join(tmpDir, 'outside-target.toml');
|
|
fs.writeFileSync(targetPath, 'model = "sonnet"\n');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
const symlinkPath = path.join(agentsDir, 'gsd-linked.toml');
|
|
try {
|
|
fs.symlinkSync(targetPath, symlinkPath, 'file');
|
|
} catch (error) {
|
|
if (error && ['EPERM', 'EACCES', 'ENOTSUP'].includes(error.code)) {
|
|
t.skip('symlink creation is not available on this platform');
|
|
return;
|
|
}
|
|
throw error;
|
|
}
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
assert.deepStrictEqual(result.checked, [], 'the symlinked entry must not appear in checked');
|
|
});
|
|
|
|
test('broken symlink in agents dir does not crash the scan — other agents still checked', (t) => {
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
const brokenTarget = path.join(tmpDir, 'does-not-exist.toml');
|
|
const brokenSymlink = path.join(agentsDir, 'gsd-broken.toml');
|
|
try {
|
|
fs.symlinkSync(brokenTarget, brokenSymlink, 'file');
|
|
} catch (error) {
|
|
if (error && ['EPERM', 'EACCES', 'ENOTSUP'].includes(error.code)) {
|
|
t.skip('symlink creation is not available on this platform');
|
|
return;
|
|
}
|
|
throw error;
|
|
}
|
|
writeAgentToml(agentsDir, 'gsd-good', 'name = "gsd-good"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n');
|
|
|
|
let result;
|
|
assert.doesNotThrow(() => {
|
|
result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
});
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
assert.deepStrictEqual(result.checked, ['gsd-good'], 'the broken symlink must be excluded, the other agent still checked');
|
|
});
|
|
|
|
test('a regular .toml file is still scanned normally (guard against over-filtering)', () => {
|
|
writeAgentToml(agentsDir, 'gsd-planner', 'name = "gsd-planner"\nmodel = "sonnet"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n');
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, false);
|
|
assert.deepStrictEqual(result.checked, ['gsd-planner']);
|
|
assert.strictEqual(result.violations.length, 1);
|
|
assert.strictEqual(result.violations[0].reason, agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL);
|
|
assert.strictEqual(result.violations[0].value, 'sonnet');
|
|
});
|
|
|
|
// # 22 — boundary: empty / whitespace-only .toml pins nothing.
|
|
test('row 22: empty / whitespace-only .toml — no violation', () => {
|
|
writeAgentToml(agentsDir, 'gsd-blank', ' \n\n\t\n');
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
});
|
|
|
|
// # 23 — hostile: an oversized (secret-shaped) model value must be truncated at
|
|
// 64 chars, matching bin/install.js's _warnCodexModelOverrideDropped truncation
|
|
// (`slice(0, 64) + '…'`) so an oversized/secret-shaped value cannot reach logs
|
|
// in full.
|
|
test('row 23: oversized model value — truncated at 64 chars, matching the installer cap', () => {
|
|
const oversized = `claude-${'x'.repeat(80)}`; // 87 chars, Anthropic-flavored (contains "claude")
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-oversized-agent',
|
|
`name = "gsd-oversized-agent"\nmodel = "${oversized}"\ndeveloper_instructions = '''\nWork.\n'''\n`,
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('codex', tmpDir);
|
|
|
|
assert.strictEqual(result.violations.length, 1);
|
|
const { value } = result.violations[0];
|
|
assert.ok(value.length <= 65, `expected value capped at 64 chars (+ ellipsis), got length ${value.length}`);
|
|
assert.notStrictEqual(value, oversized, 'the full oversized value must not reach the violation untruncated');
|
|
assert.strictEqual(value, `${oversized.slice(0, 64)}…`);
|
|
});
|
|
|
|
// # 25 — NEGATIVE PROOF. Catches: gating the runtime no-op AFTER already
|
|
// scanning the agents directory (e.g. deciding what to report only at the end),
|
|
// instead of short-circuiting before any file is read. A stray .toml that WOULD
|
|
// trip a violation if inspected must never be inspected for a non-codex runtime.
|
|
test('row 25 (negative proof): non-codex runtime with a stray violating .toml present — still NOT_CODEX, file never inspected', (t) => {
|
|
writeAgentToml(
|
|
agentsDir,
|
|
'gsd-stray',
|
|
'name = "gsd-stray"\nmodel = "sonnet"\ndeveloper_instructions = \'\'\'\nWork.\n\'\'\'\n',
|
|
);
|
|
const reads = [];
|
|
t.mock.method(fs, 'readFileSync', (...args) => {
|
|
reads.push(args[0]);
|
|
throw new Error('unreachable: readFileSync must not be called for a non-codex runtime');
|
|
});
|
|
|
|
const result = agentInstallCheck.checkCodexModelPosture('opencode', tmpDir);
|
|
|
|
assert.strictEqual(result.ok, true);
|
|
assert.deepStrictEqual(result.violations, []);
|
|
assert.strictEqual(result.reason, agentInstallCheck.POSTURE_REASON.NOT_CODEX);
|
|
assert.deepStrictEqual(reads, [], 'the stray violating file must never be read for a non-codex runtime');
|
|
});
|
|
});
|
|
|
|
// # 24 — enum lock: adding a POSTURE_REASON value is a deliberate three-way
|
|
// coordinated change (enum, emitting site, this test), not a silent drift. Locks
|
|
// the exact key set, matching the repo's established shape for reason enums
|
|
// (see verify-reapply-patches.cjs's REASON).
|
|
describe('POSTURE_REASON enum', () => {
|
|
test('row 24: Object.keys(POSTURE_REASON).sort() is locked', () => {
|
|
assert.deepStrictEqual(
|
|
Object.keys(agentInstallCheck.POSTURE_REASON).sort(),
|
|
[
|
|
'AGENTS_DIR_MISSING',
|
|
'ANTHROPIC_FLAVORED_MODEL',
|
|
'NOT_CODEX',
|
|
'ORPHANED_REASONING_EFFORT',
|
|
'UNREADABLE',
|
|
].sort(),
|
|
);
|
|
});
|
|
|
|
test('POSTURE_REASON values are the frozen snake_case wire form, not prose', () => {
|
|
assert.strictEqual(agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL, 'anthropic_flavored_model');
|
|
assert.strictEqual(agentInstallCheck.POSTURE_REASON.ORPHANED_REASONING_EFFORT, 'orphaned_reasoning_effort');
|
|
assert.strictEqual(agentInstallCheck.POSTURE_REASON.UNREADABLE, 'unreadable');
|
|
assert.strictEqual(agentInstallCheck.POSTURE_REASON.NOT_CODEX, 'not_codex');
|
|
assert.strictEqual(agentInstallCheck.POSTURE_REASON.AGENTS_DIR_MISSING, 'agents_dir_missing');
|
|
assert.ok(Object.isFrozen(agentInstallCheck.POSTURE_REASON));
|
|
});
|
|
});
|
|
|
|
// # 20 — keystone wiring: a library that works but is never called from the
|
|
// user-reachable command surface is the keystone-unwired failure the coverage
|
|
// gate exists to catch. Drives cmdValidateAgents directly (the same seam
|
|
// tests/verify.test.cjs already uses for cmdValidateHealth) and asserts through
|
|
// the command's structured JSON output — captured by monkeypatching
|
|
// node:fs.writeSync, the exact seam tests/io.test.cjs already establishes for
|
|
// io.cjs's output(), which writes via writeAllSync(1, ...) → fs.writeSync, NOT
|
|
// console.log — rather than calling checkCodexModelPosture a second time.
|
|
describe('cmdValidateAgents surfaces the Codex posture result (#3242 row 20)', () => {
|
|
let tmpDir;
|
|
let agentsDir;
|
|
|
|
beforeEach(() => {
|
|
tmpDir = createTempDir('gsd-posture-wiring-');
|
|
agentsDir = path.join(tmpDir, 'agents');
|
|
process.env['GSD_AGENTS_DIR'] = agentsDir;
|
|
process.env['GSD_RUNTIME'] = 'codex';
|
|
});
|
|
|
|
afterEach(() => {
|
|
cleanup(tmpDir);
|
|
});
|
|
|
|
test('row 20: validate agents output carries the posture result for a violating install', (t) => {
|
|
const { cmdValidateAgents } = require(
|
|
path.join(__dirname, '..', 'gsd-core', 'bin', 'lib', 'verify.cjs'),
|
|
);
|
|
writeAgentToml(
|
|
agentsDir,
|
|
EXPECTED_AGENTS[0],
|
|
`name = "${EXPECTED_AGENTS[0]}"\nmodel = "sonnet"\ndeveloper_instructions = '''\nWork.\n'''\n`,
|
|
);
|
|
|
|
const written = [];
|
|
const realWriteSync = fs.writeSync;
|
|
t.mock.method(fs, 'writeSync', (fd, data, offset, length) => {
|
|
if (fd !== 1) {
|
|
return realWriteSync.call(fs, fd, data, offset, length);
|
|
}
|
|
const chunk = Buffer.isBuffer(data)
|
|
? data.subarray(offset ?? 0, length === undefined ? data.length : (offset ?? 0) + length).toString('utf8')
|
|
: String(data);
|
|
written.push(chunk);
|
|
return Buffer.byteLength(chunk, 'utf8');
|
|
});
|
|
|
|
cmdValidateAgents(tmpDir, false);
|
|
|
|
const parsed = JSON.parse(written.join(''));
|
|
assert.ok(
|
|
parsed.codex_posture,
|
|
`expected cmdValidateAgents output to carry a codex_posture key, got keys: ${Object.keys(parsed).join(', ')}`,
|
|
);
|
|
assert.strictEqual(parsed.codex_posture.ok, false);
|
|
assert.strictEqual(parsed.codex_posture.violations.length, 1);
|
|
assert.strictEqual(parsed.codex_posture.violations[0].agent, EXPECTED_AGENTS[0]);
|
|
assert.strictEqual(
|
|
parsed.codex_posture.violations[0].reason,
|
|
agentInstallCheck.POSTURE_REASON.ANTHROPIC_FLAVORED_MODEL,
|
|
);
|
|
});
|
|
});
|
|
|
|
// ─── #2872 (ADR-2866 Phase 3): checkAgentsInstalled unchanged by the ──────
|
|
// manifest-reader de-duplication (50-test-matrix.md section 5, rows A1-A5)
|
|
//
|
|
// installer-migrations.cts's readInstallManifest now records manifestVersion
|
|
// /runtime/scope, but checkAgentsInstalled only ever consumed `.files`
|
|
// before AND after the substitution (agent-install-check.cts:191-194
|
|
// swapped an inlined try/readFileSync/JSON.parse for the shared reader,
|
|
// zero new branches) — see 40-design.md's "Rejected" #4 for why this
|
|
// function is not rewired onto the two-scope resolver instead. These rows
|
|
// prove the substitution changed nothing observable here.
|
|
describe('checkAgentsInstalled — unchanged by the manifest-reader substitution (#2872 A1-A5)', () => {
|
|
describe('A1-A4: v1/v2 manifest parity', () => {
|
|
let tmpDir;
|
|
let agentsDir;
|
|
|
|
beforeEach(() => {
|
|
tmpDir = createTempDir('gsd-agent-check-schema-');
|
|
agentsDir = path.join(tmpDir, 'agents');
|
|
process.env['GSD_AGENTS_DIR'] = agentsDir;
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
fs.writeFileSync(path.join(agentsDir, `${agent}.md`), `# ${agent}\n`);
|
|
}
|
|
});
|
|
|
|
afterEach(() => {
|
|
cleanup(tmpDir);
|
|
});
|
|
|
|
// A1 — a v1 manifest (no manifestVersion/runtime/scope key at all)
|
|
// beside the agents dir: result identical to pre-#2872 behavior.
|
|
test('A1: checkAgentsInstalled is unchanged for a v1 manifest', () => {
|
|
const manifestFiles = {};
|
|
for (const agent of EXPECTED_AGENTS) manifestFiles[`agents/${agent}.md`] = 'somehash';
|
|
fs.writeFileSync(
|
|
path.join(tmpDir, 'gsd-file-manifest.json'),
|
|
JSON.stringify({
|
|
version: '1.49.0',
|
|
timestamp: '2026-05-10T00:00:00.000Z',
|
|
mode: 'full',
|
|
files: manifestFiles,
|
|
}),
|
|
);
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled();
|
|
assert.strictEqual(result.agents_installed, true);
|
|
assert.deepStrictEqual(result.missing_agents, []);
|
|
assert.deepStrictEqual(result.incomplete_agents, []);
|
|
});
|
|
|
|
// A2 — a v2 manifest (carrying manifestVersion/runtime/scope) produces
|
|
// the exact same result: the new fields change nothing here.
|
|
test('A2: checkAgentsInstalled is unchanged for a v2 manifest', () => {
|
|
const manifestFiles = {};
|
|
for (const agent of EXPECTED_AGENTS) manifestFiles[`agents/${agent}.md`] = 'somehash';
|
|
fs.writeFileSync(
|
|
path.join(tmpDir, 'gsd-file-manifest.json'),
|
|
JSON.stringify({
|
|
manifestVersion: 2,
|
|
version: '1.60.0',
|
|
timestamp: '2026-08-01T00:00:00.000Z',
|
|
mode: 'full',
|
|
runtime: 'claude',
|
|
scope: 'global',
|
|
files: manifestFiles,
|
|
}),
|
|
);
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled();
|
|
assert.strictEqual(result.agents_installed, true);
|
|
assert.deepStrictEqual(result.missing_agents, []);
|
|
assert.deepStrictEqual(result.incomplete_agents, []);
|
|
});
|
|
|
|
// A3 — no manifest at all (readInstallManifest's absent-file branch):
|
|
// the completeness check is skipped, no throw — today's behavior.
|
|
// Presence (the .md files above) still makes the install look complete.
|
|
test('A3: an unreadable/absent manifest still skips the completeness check', () => {
|
|
let result;
|
|
assert.doesNotThrow(() => {
|
|
result = agentInstallCheck.checkAgentsInstalled();
|
|
});
|
|
assert.deepStrictEqual(result.incomplete_agents, []);
|
|
assert.strictEqual(result.agents_installed, true);
|
|
});
|
|
|
|
// A4 — manifest present, one agent's .toml tracked but absent on disk:
|
|
// still reported as incomplete, same as before the substitution.
|
|
test('A4: still reports an incomplete agent', () => {
|
|
const targetAgent = EXPECTED_AGENTS[0];
|
|
const manifestFiles = {};
|
|
for (const agent of EXPECTED_AGENTS) manifestFiles[`agents/${agent}.md`] = 'somehash';
|
|
manifestFiles[`agents/${targetAgent}.toml`] = 'somehash';
|
|
fs.writeFileSync(
|
|
path.join(tmpDir, 'gsd-file-manifest.json'),
|
|
JSON.stringify({
|
|
manifestVersion: 2,
|
|
version: '1.60.0',
|
|
timestamp: '2026-08-01T00:00:00.000Z',
|
|
mode: 'full',
|
|
runtime: 'claude',
|
|
scope: 'global',
|
|
files: manifestFiles,
|
|
}),
|
|
);
|
|
|
|
const result = agentInstallCheck.checkAgentsInstalled();
|
|
assert.ok(
|
|
result.incomplete_agents.includes(targetAgent),
|
|
`expected ${targetAgent} in incomplete_agents, got: ${JSON.stringify(result.incomplete_agents)}`,
|
|
);
|
|
assert.strictEqual(result.agents_installed, false);
|
|
});
|
|
});
|
|
|
|
// A5 — getAgentsDir's local-install probe is deliberately NOT rewired to
|
|
// readInstallManifest: it stays an lstat-based (symlink-unaware) presence
|
|
// check (agent-install-check.cts:123, "Not touched" in 40-design.md's
|
|
// blast-radius table), so a manifest path that is itself a SYMLINK is
|
|
// still ignored, exactly as before the substitution.
|
|
test('A5: getAgentsDir still ignores a symlinked manifest', (t) => {
|
|
const projectRoot = createTempDir('gsd-local-codex-schema-');
|
|
const globalHome = createTempDir('gsd-global-codex-schema-');
|
|
const localConfigDir = path.join(projectRoot, '.codex');
|
|
const localAgentsDir = path.join(localConfigDir, 'agents');
|
|
const realManifestTarget = path.join(projectRoot, 'real-manifest.json');
|
|
t.after(() => cleanup(projectRoot));
|
|
t.after(() => cleanup(globalHome));
|
|
fs.mkdirSync(localAgentsDir, { recursive: true });
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
fs.writeFileSync(path.join(localAgentsDir, `${agent}.toml`), `name = "${agent}"\n`);
|
|
}
|
|
fs.writeFileSync(realManifestTarget, JSON.stringify({ files: {} }));
|
|
const manifestPath = path.join(localConfigDir, 'gsd-file-manifest.json');
|
|
createCompleteAgents(path.join(globalHome, 'agents'));
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
try {
|
|
fs.symlinkSync(realManifestTarget, manifestPath, 'file');
|
|
} catch (error) {
|
|
if (error && ['EPERM', 'EACCES', 'ENOTSUP'].includes(error.code)) {
|
|
t.skip('symlink creation is not available on this platform');
|
|
return;
|
|
}
|
|
throw error;
|
|
}
|
|
|
|
// getAgentsDir's own probe (`fs.lstatSync(manifestPath).isFile()`) sees
|
|
// the manifest path as a symlink, not a regular file — isFile() is false
|
|
// for the link itself even though its target is a regular file — so the
|
|
// local install is NOT selected; the global fallback wins instead.
|
|
assert.strictEqual(agentInstallCheck.getAgentsDir('codex', projectRoot), path.join(globalHome, 'agents'));
|
|
assert.notStrictEqual(agentInstallCheck.getAgentsDir('codex', projectRoot), localAgentsDir);
|
|
});
|
|
});
|
|
|
|
// ─── 4. Sandbox-mode drift (#3897 rung 3, ADR-3473 §8.3 criterion 3) ──────────
|
|
//
|
|
// Spec: .gsd/phase/feat-3897-adr3473-83-rungs/{40-design,50-test-matrix}.md, S8/T28.
|
|
//
|
|
// `checkAgentsInstalled` above (and `checkCodexModelPosture`, the established
|
|
// sibling pattern for a codex-only posture check — #3242, "A new sibling
|
|
// export... presence is checkAgentsInstalled's job; this function's job starts
|
|
// only once the runtime is confirmed codex and only inspects posture") checks
|
|
// FILE PRESENCE and MANIFEST COMPLETENESS only. Neither inspects whether an
|
|
// installed .toml's `sandbox_mode` line agrees with what that role's tool
|
|
// contract says it should be — a TOML that disagrees passes `validate agents`
|
|
// today. `checkCodexSandboxPosture` is this test's anticipated name for the new
|
|
// sibling check (mirroring `checkCodexModelPosture`'s exact shape); the exact
|
|
// export name is the implementer's call, but the export MUST exist for this
|
|
// row's acceptance criterion (§8.3 criterion 3) to be met.
|
|
//
|
|
// RED today: the export does not exist at all.
|
|
describe('checkCodexSandboxPosture (#3897 rung 3 — not yet implemented, RED until it lands)', () => {
|
|
test('T28 validateAgentsFailsOnSandboxDrift_3897: a TOML whose sandbox_mode disagrees with the derived expectation must fail, naming role/expected/found', (t) => {
|
|
const globalHome = createTempDir('gsd-sandbox-drift-');
|
|
t.after(() => cleanup(globalHome));
|
|
const agentsDir = path.join(globalHome, 'agents');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
|
|
// gsd-executor declares tools: Read, Write, Edit (agents/gsd-executor.md) —
|
|
// it MUST derive workspace-write. Install a drifted TOML claiming
|
|
// read-only instead, matching the shape generateCodexAgentToml emits.
|
|
fs.writeFileSync(
|
|
path.join(agentsDir, 'gsd-executor.toml'),
|
|
'name = "gsd-executor"\ndescription = "Executes plans"\nsandbox_mode = "read-only"\n' +
|
|
"developer_instructions = '''\nExecute.\n'''\n",
|
|
);
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
if (agent === 'gsd-executor') continue;
|
|
fs.writeFileSync(path.join(agentsDir, `${agent}.toml`), `name = "${agent}"\n`);
|
|
}
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
|
|
assert.equal(
|
|
typeof agentInstallCheck.checkCodexSandboxPosture,
|
|
'function',
|
|
'agent-install-check.cjs must export a sandbox-posture check (#3897 rung 3, ADR-3473 §8.3 criterion 3) — it does not exist yet, so a drifted sandbox_mode currently passes validate agents silently',
|
|
);
|
|
|
|
const result = agentInstallCheck.checkCodexSandboxPosture('codex');
|
|
assert.equal(
|
|
result.ok,
|
|
false,
|
|
'a TOML whose sandbox_mode disagrees with the expected derived value must fail the check',
|
|
);
|
|
const violation = result.violations.find((v) => v.agent === 'gsd-executor');
|
|
assert.ok(violation, 'the violation must name the drifted agent');
|
|
assert.equal(violation.expected, 'workspace-write', 'the violation must name the EXPECTED sandbox_mode');
|
|
assert.equal(violation.found, 'read-only', 'the violation must name the FOUND (installed) sandbox_mode');
|
|
});
|
|
|
|
// #3897 rung 4 (isolated correctness review, MINOR finding 2): `found` used
|
|
// to be read via a naive whole-file regex, unlike the block-aware scanner
|
|
// the sibling checkCodexModelPosture uses. Reviewer's fixture: no header
|
|
// sandbox_mode pin at all, but a `sandbox_mode = "..."`-shaped line INSIDE
|
|
// the developer_instructions block (prose a role's own prompt might
|
|
// legitimately discuss). The naive regex misread that prose as a live
|
|
// value and manufactured a FALSE violation.
|
|
test('a sandbox_mode-shaped line INSIDE developer_instructions is never read as a live value (no false violation)', (t) => {
|
|
const globalHome = createTempDir('gsd-sandbox-block-aware-');
|
|
t.after(() => cleanup(globalHome));
|
|
const agentsDir = path.join(globalHome, 'agents');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
|
|
// gsd-nyquist-auditor's real canonical tools: declares Write/Edit, so the
|
|
// derived expectation is workspace-write. Install a TOML that correctly
|
|
// has NO header sandbox_mode pin, but whose developer_instructions block
|
|
// contains a line that is shaped exactly like a live sandbox_mode pin.
|
|
fs.writeFileSync(
|
|
path.join(agentsDir, 'gsd-nyquist-auditor.toml'),
|
|
'name = "gsd-nyquist-auditor"\ndescription = "Fills Nyquist validation gaps"\n' +
|
|
"developer_instructions = '''\n" +
|
|
'When configuring the sandbox, use:\n' +
|
|
'sandbox_mode = "workspace-write"\n' +
|
|
"'''\n",
|
|
);
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
if (agent === 'gsd-nyquist-auditor') continue;
|
|
fs.writeFileSync(path.join(agentsDir, `${agent}.toml`), `name = "${agent}"\n`);
|
|
}
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
|
|
const result = agentInstallCheck.checkCodexSandboxPosture('codex');
|
|
const violation = result.violations.find((v) => v.agent === 'gsd-nyquist-auditor');
|
|
assert.equal(
|
|
violation,
|
|
undefined,
|
|
'a sandbox_mode-shaped line inside developer_instructions must never be read as the installed value — it must not manufacture a violation for an agent with no real header pin',
|
|
);
|
|
});
|
|
|
|
// #3897 rung 4 (isolated correctness review, MINOR finding 3):
|
|
// truncatePostureValue exists at agent-install-check.cts precisely for
|
|
// this, and the model-posture sibling applies it — the sandbox check did
|
|
// not, so an oversized (or secret-shaped) sandbox_mode value could reach
|
|
// `validate agents --raw` output at full length.
|
|
test('an oversized sandbox_mode value is truncated in the emitted violation (CLI JSON)', (t) => {
|
|
const globalHome = createTempDir('gsd-sandbox-truncate-');
|
|
t.after(() => cleanup(globalHome));
|
|
const agentsDir = path.join(globalHome, 'agents');
|
|
fs.mkdirSync(agentsDir, { recursive: true });
|
|
|
|
const oversized = 'x'.repeat(300);
|
|
fs.writeFileSync(
|
|
path.join(agentsDir, 'gsd-executor.toml'),
|
|
`name = "gsd-executor"\ndescription = "Executes plans"\nsandbox_mode = "${oversized}"\n` +
|
|
"developer_instructions = '''\nExecute.\n'''\n",
|
|
);
|
|
for (const agent of EXPECTED_AGENTS) {
|
|
if (agent === 'gsd-executor') continue;
|
|
fs.writeFileSync(path.join(agentsDir, `${agent}.toml`), `name = "${agent}"\n`);
|
|
}
|
|
process.env['CODEX_HOME'] = globalHome;
|
|
|
|
const result = agentInstallCheck.checkCodexSandboxPosture('codex');
|
|
const violation = result.violations.find((v) => v.agent === 'gsd-executor');
|
|
assert.ok(violation, 'the violation must name the drifted agent');
|
|
assert.ok(
|
|
violation.found.length <= 65,
|
|
`violation.found must be truncated (<=64 chars + ellipsis), got length ${violation.found.length}`,
|
|
);
|
|
assert.ok(violation.found.endsWith('…'), 'a truncated value must end with the ellipsis marker');
|
|
|
|
// The full-length value must never reach the emitted JSON at all.
|
|
const serialized = JSON.stringify(result);
|
|
assert.ok(!serialized.includes(oversized), 'the untruncated 300-char value must never appear in the emitted CLI JSON');
|
|
});
|
|
});
|
|
|
|
// T29 (validate agents still fails when a file is missing, S9) is intentionally
|
|
// NOT duplicated here — it is already covered by the existing "missing dir"
|
|
// coverage documented in this file's own header comment ("missing dir →
|
|
// agents_installed:false, missing_agents = all expected") and exercised above.
|
|
// It must stay green; nothing in this rung touches that path.
|