Files
msd-core/tests/execute-phase-active-flags.test.cjs
Tom Boucher 15af0f5536 enhance(#3951): B6+B7 — widen two unreachable lint rules and make the guard ledger true (#3965)
* fix(#3951): two lint rules that could not reach the code they govern

B6 names two widenings. Measuring them first turned up a defect the criterion did
not know about, and refuted the reason it gave for one of them.

1. no-adhoc-markdown-parsing self-gates on its own filename.

   Lines 107-110 short-circuit create() to {} unless the path matches
   /(?:^|\/)src\/[^/]+\.cts$/. B6 says to widen the files: glob in
   eslint.config.mjs - but doing only that ships an INERT rule, because the gate
   still returns {} for every new path. Both halves have to change, and the gate
   is the load-bearing one.

   That same regex hides a live hole: [^/]+ is FLAT-ONLY, so it requires the file
   to sit directly in src/. The registered glob is src/**/*.cts, which includes
   subdirectories. 28 .cts files - health-diagnostic-rules/ (10),
   installer-migrations/ (11), observability/ (3), host-integration-adapters/ (2),
   vendor/ (2) - are inside the registered glob and silently skipped.

   Measured with the gate neutralized: 0 violations there today. The hole is
   hiding nothing right now, and is fixed anyway, because "no violations today" is
   not a property that keeps holding.

   The fix is not invented: require-subprocess-timeout.cjs:196 already carries the
   correct form of this guard, /(?:^|\/)src\/.*\.cts$/ with .*, one directory over.
   Checked the other 21 rules for the same bug - no-adhoc-regex-escape and
   no-private-binary-resolution short-circuit only to exempt their own seam file,
   which is the right shape, and no-crlf-fragile-split has no filename gate at
   all. This bug is unique to the one rule.

2. no-adhoc-regex-escape could not see the shape that actually occurs.

   Line 396 gated the whole UNSAFE-NEW-REGEXP arm on arg.type === 'Identifier'.
   Every check below it - the _SOURCE provenance check, the
   isSoleReturnOfOwnParameter shape - lives inside that branch, so
   new RegExp(obj['key']) and new RegExp(cfg.pattern) were never examined at all.
   Runtime data arrives as a property access far more often than as a bare
   identifier, which is exactly why this rule never fired on the #3477 ReDoS.

   Widened to MemberExpression, measured by AST walk across all five registered
   blocks rather than by grep. 27 sites, zero TSAsExpression:

     18  safe new RegExp(X.source, flags)  -> exempted, keyed strictly on the
         PROPERTY being `source`, never on the object. Keying on the object would
         wave through X.anything and buy nothing. B6 estimated ~10; that was an
         undercount.
      3  _SOURCE-suffixed constants reached through a required module namespace
         (phaseId.BRACKET_PHASE_TOKEN_SOURCE) -> the same provenance-exempt class
         the rule already recognizes for bare identifiers, extended to reach them.
         Without this the widening produces 3 false flags.
      6  real findings -> marked, each a test extracting a pattern from a shipped
         file at test time, where the runtime contract IS the product.

   Deliberately the NARROW MemberExpression form. The rule's own
   isSoleReturnOfOwnParameter doc comment records that an earlier broad
   "any non-literal identifier" heuristic produced ~25 false positives and was
   rejected; a re-run of the census after this change flags exactly the 6 above
   and nothing else.

Verified by execution, not by reading: the gate now accepts src/<subdir>/x.cts,
still accepts flat src/x.cts, and still exempts paths outside src/ - each pinned
by a test proven to fail against the old regex. build:lib, lint and lint:ci all
exit 0.

Refs #3951

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#3951): give no-adhoc-markdown-parsing its reach, and fix the 80 parses it finds

The rule self-gates on filename AND is registered on one glob, so widening either
half alone is inert. Both move here: the gate now accepts tests/**/*.cjs and
scripts/**/*.cjs alongside src/**/*.cts, and eslint.config.mjs registers it on the
same two.

A test pins that the gate and the registration AGREE, in both directions. The
original defect was a gate narrower than its registration; the failure mode of
this fix is a gate wider than its registration. Both are silent, so the test
asserts the pair rather than either half.

80 violations across 43 files, all in tests/, zero in scripts/. 70 are routed
through the existing seams - scanFencedBlocks, collectSection, stripFencedCode,
tokenizeHeadings from markdown-sectionizer; splitTableRow, parseMarkdownTable,
findTableWithColumns from markdown-table. Headerless STATE.md tables use
splitTableRow per line, because parseMarkdownTable needs a real delimiter row.

10 are suppressed, 12.5%, well under the third that would have meant the rule is
mis-scoped for tests/ rather than the tests carrying debt. Each names its reason:
three regression guards (#3873 / bug-#21) are deliberately independent of the
generator's own fence handling, and routing them through the seam would have them
test the generator against itself; one is a negative-text probe that extracts
nothing; six are a shell-pipe-to-jq detector whose regex coincidentally matches the
table fingerprint and is not markdown parsing at all.

All ten sit in tests whose subject is .md content, which is normally a reason to
prefer the seam. The marker used is allow-adhoc-markdown, distinct from
no-source-grep's allow-test-rule, and lint:ci's lint-allow-test-rule-refs reports
the same 280/280 unverified count as before - checked rather than assumed, because
those two markers are easy to conflate.

The widening earned its keep immediately: it found a test that passed for the
wrong reason.

  tests/config-field-docs.test.cjs asserted notEqual(<cell>, '600') against the
  TYPE column instead of the DEFAULT column. notEqual('number', '600') is true
  forever, so the guard against workflow.subagent_timeout regressing to the old
  seconds default could never fire. docs/CONFIGURATION.md:434 is
  `| workflow.subagent_timeout | number | 300000 | ... |`, so the default is cell
  index 2; the assertion is now row-scoped through splitTableRow and reads 300000.

That is the argument for the widening in one case: the violation was invisible to
lint, the suite was green, and the assertion was vacuous. A rule that cannot reach
a file cannot tell you the file is lying.

Not fixed here, and recorded rather than assumed: #3426/#3239 are NOT reachable by
this widening. tests/package-legitimacy-gate.test.cjs yields zero violations even
with the gate bypassed - its hand-rolled scans are real, but built from line
filters and split('|') rather than the regex-literal fingerprints this rule
detects. They need new detectors. The epic assumed a wider glob would catch them.

build:lib, lint and lint:ci all exit 0; the post-fix census across tests/** and
scripts/** is 0 violations.

Refs #3951

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* fix(#3951): B7 — and #3356's defects were still live in the code

B7 asks that each closed child be driven fail-first with a behavioral identity
test at the CONSUMER's output. Four of eleven children had no test citing their
issue number. Auditing them by BEHAVIOR rather than by number-grep changed the
answer for three of the four.

#3364 and #2540 — traceability only. Both were implemented by #3941 and their
consumer-output tests exist and were shown failing-first; neither cited its
originating issue, so an audit that greps for the number reports them uncovered.
Tagged the specific asserting test in each file, following the citation form those
files already use.

#3372 — covered, but only at helper level, and the triage narrowed it. Of the four
commands the issue names, only estimate-cli's collectCalibrationSamples actually
enumerates phase dirs from disk; smart-entry, audit and roadmap-upgrade derive from
ROADMAP/body text and never reach the sentinel path, so they are benign by
construction and were left alone rather than "fixed" into churn. The existing #3882
rows asserted the helper's return value. Added a consumer-output test driving
`query estimate-calibrate` and asserting sample_count and the persisted document.
RED proof: reverted collectCalibrationSamples to a raw readdirSync and ran the real
CLI - sample_count 3, sentinel leaked; restored - sample_count 2.

#3356 — NOT covered, and BOTH halves of the defect were still live in source. The
issue is closed; the bug was not fixed. Fixed here rather than writing tests that
document a bug as correct.

  Defect 1, the contradicted row. quick.md:627 claimed
  `quick-tasks-append` performs "the equivalent write" to the Step 7c row. It did
  not: the `#` cell was a positional ordinal and `Directory` read `—`, because the
  route had no way to receive a quick id or task directory. Added OPTIONAL
  `--quick-id` / `--slug` / `--directory`. A caller with neither - fast.md, the
  original #2133 caller - omits them and gets the byte-identical prior row, so
  nothing existing changes. A caller that HAS a real id and directory now gets the
  canonical row quick.md:632 renders. The false-equivalence sentence itself is
  corrected rather than left to mislead the next reader.

  Defect 2, the forced re-derive. The route called readModifyWriteStateMd with no
  options, so a body-only append to the Quick Tasks table triggered a full
  re-derive of the disk-derived progress.* frontmatter. Every other body-only
  writer passes { resync: false } - src/state.cts's own docstring prescribes it -
  and this route was the lone outlier. RED proof: reverted the option, seeded a
  project with 2 real phase dirs and a curated total_phases of 25, ran
  quick-tasks-append; total_phases collapsed to 2. Restored; it stayed 25.

That second one is the shape this epic exists to close: a silent write that
replaces curated state with a re-derivation nobody asked for, exit 0 throughout.

build:lib, lint and lint:ci all exit 0.

Refs #3951

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* docs(#3951): amend B6's ledger to what was measured, and document the new flags

The ADR gains a ledger amendment in its own correction style - the sixth wrong
premise it records, found the same way as the other five, by measuring before
building.

B6 says the net guard count must fall. It rose: 62 -> 69, +7, measured from the
epic's filing commit to origin/next. The attribution is the point, though. Five of
the seven came from PRs unrelated to this epic, one was added by a phase of it, and
the epic did retire something sub-file - #3884 removed a detector with an explicit
"net: -1 detector, 0 added" ledger. Every named casualty is load-bearing, two
already carry retractions in this same document, and a sweep of all 22 rules plus
every scripts/lint-* found no provably dead guard. There is no honest way to make
the count fall; forcing it would trade coverage for a number, which is the Goodhart
outcome Decision 6 exists to prevent.

The amendment also records that B6's own prescribed fix for one widening was inert.
no-adhoc-markdown-parsing self-gates on its filename, so widening only the files:
glob - which is what the criterion says to do - ships a rule that still returns {}
for every new path. And #3426/#3239 are not reachable by that widening at all;
their scans use line filters and split('|'), not the regex fingerprints the rule
detects. The roster row tracked them against the wrong mechanism.

Three roster rows updated from aspiration to fact: the two widenings are DONE with
their measured counts, and lint-phase-enumeration-drift is marked RETAINED rather
than "expected casualty - verify before retiring", because Phase 5 verified it and
kept it.

The rule Decision 6 should carry forward is stated plainly: a guard ledger is a
claim about COVERAGE, not about COUNT. "Net count must fall" is measurable and
wrong. "Every guard is reachable, and each retirement names what makes its defect
unrepresentable" is the property that was actually wanted.

CLI-TOOLS.md documents the optional --quick-id/--slug/--directory flags and says
plainly that omitting them keeps the pre-#3356 row byte-identical, plus that the
append no longer re-derives progress frontmatter.

New features fragment (id 3951); FEATURES.md regenerated rather than hand-edited.
Changeset is Changed, pr:0 pending backfill.

Refs #3951

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(#3951): correct four rows that pinned the lint rule's old narrow reach

The remote suite came back RED with 5 failures, all in tests/eslint-rules.test.cjs.
They are stale tests, not a regression: four rows assert that
no-adhoc-markdown-parsing is inert outside src/*.cts, which is exactly the
contract this deliverable changes.

Confirmed by reading rather than inferred from the names - the row at :1981 used
filename: 'tests/some.test.cjs' and filename: 'scripts/helper.cjs', the two roots
the rule now covers on purpose.

Worth recording WHY local gates missed this. npm run lint and lint:ci were green,
and the touched test files passed standalone. Lint only reports violations in real
files; these rows assert the rule's REACH using synthetic RuleTester filenames, so
nothing but the full suite could see them. Local green on a rule change says
nothing about the rule's own tests.

Each row is rewritten with BOTH halves rather than flipped from valid to invalid:

  - the same fingerprint under tests/ or scripts/ is now flagged, with the right
    messageId
  - the negative space is preserved - the same fingerprint under a path outside
    all three roots (gsd-core/bin/lib/foo.cjs) is still NOT flagged

The second half is the one that matters. Without it the rule has no boundary and
nothing would catch an over-wide gate later, which is the mirror image of the bug
this deliverable just fixed.

Each row is renamed to state the current contract; the old names said
"non-src/*.cts ... is not flagged" and would have been actively misleading once
the bodies changed.

Proven to test the widening rather than restate it: every flagged half was run
against HEAD~2's pre-widening rule and does NOT fire there, then against the
current rule and does. 12/12 on that probe; the full file is 178/178.

Swept for the same staleness elsewhere and found none.
require-subprocess-timeout's own "inert outside src/*.cts" row is untouched -
that rule's gate was not widened here - and no-adhoc-regex-escape's test file
already carries correctly-targeted rows.

Refs #3951

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(#3951): acknowledge the quick.md growth the attribution guard reported

The full suite came back RED with one failure, and it is mine:

  1 file(s) grew without an acknowledgment:
    quick.md grew 364 bytes

gsd-core/workflows/quick.md is runtime-loaded emitted content, so correcting
its false 'performs the equivalent write' claim trips emitted-attribution by
construction. This is the acknowledgment, not a workaround - there is nothing
to regenerate.

The fragment names ONE path, which is the only one the guard reported. The four
spent acknowledgments it also listed (audit-uat, plan-phase, progress, review)
belong to other fragments whose ripple the base already absorbs; they are inert,
not failures, and are deliberately NOT copied here - naming paths I did not
change would make this record false in the other direction.

Byte figure corrected before committing: the guard reported 37220 -> 37584
(+364), but origin/next has since moved and quick.md is 37232 there now, so the
measured delta is +352. The reason text says so and names the base as a moving
figure rather than pinning a number that is already stale.

Refs #3951

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* test(#3951): move the quick.md growth ack to a trailer, delete the obsolete fragment

The acknowledgment mechanism changed under this branch. Merging next brought in
the redesign - it also deleted .github/workflows/ack-fragment-sweep.yml, which
was in the merge status and which I did not register at the time - and the guard
now says so directly:

  Add a trailer to a commit in this PR (never a new file).
    Emitted-Drift-Ack-Growth: quick.md - <why this growth is deliberate>

So tests/emitted-drift-acks/3951-quick-append-equivalence.json is obsolete on
arrival. A fragment file is no longer read by anything, and leaving it would be a
dead record that looks like an active one. It is deleted here rather than kept
"just in case".

The byte figure moved again with the merge: 37232 -> 37596, +364. The earlier
fragment said +352, measured before the merge auto-merged quick.md itself. The
trailer carries no number, which is the better design - the figure was stale
twice in two attempts.

Refs #3951

Emitted-Drift-Ack-Growth: quick.md — #3356/#3951 replaces a false claim with an accurate one. Line 627 said the `quick-tasks-append` shortcut "performs the equivalent write" to the Step 7c row rendered above it; it did not, and that was the documented half of #3356 — with no quick id or task directory the route emitted a positional ordinal in `#` and an em-dash in `Directory`, a visibly different row. The corrected sentence has to carry three facts the original elided: what the shortcut actually writes when it has neither input, that this is honest behavior for its real caller (`fast.md`, which has neither), and how a caller with both now gets the byte-identical canonical row via the new optional `--quick-id`/`--slug`/`--directory` flags. Prose is the product here — an executing agent reads this line to decide whether the shortcut is safe for its case, and a shorter correction would either drop the flags (leaving the reader unable to act on the fix) or drop the limitation (recreating the false claim in gentler words).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

* chore(#3951): backfill changeset pr number

Refs #3951

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>

---------

Co-authored-by: sim <sim@local>
Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
2026-08-27 23:10:49 -04:00

670 lines
31 KiB
JavaScript

// allow-test-rule: source-text-is-the-product
// Workflow .md / agent .md / command .md / reference .md files — their text
// IS what the runtime loads. Testing text content tests the deployed contract.
// Per CONTRIBUTING.md exception matrix.
/**
* Execute-phase active flag prompt tests
*
* Guards against prompt wording that makes optional flags look active by default.
* This is especially important for weaker runtimes that may infer `--gaps-only`
* from the command docs instead of the literal user arguments.
*/
const { test, describe } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('fs');
const path = require('path');
const { scanFencedBlocks } = require('../gsd-core/bin/lib/markdown-sectionizer.cjs');
const COMMAND_PATH = path.join(__dirname, '..', 'commands', 'gsd', 'execute-phase.md');
describe('execute-phase command: active flags are explicit', () => {
test('command file exists', () => {
assert.ok(fs.existsSync(COMMAND_PATH), 'commands/gsd/execute-phase.md should exist');
});
test('objective says documented flags are not implied active', () => {
const content = fs.readFileSync(COMMAND_PATH, 'utf-8');
// eslint-disable-next-line local/no-unbounded-quantifier -- parses this repo's own command .md content, fixed-size author-controlled content
const objectiveMatch = content.match(/<objective>([\s\S]*?)<\/objective>/);
assert.ok(objectiveMatch, 'should have <objective> section');
assert.ok(
objectiveMatch[1].includes('available behaviors, not implied active behaviors'),
'objective should state that documented flags are not automatically active'
);
assert.ok(
objectiveMatch[1].includes('appears in `$ARGUMENTS`'),
'objective should tie flag activation to literal $ARGUMENTS presence'
);
});
test('context separates available flags from active flags', () => {
const content = fs.readFileSync(COMMAND_PATH, 'utf-8');
assert.ok(
content.includes('Available optional flags (documentation only'),
'context should clearly label flags as documentation only'
);
assert.ok(
content.includes('Active flags must be derived from `$ARGUMENTS`'),
'context should have a separate active-flags section'
);
});
test('context explicitly warns against inferring inactive flags', () => {
const content = fs.readFileSync(COMMAND_PATH, 'utf-8');
assert.ok(
content.includes('Do not infer that a flag is active just because it is documented in this prompt'),
'context should forbid inferring flags from documentation alone'
);
assert.ok(
content.includes('`--interactive` is active only if the literal `--interactive` token is present in `$ARGUMENTS`'),
'context should apply the same active-flag rule to --interactive'
);
assert.ok(
content.includes('If none of these tokens appear, run the standard full-phase execution flow'),
'context should define the no-flags fallback behavior'
);
});
});
// ────────────────────────────────────────────────────────────────────────
// Folded from tests/bug-2396-makefile-test-priority.test.cjs — consolidation epic #1969 (B4 #1973)
// ────────────────────────────────────────────────────────────────────────
{
const { describe: __foldDescribe } = require('node:test');
__foldDescribe("folded:bug-2396-makefile-test-priority (consolidation epic #1969 B4 #1973)", () => {
// allow-test-rule: source-text-is-the-product (see #2396)
// Workflow .md / agent .md / command .md / reference .md files — their text
// IS what the runtime loads. Testing text content tests the deployed contract.
// Per CONTRIBUTING.md exception matrix.
/**
* Regression test for #2396: hardcoded host-level test commands bypass
* container-only project Makefiles.
*
* Fix: execute-phase.md and audit-fix.md must check for
* Makefile with a test target (and other wrappers) before falling through
* to hardcoded language-sniffed commands.
*/
'use strict';
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('fs');
const path = require('path');
const EXECUTE_PHASE_PATH = path.join(__dirname, '..', 'gsd-core', 'workflows', 'execute-phase.md');
const AUDIT_FIX_PATH = path.join(__dirname, '..', 'gsd-core', 'workflows', 'audit-fix.md');
// #1857: execute-phase's regression-gate test-command resolution was extracted
// to this step file (execute-phase.md is size-frozen — phase-6 capstone).
// #2932: steps/regression-gate.md now only discovers prior-phase test files and
// delegates (via "Read and execute") to steps/regression-gate-run.md, which
// carries the actual test-command resolution (Makefile/config-get priority).
const REGRESSION_GATE_PATH = path.join(__dirname, '..', 'gsd-core', 'workflows', 'execute-phase', 'steps', 'regression-gate-run.md');
function assertMakefileCheckBeforeNpmTest(filePath, label) {
const content = fs.readFileSync(filePath, 'utf-8');
// Must check for Makefile with test target
// eslint-disable-next-line local/no-unbounded-quantifier -- parses maintainer-authored workflow markdown, bounded prose, not adversarial input
const hasMakefileCheck = /Makefile.*grep.*test:|grep.*test:.*Makefile/s.test(content) ||
(content.includes('Makefile') && content.includes('"^test:"'));
assert.ok(
hasMakefileCheck,
`${label}: must check for Makefile with test: target before falling through to hardcoded commands`
);
// make test must appear before npm test in the file
const makeTestIdx = content.indexOf('make test');
const npmTestIdx = content.indexOf('npm test');
assert.ok(makeTestIdx !== -1, `${label}: must contain "make test"`);
assert.ok(npmTestIdx !== -1, `${label}: must still contain "npm test" as fallback`);
assert.ok(
makeTestIdx < npmTestIdx,
`${label}: "make test" must appear before "npm test" (Makefile takes priority)`
);
}
function assertConfigGetBeforeMakefile(filePath, label) {
const content = fs.readFileSync(filePath, 'utf-8');
// Must check workflow.test_command config before Makefile sniff.
// Verify within each bash code block: the workflow.test_command lookup
// appears before the Makefile grep in the same block.
assert.ok(
content.includes('workflow.test_command'),
`${label}: must check workflow.test_command config before Makefile/language sniff`
);
// Extract bash blocks to check ordering within each block.
// Use the actual Makefile test ([ -f "Makefile" ]) not just the word "Makefile"
// (which appears in comments before the config-get call).
const lines = content.split(/\r?\n/);
let anyBlockCorrectlyOrdered = false;
for (const fenced of scanFencedBlocks(lines)) {
if (fenced.closeLineIdx === -1) continue;
if ((fenced.infoString || '').trim() !== 'bash') continue;
const block = lines.slice(fenced.openLineIdx + 1, fenced.closeLineIdx).join('\n');
if (block.includes('workflow.test_command') && block.includes('[ -f "Makefile"')) {
const configIdx = block.indexOf('workflow.test_command');
const makefileIdx = block.indexOf('[ -f "Makefile"');
if (configIdx < makefileIdx) {
anyBlockCorrectlyOrdered = true;
break;
}
}
}
assert.ok(
anyBlockCorrectlyOrdered,
`${label}: within a bash block, workflow.test_command config check must appear before Makefile test ([ -f "Makefile" ])`
);
}
describe('bug-2396: Makefile test target must take priority over hardcoded commands', () => {
test('execute-phase.md exists', () => {
assert.ok(fs.existsSync(EXECUTE_PHASE_PATH), 'execute-phase.md should exist');
});
test('audit-fix.md exists', () => {
assert.ok(fs.existsSync(AUDIT_FIX_PATH), 'audit-fix.md should exist');
});
test('regression-gate step: Makefile check precedes npm test (#1857 — extracted from execute-phase.md)', () => {
assertMakefileCheckBeforeNpmTest(REGRESSION_GATE_PATH, 'regression-gate.md');
});
test('audit-fix.md: Makefile check precedes npm test', () => {
assertMakefileCheckBeforeNpmTest(AUDIT_FIX_PATH, 'audit-fix.md');
});
test('regression-gate step: workflow.test_command config checked first (within bash block) (#1857)', () => {
assertConfigGetBeforeMakefile(REGRESSION_GATE_PATH, 'regression-gate.md');
});
test('audit-fix.md: workflow.test_command config checked first (within bash block)', () => {
assertConfigGetBeforeMakefile(AUDIT_FIX_PATH, 'audit-fix.md');
});
});
});
}
// ────────────────────────────────────────────────────────────────────────
// Folded from tests/bug-2516-inherit-model-execute-phase.test.cjs — consolidation epic #1969 (B4 #1973)
// ────────────────────────────────────────────────────────────────────────
{
const { describe: __foldDescribe } = require('node:test');
__foldDescribe("folded:bug-2516-inherit-model-execute-phase (consolidation epic #1969 B4 #1973)", () => {
// allow-test-rule: source-text-is-the-product (see #2516)
// Workflow .md / agent .md / command .md / reference .md files — their text
// IS what the runtime loads. Testing text content tests the deployed contract.
// Per CONTRIBUTING.md exception matrix.
/**
* Regression test for bug #2516
*
* When `.planning/config.json` has `model_profile: "inherit"`, the
* `init.execute-phase` query returns `executor_model: "inherit"`. The
* execute-phase workflow was passing this literal string directly to the
* Task tool via `model="{executor_model}"`, causing Task to fall back to
* its default model instead of inheriting the orchestrator model.
*
* Fix: the workflow must document that when `executor_model` is `"inherit"`,
* the `model=` parameter must be OMITTED from Task() calls entirely.
* Omitting `model=` causes Claude Code to inherit the current orchestrator
* model automatically.
*/
'use strict';
const { test, describe } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('fs');
const path = require('path');
const WORKFLOW_PATH = path.join(
__dirname,
'..',
'gsd-core',
'workflows',
'execute-phase.md'
);
describe('bug #2516: executor_model "inherit" must not be passed literally to Task()', () => {
test('workflow file exists', () => {
assert.ok(fs.existsSync(WORKFLOW_PATH), 'gsd-core/workflows/execute-phase.md should exist');
});
test('workflow contains instructions for handling the "inherit" case', () => {
assert.ok(fs.existsSync(WORKFLOW_PATH), 'gsd-core/workflows/execute-phase.md should exist');
const content = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
const hasInheritInstruction =
content.includes('"inherit"') &&
(content.includes('omit') || content.includes('Omit') || content.includes('omitting') || content.includes('Omitting'));
assert.ok(
hasInheritInstruction,
'execute-phase.md must document that when executor_model is "inherit", ' +
'the model= parameter must be omitted from Task() calls. ' +
'Found "inherit" mention: ' + content.includes('"inherit"') + '. ' +
'Found omit mention: ' + (content.includes('omit') || content.includes('Omit'))
);
});
test('workflow does not instruct passing model="inherit" literally to Task', () => {
assert.ok(fs.existsSync(WORKFLOW_PATH), 'gsd-core/workflows/execute-phase.md should exist');
const content = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
// The workflow must not have an unconditional model="{executor_model}" template
// that would pass "inherit" through. It should document conditional logic.
const hasConditionalModelParam =
content.includes('inherit') &&
(
content.includes('Only set `model=`') ||
content.includes('only set `model=`') ||
content.includes('Only set model=') ||
content.includes('omit the `model=`') ||
content.includes('omit the model=') ||
content.includes('omit `model=`') ||
content.includes('omit model=')
);
const lines = content.split(/\r?\n/);
const hasLiteralInheritInTask = lines.some(line => {
if (!/model\s*=\s*["']inherit["']/.test(line)) return false;
// Exclude instructional/explanatory lines that document what NOT to do
return !/\b(not|NOT|don'?t|do not|DO NOT|never|NEVER)\b/.test(line);
});
assert.ok(
!hasLiteralInheritInTask,
'execute-phase workflow must not pass literal "inherit" string to Task() model parameter'
);
assert.ok(
hasConditionalModelParam && !hasLiteralInheritInTask,
'execute-phase.md must conditionally omit model= when executor_model is "inherit", never pass it literally. ' +
'The unconditional model="{executor_model}" template would pass the literal ' +
'string "inherit" to Task(), which falls back to the default model instead ' +
'of the orchestrator model (root cause of #2516).'
);
// Guard against a future contributor adding an unconditional model="{executor_model}"
// template alongside the conditional docs — that would pass "inherit" literally to Task().
const hasUnsafeTemplate = lines.some(line => {
if (!/model\s*=\s*['"]\{executor_model\}['"]/.test(line)) return false;
return !/\b(not|NOT|do not|DO NOT|don'?t|never|NEVER|omit)\b/i.test(line);
});
assert.ok(!hasUnsafeTemplate,
'execute-phase.md must not contain an unconditional model="{executor_model}" template — ' +
'it would pass "inherit" literally to Task() when executor_model is "inherit"'
);
});
test('workflow documents that omitting model= causes inheritance from orchestrator', () => {
assert.ok(fs.existsSync(WORKFLOW_PATH), 'gsd-core/workflows/execute-phase.md should exist');
const content = fs.readFileSync(WORKFLOW_PATH, 'utf-8');
const hasInheritanceExplanation =
content.includes('inherit') &&
(
content.includes('orchestrator model') ||
content.includes('orchestrator\'s model') ||
content.includes('inherits the') ||
content.includes('inherit the current')
);
assert.ok(
hasInheritanceExplanation,
'execute-phase.md must explain that omitting model= causes Claude Code to ' +
'inherit the current orchestrator model — this is the mechanism that makes ' +
'"inherit" work correctly.'
);
});
});
});
}
// ────────────────────────────────────────────────────────────────────────
// Folded from tests/bug-2002-offer-next-context.test.cjs — consolidation epic #1969 (B4 #1973)
// ────────────────────────────────────────────────────────────────────────
{
const { describe: __foldDescribe } = require('node:test');
__foldDescribe("folded:bug-2002-offer-next-context (consolidation epic #1969 B4 #1973)", () => {
/**
* Regression tests for bug #2002
*
* offer_next in execute-phase.md must present conditional next steps
* based on whether CONTEXT.md already exists for the next phase.
* The previous flat list offered all options equally with no primary
* recommendation, leaving agents without guidance on the correct first step.
*
* Fixed: offer_next now checks for {next}-CONTEXT.md in the phase directory.
* - If CONTEXT.md is missing: primary suggestion is /gsd-discuss-phase
* - If CONTEXT.md exists: primary suggestion is /gsd-plan-phase
*/
'use strict';
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const workflowPath = path.resolve(
__dirname, '..', 'gsd-core', 'workflows', 'execute-phase.md'
);
describe('bug #2002: next-step suggestion checks CONTEXT.md (now via transition offer_next_phase, reached by execute-phase post-completion delegation — #1526)', () => {
// #1526: execute-phase no longer carries an inline offer_next step — it delegates
// post-completion to the transition workflow, whose offer_next_phase step performs
// the #2002 CONTEXT.md-gated next-step suggestion. These tests track that behavior
// in its new home (transition.md) and assert the delegation reaches it.
const transPath = path.resolve(__dirname, '..', 'gsd-core', 'workflows', 'transition.md');
let offerNextPhase;
test('setup: transition.md offer_next_phase section is readable', () => {
const trans = fs.readFileSync(transPath, 'utf-8');
const start = trans.indexOf('<step name="offer_next_phase">');
const end = trans.indexOf('</step>', start);
assert.notEqual(start, -1, 'transition.md must have an offer_next_phase step');
offerNextPhase = trans.slice(start, end);
assert.ok(offerNextPhase.length > 0, 'offer_next_phase section must be non-empty');
});
test('#1526: execute-phase delegates post-completion to transition (no inline offer_next step)', () => {
const wf = fs.readFileSync(workflowPath, 'utf-8');
assert.ok(wf.includes('delegate_post_completion_to_transition'), 'execute-phase must delegate post-completion to transition');
assert.ok(wf.includes('@~/.claude/gsd-core/workflows/transition.md'), 'execute-phase must @-include transition.md');
assert.equal(wf.includes('<step name="offer_next">'), false, 'inline offer_next step is intentionally removed (delegated to transition.offer_next_phase)');
});
test('offer_next_phase checks for CONTEXT.md existence (#2002 preserved)', () => {
assert.ok(
offerNextPhase.includes('CONTEXT.md'),
'offer_next_phase must reference CONTEXT.md to determine primary next step'
);
});
test('offer_next_phase presents /gsd-discuss-phase when CONTEXT.md does not exist', () => {
assert.ok(
/CONTEXT\.md.*does not exist|CONTEXT\.md.*not.*exist|If CONTEXT\.md does/i.test(offerNextPhase) ||
/discuss-phase/i.test(offerNextPhase),
'offer_next_phase must present /gsd-discuss-phase as primary when CONTEXT.md does not exist'
);
});
test('offer_next_phase presents /gsd-plan-phase when CONTEXT.md exists', () => {
assert.ok(
/CONTEXT\.md.*exists|exists.*CONTEXT\.md|If CONTEXT\.md/i.test(offerNextPhase),
'offer_next_phase must present /gsd-plan-phase as primary when CONTEXT.md exists'
);
});
test('offer_next_phase contains at least one conditional guard before listing commands', () => {
assert.ok(
/If CONTEXT\.md/i.test(offerNextPhase),
'offer_next_phase must contain at least one "If CONTEXT.md" conditional guard'
);
});
});
});
}
// ────────────────────────────────────────────────────────────────────────
// Folded from tests/fix-3177-execute-phase-dispatch-claim.test.cjs — test hygiene #3334 (H3)
// ────────────────────────────────────────────────────────────────────────
{
const { describe: __foldDescribe } = require('node:test');
__foldDescribe("folded:fix-3177-execute-phase-dispatch-claim (test hygiene #3334 H3)", () => {
// allow-test-rule: runtime-contract-is-the-product #3177 — the workflow markdown is loaded
// verbatim into the agent's context and the matrix/descriptor ARE the negotiated host
// contract; asserting agreement between those documents is behavioral, not source-grep.
'use strict';
const { test, describe } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const fc = require('fast-check');
const ROOT = path.join(__dirname, '..');
const WORKFLOW = path.join(ROOT, 'gsd-core', 'workflows', 'execute-phase.md');
const MATRIX = path.join(ROOT, 'docs', 'reference', 'host-integration-capability-matrix.md');
const DESCRIPTOR = path.join(ROOT, 'capabilities', 'claude', 'capability.json');
const workflowText = () => fs.readFileSync(WORKFLOW, 'utf8');
/**
* Value of `| <field> | <value> | …` inside the `## <host>` section of the matrix.
*
* Anchored on a whole heading LINE, not a substring: `## claude` must never match
* `## claude-local`, and the section must end at the next `## ` heading so a field
* absent from this host can never be answered from the next host's table.
*
* @param {string} matrix - full matrix document text
* @param {string} host - section name, e.g. `claude`
* @param {string} field - row label, e.g. `dispatch.background`
* @returns {string|null} trimmed cell value, or null when the section or row is absent
*/
function matrixField(matrix, host, field) {
const lines = matrix.split('\n');
const start = lines.findIndex((l) => l.trim() === `## ${host}`);
if (start === -1) return null;
let end = lines.length;
for (let i = start + 1; i < lines.length; i += 1) {
if (lines[i].startsWith('## ')) { end = i; break; }
}
const row = lines.slice(start + 1, end).find((l) => l.startsWith(`| ${field} |`));
if (!row) return null;
return row.split('|')[2].trim();
}
describe('#3177: execute-phase.md states Claude Code dispatch truthfully', () => {
test('execute-phase.md never claims Claude Code Agent() blocks or returns synchronously', () => {
// Row 1 — the failing-first regression. Both stale sentences, by their own text.
const text = workflowText();
const stale = ['blocks until complete', 'returns synchronously'];
const present = stale.filter((phrase) => text.includes(phrase));
assert.deepEqual(
present, [],
'execute-phase.md still asserts synchronous Claude Code dispatch. Claude Code backgrounds '
+ 'subagents by default (v2.1.198+); `run_in_background: false` is the opt-out.',
);
});
test('the Claude Code dispatch bullet states the background-by-default model', () => {
// Row 2 — the corrected sentence must actually SAY the true thing, not merely
// omit the false one. A deletion would pass row 1 while teaching nothing.
const bullet = workflowText()
.split('\n')
.find((l) => l.startsWith('- **Claude Code:**'));
assert.ok(bullet, 'the <runtime_compatibility> Claude Code bullet must exist');
assert.match(
bullet, /backgrounded by default/,
'the bullet must state that dispatch is backgrounded by default',
);
assert.match(
bullet, /verify completion/,
'the bullet must point at completion verification. This workflow deliberately backgrounds '
+ 'its executors (the multi-plan path prescribes run_in_background: true), so the blocking '
+ 'opt-out is not the guidance here — confirming completion is.',
);
assert.ok(
bullet.includes('Agent(subagent_type="gsd-executor"'),
'the bullet must still carry the dispatch mechanism the rest of the file depends on',
);
});
test('the workflow prose and the capability matrix agree on claude dispatch.background', () => {
// Row 3 — the parity guard. This is the assertion that outlives the wording:
// flip the matrix to `false` without touching the prose (or vice versa) and this reds.
const declared = matrixField(fs.readFileSync(MATRIX, 'utf8'), 'claude', 'dispatch.background');
assert.equal(declared, 'true', 'matrix must document claude dispatch.background');
const bullet = workflowText()
.split('\n')
.find((l) => l.startsWith('- **Claude Code:**'));
const proseSaysBackground = /backgrounded by default/.test(bullet ?? '');
assert.equal(
proseSaysBackground, declared === 'true',
'execute-phase.md and the host-integration matrix disagree about whether claude backgrounds '
+ 'its subagent dispatch. They describe the same runtime; exactly one of them is wrong (#3177).',
);
});
test('the claude descriptor and the matrix agree on dispatch.background', () => {
// Row 4 — the other half of the divergence class. The matrix is generated from
// the descriptor, so this pins the generator's output to its input.
const descriptor = JSON.parse(fs.readFileSync(DESCRIPTOR, 'utf8'));
const declared = matrixField(fs.readFileSync(MATRIX, 'utf8'), 'claude', 'dispatch.background');
assert.equal(
String(descriptor.runtime.hostIntegration.dispatch.background), declared,
'capabilities/claude/capability.json and the rendered matrix disagree',
);
});
test('the Codex orchestrator rule is not swept by the Claude Code correction', () => {
// Row 5 — negative space. Codex dispatch IS synchronous. A regex sweep for
// "return its result" would introduce a NEW falsehood here; this catches that.
const text = workflowText();
const codexRules = text
.split('\n')
.filter((l) => l.includes('ORCHESTRATOR RULE — CODEX RUNTIME'));
assert.ok(codexRules.length >= 2, 'both Codex orchestrator rules must survive');
for (const rule of codexRules) {
assert.ok(
rule.includes('Wait for the subagent to return its result'),
'Codex dispatch is genuinely synchronous — its wait rule must not be corrected away',
);
}
});
test('the Copilot and multi-plan dispatch rules survive the correction', () => {
// Row 6 — negative space. These were already true and are adjacent to the edit.
const text = workflowText();
assert.ok(
text.includes('- **Copilot:** Subagent spawning does not reliably return completion signals.'),
'the Copilot bullet must survive verbatim',
);
assert.ok(
text.includes('one at a time with `run_in_background: true`'),
'the multi-plan wave prescription must survive verbatim',
);
assert.ok(
text.includes('If `Agent` IS available (top-level Claude'),
'the spawn mandate is derived from TOOL AVAILABILITY, not from blocking — it must survive',
);
});
});
describe('#3177: matrix section extraction is bounded by its heading', () => {
const matrix = () => fs.readFileSync(MATRIX, 'utf8');
test('section extraction — first row of the section', () => {
// limit-1: the row immediately after the `## claude` heading is INSIDE.
assert.equal(matrixField(matrix(), 'claude', 'embeddingMode'), 'imperative');
});
test('section extraction — last row before the next heading', () => {
// limit: the final row of `## claude` is still INSIDE.
assert.ok(matrixField(matrix(), 'claude', 'dispatch.isolation').startsWith('harness-worktree'));
});
test('section extraction — a row in the next section never leaks in', () => {
// limit+1: a field absent from claude must be null rather than silently
// resolved from `## codex` below it, and two hosts with different values
// must never resolve to the same cell.
const doc = matrix();
assert.equal(matrixField(doc, 'claude', 'dispatch.background'), 'true');
assert.equal(matrixField(doc, 'claude', '__definitely_not_a_field__'), null);
assert.notEqual(
matrixField(doc, 'claude', 'effortSurface'),
matrixField(doc, 'kilo', 'effortSurface'),
'two hosts with different values must not resolve to the same cell',
);
});
test('fc property: field extraction never leaks across ## boundaries', () => {
const hostArb = fc.stringMatching(/^[a-z][a-z0-9-]{0,12}$/);
const valueArb = fc.stringMatching(/^[a-z0-9]{1,10}$/);
fc.assert(
fc.property(
fc.uniqueArray(fc.tuple(hostArb, valueArb), {
minLength: 2, maxLength: 6, selector: ([h]) => h,
}),
fc.nat(),
(sections, pick) => {
const doc = sections
.map(([host, value]) => `## ${host}\n\n| axis | value |\n| f | ${value} |\n`)
.join('\n');
const [host, value] = sections[pick % sections.length];
// Exactly the requested section's value, never a neighbor's.
assert.equal(matrixField(doc, host, 'f'), value);
// A longer name that merely EXTENDS a real heading resolves to nothing.
// `_` is outside hostArb's alphabet, so this probe can NEVER collide with
// another generated section — the `-local` form could, and did.
assert.equal(matrixField(doc, `${host}_x`, 'f'), null);
},
),
{ numRuns: 200 },
);
});
});
describe('#3177: debug.md dispatches its session manager in the foreground', () => {
const DEBUG_WF = path.join(ROOT, 'gsd-core', 'workflows', 'debug.md');
const debugText = () => fs.readFileSync(DEBUG_WF, 'utf8');
/**
* Every fenced `Agent( … )` block in debug.md that dispatches the session manager.
*
* Anchored on a line that is exactly `Agent(` so the PROSE mention of
* `Agent(subagent_type="gsd-debug-session-manager", …)` inside the blockquote at
* :206 is not mistaken for a dispatch. Positional selection (first match wins) was
* the original bug here: it silently checked the continue path while the
* new-session path went unexamined.
*/
function sessionManagerDispatches(text) {
const blocks = [];
for (const m of text.matchAll(/^Agent\($/gm)) {
const close = text.indexOf('\n)', m.index);
if (close === -1) continue;
const block = text.slice(m.index, close);
if (block.includes('subagent_type="gsd-debug-session-manager"')) blocks.push(block);
}
return blocks;
}
test('every session-manager spawn carries the run_in_background: false opt-out', () => {
// #2196 required this dispatch be foreground and blocking so the orchestrator
// receives the session summary inline; debug.md still says "Wait for it; do not
// background it" and "Display the compact summary returned by the session
// manager". Claude Code backgrounds subagents by DEFAULT, so that intent only
// holds if each call states the opt-out explicitly — prose alone silently
// reinstated the exact lost-handoff failure #2196 was filed to fix.
const blocks = sessionManagerDispatches(debugText());
assert.equal(
blocks.length, 2,
'debug.md dispatches the session manager on BOTH the new-session and continue paths; '
+ 'a change to that count means a dispatch was added or removed and must be re-checked.',
);
for (const block of blocks) {
assert.match(
block, /run_in_background\s*=\s*false/,
'every gsd-debug-session-manager dispatch must pass run_in_background=false — without '
+ 'it Claude Code backgrounds the spawn and the compact summary never returns (#2196).',
);
}
});
test('debug.md does not assert the spawn is inherently foreground', () => {
// The old premise ("is FOREGROUND and BLOCKING") was a property claim about the
// host, not an instruction — and it was false for the same reason as #3177.
assert.ok(
!debugText().includes('is FOREGROUND and BLOCKING'),
'debug.md must not claim the Agent() call is inherently foreground; it must name the '
+ 'run_in_background: false opt-out that actually makes it so.',
);
});
});
});
}