* fix(#2589): use --raw/--pick for config/model/verify lookups, drop jq dep The reviewer/workflow config lookups resolved scalars and object fields with a `gsd_run query <cmd> … | jq … 2>/dev/null || <default>` shape. On any machine without jq (the default on Windows/Git-Bash) the jq stage fails with exit 127, the failure is swallowed by 2>/dev/null + the trailing || default, and the variable comes back EMPTY — the configured per-lane model/host/budget is silently dropped and the lane falls back to CLI defaults with no diagnostic. gsd-tools ships native flags that do the same job with no external dep: config-get <key> --raw (strips JSON quotes off a scalar) resolve-model <id> --pick model (descends an object) resolve-execution … --pick <f> (same) verification.status … --pick status Replaced every jq-piped config/model/verify lookup across review.md (×23), plan-phase.md, ship.md, debug.md (incl. the redundant boolean coercion — --raw returns true/false as bare tokens natively), autonomous.md (×2), ai-integration-phase.md (×4), and eval-review.md. The legitimate structured-JSON jq sites that parse HTTP curl responses (.choices[0], jq -rs, jq -n --rawfile) are untouched — only the jq-replaceable lookups moved to the native flags. Adds tests/fix-2589-config-get-no-jq.test.cjs: a source-invariant guard asserting no audited workflow pipes config-get/resolve-model/resolve-execution/verification.status to jq (fails-first on the pre-fix text, passes after). * test(#2589): update autonomous-converge jq assertion to --pick; regen golden fixtures Two test consequences of the workflow-doc edits in the prior commit: 1. tests/autonomous-converge.test.cjs pinned the OLD jq-dependent shape (`verification.status … | jq -r '.status//empty'`) as the canonical routing contract. The test's INTENT is correct (route human validation through canonical verification.status) but it over-specified the MECHANISM (the jq pipe). Updated the assertion to match the new native --pick status shape; the contract being guarded (canonical verification.status read before the human_needed branch) is unchanged. 2. The golden-install-parity fixtures (19 runtimes) record a content hash of every installed workflow .md; the 7 edited workflows changed those hashes. Regenerated via `npm run gen:golden` (the test's own failure message instructs this). Only the 7 edited workflow hashes changed in each fixture. * fix(#2589): declare jq a prerequisite for the lanes that still need it; repair test file Three defects in the first cut of the #2589 fix: 1. tests/autonomous-converge.test.cjs was a JavaScript syntax error. The regex literal /...2>\/dev/null .../ left the second slash unescaped, terminating the literal early and parsing `null` as regex flags: SyntaxError: Invalid regular expression flags The whole file failed to load, so every assertion in it — including the #1522 and #1526 guards — silently stopped running. Replaced with the string-compare form already used at line 202 for the sibling shell-snippet assertion. 2. lint:ci failed. tests/fix-2589-config-get-no-jq.test.cjs buckets into the capped `config` production module via its `config-get-...` effective prefix, making it a novel offender against the 2-file cap. The test is about workflow documents, not the config module, so it is renamed to fix-2589-workflow-jq-dependency.test.cjs (free prefix) rather than growing the allowlist with a module that does not actually need a 5th test file. 3. The fix deleted the repo's only jq-prerequisite declaration. review.md:244 ("install jq if missing") was the anchor plan-review-convergence.md cites by line number, and it went away with the jq pipes — while the ollama, lm_studio, llama_cpp, opencode, and agy lanes still hard-require jq to parse HTTP /v1/chat/completions responses, opencode's JSONL event stream, and agy's conversation cache. On a jq-less host those five lanes swallow exit 127 into empty output: the same silent-degradation class #2589 exists to close. detect_clis now probes jq alongside the other prerequisites and emits jq:available / jq:missing, and the five dependent lanes are treated as undetected when it is absent, with an install hint. The six lanes that do not need jq stay selectable. plan-review-convergence.md now cites the section by name instead of a line number that moves. Regression guards added to the renamed test file: review.md must keep the jq probe and must name all five dependent lanes, and no workflow may cite review.md by line number. Workflow-size baseline and the 19 install-parity goldens regenerated for the review.md / plan-review-convergence.md edits. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TCwhbMuY37DzRMCfzTABJ * fix(#2589): decide the ship verification gate on a single verification.status read Isolated review finding (medium). Pre-fix, ship.md captured verification.status ONCE into $VERIFICATION and picked status / next_action / next_command off that cached JSON with three jq calls. --pick takes a single dot-path field, so the mechanical conversion issued three separate queries up front: three node spawns that each re-read the phase VERIFICATION.md and re-derive the commit-time vs mtime staleness comparison, on every ship — including the common passing path that never uses the two message fields. It also meant the gate's verdict and the message shown to the user were derived from three reads with no guarantee they observed the same state. The gate now reads `status` once and decides. The two message-only fields are read on the blocking path only, after PHASE_VERIFICATION_INCOMPLETE is already determined — so the passing path costs one query instead of three, and a concurrent write between reads can no longer make the gate and its message disagree, because the block/allow decision no longer depends on them. Adding a multi-field --pick to gsd-tools would have collapsed this to one query, but that changes the flag's output contract and belongs in its own change. Regression guard in tests/fix-2589-workflow-jq-dependency.test.cjs: ship.md must read verification.status exactly three times total, the block decision must follow the status read, and next_action / next_command must both appear after the blocking prose so they cannot drift back onto the passing path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TCwhbMuY37DzRMCfzTABJ * docs(#2589): document the jq prerequisite for the five reviewer lanes that need it /gsd-review's ollama, lm_studio, llama_cpp, opencode, and agy lanes parse JSON GSD does not produce (OpenAI-compatible /v1/chat/completions responses, OpenCode's JSONL event stream, Antigravity's conversation cache), so they require jq on PATH. Nothing in docs/ said so. Records which five lanes need it, which six do not, that reading configured models/hosts/budgets no longer requires jq at all, and what /gsd-review now does when jq is absent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_015TCwhbMuY37DzRMCfzTABJ * chore(#2589): backfill changeset pr number (#2673) --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
223 lines
9.6 KiB
JavaScript
223 lines
9.6 KiB
JavaScript
// allow-test-rule: source-text-is-the-product
|
|
// The autonomous command and workflow markdown are runtime-loaded contracts.
|
|
// Checking their text verifies the shipped slash-command behavior.
|
|
|
|
'use strict';
|
|
|
|
const { describe, test } = require('node:test');
|
|
const assert = require('node:assert/strict');
|
|
const fs = require('node:fs');
|
|
const path = require('node:path');
|
|
|
|
const REPO_ROOT = path.join(__dirname, '..');
|
|
const COMMAND_PATH = path.join(REPO_ROOT, 'commands', 'gsd', 'autonomous.md');
|
|
const WORKFLOW_PATH = path.join(REPO_ROOT, 'gsd-core', 'workflows', 'autonomous.md');
|
|
const COMMANDS_DOC_PATH = path.join(REPO_ROOT, 'docs', 'COMMANDS.md');
|
|
const HOW_TO_PATH = path.join(REPO_ROOT, 'docs', 'how-to', 'run-phases-autonomously.md');
|
|
|
|
function read(filePath) {
|
|
return fs.readFileSync(filePath, 'utf8');
|
|
}
|
|
|
|
describe('autonomous --converge flag (#711)', () => {
|
|
test('command advertises --converge and documents --cross-ai as alias', () => {
|
|
const command = read(COMMAND_PATH);
|
|
|
|
assert.match(
|
|
command,
|
|
/^argument-hint:.*--converge/m,
|
|
'autonomous command should advertise --converge in argument-hint',
|
|
);
|
|
assert.match(command, /--cross-ai/, 'autonomous command should document --cross-ai alias');
|
|
assert.match(
|
|
command,
|
|
/workflow\.plan_review_convergence=true/,
|
|
'autonomous command should mention the existing convergence feature gate',
|
|
);
|
|
});
|
|
|
|
test('workflow parses converge aliases into a plan strategy', () => {
|
|
const workflow = read(WORKFLOW_PATH);
|
|
|
|
assert.match(workflow, /PLAN_STRATEGY="local"/, 'workflow should default to local planning');
|
|
assert.match(workflow, /PLAN_STRATEGY="converge"/, 'workflow should opt into converge planning');
|
|
assert.match(workflow, /converge\|cross-ai/, 'workflow should accept --converge and --cross-ai');
|
|
});
|
|
|
|
test('workflow fails fast when convergence is requested but disabled', () => {
|
|
const workflow = read(WORKFLOW_PATH);
|
|
|
|
assert.match(
|
|
workflow,
|
|
/config-get workflow\.plan_review_convergence/,
|
|
'workflow should check workflow.plan_review_convergence before planning',
|
|
);
|
|
assert.match(
|
|
workflow,
|
|
/gsd config-set workflow\.plan_review_convergence true/,
|
|
'workflow should print the enable command instead of silently downgrading',
|
|
);
|
|
});
|
|
|
|
test('workflow routes planning through plan-review-convergence when enabled', () => {
|
|
const workflow = read(WORKFLOW_PATH);
|
|
|
|
assert.match(
|
|
workflow,
|
|
/Skill\(skill="gsd-plan-review-convergence", args="\$\{PHASE_NUM\} \$\{CONVERGENCE_ARGS\}"\)/,
|
|
'non-interactive converge mode should call gsd-plan-review-convergence',
|
|
);
|
|
assert.match(
|
|
workflow,
|
|
/Run plan convergence for phase \$\{PHASE_NUM\}: Skill\(skill=\\"gsd-plan-review-convergence\\"/,
|
|
'interactive converge mode should dispatch plan convergence in the background agent',
|
|
);
|
|
assert.match(
|
|
workflow,
|
|
/Skill\(skill="gsd-plan-phase", args="\$\{PHASE_NUM\}"\)/,
|
|
'local planning path should remain available for default autonomous runs',
|
|
);
|
|
});
|
|
|
|
test('workflow forwards reviewer flags and max cycles to convergence', () => {
|
|
const workflow = read(WORKFLOW_PATH);
|
|
const reviewerFlags = [
|
|
'--codex',
|
|
'--gemini',
|
|
'--claude',
|
|
'--opencode',
|
|
'--ollama',
|
|
'--lm-studio',
|
|
'--llama-cpp',
|
|
'--all',
|
|
'--text',
|
|
];
|
|
|
|
assert.match(workflow, /CONVERGENCE_ARGS/, 'workflow should build convergence pass-through args');
|
|
for (const flag of reviewerFlags) {
|
|
assert.ok(workflow.includes(flag), `workflow should pass through ${flag}`);
|
|
}
|
|
assert.match(workflow, /--max-cycles/, 'workflow should pass through --max-cycles N');
|
|
});
|
|
|
|
test('docs show autonomous convergence usage', () => {
|
|
const commandsDoc = read(COMMANDS_DOC_PATH);
|
|
const howTo = read(HOW_TO_PATH);
|
|
|
|
assert.match(commandsDoc, /--converge/, 'COMMANDS.md should document --converge');
|
|
assert.match(commandsDoc, /--cross-ai/, 'COMMANDS.md should document --cross-ai alias');
|
|
assert.match(howTo, /\/gsd-autonomous --only 4 --converge/, 'how-to should show single-phase converge usage');
|
|
});
|
|
});
|
|
|
|
describe('autonomous verification deferral contract', () => {
|
|
test('workflow records explicit deferred states instead of silently advancing (#1525)', () => {
|
|
const workflow = read(WORKFLOW_PATH);
|
|
|
|
assert.match(workflow, /verification_deferred_human/);
|
|
assert.match(workflow, /verification_deferred_gaps/);
|
|
assert.match(workflow, /Deferred Verification/);
|
|
assert.match(workflow, /gsd:verify-work \$\{PHASE_NUM\}/);
|
|
assert.match(workflow, /gsd:plan-phase \$\{PHASE_NUM\} --gaps/);
|
|
assert.match(
|
|
workflow,
|
|
/\| \$\{PHASE_NUM\} \| verification_deferred_human \| \/gsd:verify-work \$\{PHASE_NUM\} \|/,
|
|
'human deferral must persist the exact deferred STATE row',
|
|
);
|
|
assert.match(
|
|
workflow,
|
|
/\| \$\{PHASE_NUM\} \| verification_deferred_gaps \| \/gsd:plan-phase \$\{PHASE_NUM\} --gaps \|/,
|
|
'gap deferral must persist the exact deferred STATE row',
|
|
);
|
|
assert.doesNotMatch(
|
|
workflow,
|
|
/Human validation deferred` and proceed to iterate step/,
|
|
'human-needed deferral must not silently proceed to the next phase',
|
|
);
|
|
assert.doesNotMatch(
|
|
workflow,
|
|
/Gaps deferred` and proceed to iterate step/,
|
|
'gap deferral must not silently proceed to the next phase',
|
|
);
|
|
assert.match(
|
|
workflow,
|
|
/Skip deferred phases on autonomous re-entry/,
|
|
'reruns must explicitly skip deferred verification phases',
|
|
);
|
|
assert.match(
|
|
workflow,
|
|
/Deferred Verification \(Skipped on Re-entry\)/,
|
|
'workflow should surface skipped deferred phases and their resume commands',
|
|
);
|
|
});
|
|
|
|
test('workflow runs normal transition post-processing after passed verification (#1526)', () => {
|
|
const workflow = read(WORKFLOW_PATH);
|
|
const passedIdx = workflow.indexOf('**If `passed`:**');
|
|
const transitionIdx = workflow.indexOf('transition.md', passedIdx);
|
|
const iterateIdx = workflow.indexOf('Proceed to iterate step', passedIdx);
|
|
|
|
assert.ok(transitionIdx > passedIdx, 'passed verification must invoke transition.md');
|
|
assert.ok(
|
|
transitionIdx < iterateIdx,
|
|
'normal transition post-processing must run before autonomous iterates',
|
|
);
|
|
});
|
|
|
|
test('workflow reads canonical verification status before human-needed promotion (#1522)', () => {
|
|
const workflow = read(WORKFLOW_PATH);
|
|
const waitIdx = workflow.indexOf('After execute, read canonical verification');
|
|
const humanNeededIdx = workflow.indexOf('**If `human_needed`:**', waitIdx);
|
|
const promoteIdx = workflow.indexOf('set VERIFICATION frontmatter `status: passed`', humanNeededIdx);
|
|
const section = workflow.slice(waitIdx, humanNeededIdx);
|
|
|
|
assert.ok(waitIdx !== -1, 'workflow must document the post-execution verification read');
|
|
assert.ok(humanNeededIdx > waitIdx, 'human_needed branch must follow verification status read');
|
|
assert.ok(promoteIdx > humanNeededIdx, 'human_needed branch must contain the promotion action');
|
|
// #2589: the verification read uses the native --pick flag (no jq dependency).
|
|
// String-based check (not a regex literal) so the assertion stays robust to
|
|
// shell metacharacters in the snippet and parses cleanly under espree.
|
|
assert.ok(
|
|
section.includes('VERIFY_STATUS=$(gsd_run query verification.status "${PHASE_DIR}" --pick status 2>/dev/null || true)'),
|
|
'autonomous must route human validation through canonical verification.status via the native --pick flag',
|
|
);
|
|
assert.doesNotMatch(
|
|
section,
|
|
/grep "\^status:"/,
|
|
'autonomous must not route stale human_needed reports from raw frontmatter',
|
|
);
|
|
});
|
|
|
|
test('workflow discovers incomplete phases from canonical verification projection (#1522)', () => {
|
|
const workflow = read(WORKFLOW_PATH);
|
|
const discoverStart = workflow.indexOf('<step name="discover_phases">');
|
|
const discoverEnd = workflow.indexOf('</step>', discoverStart);
|
|
const iterateStart = workflow.indexOf('<step name="iterate">');
|
|
const iterateEnd = workflow.indexOf('</step>', iterateStart);
|
|
const discoverStep = workflow.slice(discoverStart, discoverEnd);
|
|
const iterateStep = workflow.slice(iterateStart, iterateEnd);
|
|
|
|
assert.match(discoverStep, /INIT_MANAGER=\$\(gsd_run query init\.manager\)/);
|
|
assert.ok(
|
|
discoverStep.includes('if [[ "$INIT_MANAGER" == @file:* ]]; then INIT_MANAGER=$(cat "${INIT_MANAGER#@file:}"); fi'),
|
|
'autonomous discovery must dereference large init.manager payloads before parsing',
|
|
);
|
|
assert.match(discoverStep, /phase_complete !== true/);
|
|
assert.match(discoverStep, /verification_status !== "passed"/);
|
|
assert.match(discoverStep, /STATE_CONTENT=\$\(cat \.planning\/STATE\.md 2>\/dev\/null \|\| true\)/);
|
|
assert.match(discoverStep, /drop any phase whose number appears in the deferred-phase map/);
|
|
assert.doesNotMatch(discoverStep, /ROADMAP=\$\(gsd_run query roadmap\.analyze\)/);
|
|
assert.doesNotMatch(discoverStep, /disk_status !== "complete"/);
|
|
|
|
assert.match(iterateStep, /INIT_MANAGER=\$\(gsd_run query init\.manager\)/);
|
|
assert.ok(
|
|
iterateStep.includes('if [[ "$INIT_MANAGER" == @file:* ]]; then INIT_MANAGER=$(cat "${INIT_MANAGER#@file:}"); fi'),
|
|
'autonomous iteration must dereference large init.manager payloads before parsing',
|
|
);
|
|
assert.match(iterateStep, /phase_complete !== true/);
|
|
assert.match(iterateStep, /verification_status !== "passed"/);
|
|
assert.match(iterateStep, /STATE_CONTENT=\$\(cat \.planning\/STATE\.md 2>\/dev\/null \|\| true\)/);
|
|
assert.match(iterateStep, /drop deferred phases from the autonomous queue/);
|
|
});
|
|
});
|