From ef951098a61e800d8b15d218b864bf304733fc52 Mon Sep 17 00:00:00 2001 From: Tom Boucher Date: Mon, 18 May 2026 13:05:02 -0400 Subject: [PATCH] chore(3686): add release-tarball lifecycle smoke to install-smoke workflow (#3692) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * chore(3686): add release-tarball lifecycle smoke to install-smoke workflow Closes #3686. Adds a non-interactive lifecycle smoke that runs against the installed tarball (not the working tree). Catches the two recent release-time bug classes that the working-tree test suite cannot see: * #3684 — symbol mismatches between init.cjs imports and secrets.cjs exports that landed in v1.42.3 (closed/fixed-pending-release). * #3668 — bare `gsd-sdk` invocations in 75 of 78 workflow files with no `command -v gsd-sdk … elif node "$GSD_TOOLS"` fallback (open). Shape: * `scripts/release-tarball-smoke.cjs` — pure CJS module exporting a frozen `SMOKE` enum and a `runSmoke({ tarballPath, installPrefix, expectedVersion, fixtureDir, lifecycleCommands })` function. CLI `--json` mode prints `JSON.stringify(result)` and exits 0 iff `result.code === SMOKE.OK`. Install is `--prefix ` so it does not pollute global node_modules. * `tests/release-tarball-smoke.test.cjs` — 6 tests covering happy path, version mismatch, lifecycle command file resolution, missing-command detection, sdk binary callability, and structural workflow-body checks. Tests assert on the SMOKE enum directly; no `assert.match` on rendered prose, no try/finally in test bodies, no source-grep theater. Uses `before`/`after` to pack+install once across the test file. * `tests/release-tarball-smoke-workflow.test.cjs` — 7 structural assertions on the parsed install-smoke.yml IR (workflow_call trigger preserved, lifecycle step calls release-tarball-smoke.cjs with --json, jq check enforces result.code === "ok", path filter includes the new files, artifact-on-failure step present). * `.github/workflows/install-smoke.yml` — extended (not duplicated). New "Lifecycle smoke" step after the existing version check, on the same matrix. Artifact upload on failure for debugging. Path filter now triggers on changes to the new script + test. Per CONTRIBUTING.md §"Prohibited: Raw Text Matching on Test Outputs" this PR avoids the same anti-pattern that caused PR #3666 to be reverted (PR #3688) — the script returns frozen enum codes, tests assert on the enum, never on stdout strings. Workflow-body checks in Cycle 3 are INFORMATIONAL (count returned, not enforced) on this PR. After #3668's fix lands and the 75 missing fallbacks are added, the lane can be tightened to enforce zero. Co-Authored-By: Claude Opus 4.7 (1M context) * fix(3686): harden release smoke workflow and query scanner * fix(3686): move tarball-smoke test to install suite to fix Windows ETIMEDOUT + coverage OOM Windows (Node 22/24/26, jobs 76565215694/76565215710/76565215812): release-tarball-smoke.test.cjs had no suite marker so run-tests.cjs classified it as 'unit', running it on Windows PR CI. The before() hook calls execFileSync(npm.cmd install -g ...) with a 55 s timeout; on Windows GHA runners this npm global install consistently hits ETIMEDOUT (~62 s observed), causing all 3 Windows lanes to fail. Coverage (job 76565215085): c8 ran test:coverage:unit (unit suite only) with V8 coverage tracking active across child processes. The tarball-smoke test's before() hook spawned npm install subprocesses while c8 held V8 coverage descriptors open, driving the Node heap to 4 GB+ and triggering an OOM abort during report generation (exit code 134, all 5682 tests had already passed). Fix: rename to tests/release-tarball-smoke.install.test.cjs so run-tests.cjs routes it to the 'install' suite. The install suite is already skipped on PR CI by design (test.yml lines 179-181: only runs on main push). The dedicated install-smoke.yml workflow continues to exercise this test on its own matrix. Also update the install-smoke.yml PR path filter and the structural wiring test assertion to match the new filename. Co-Authored-By: Claude Sonnet 4.6 * refactor(test): route tarball-smoke install test through tests/helpers.cjs Replaces direct fs.mkdtempSync and execFileSync calls in tests/release-tarball-smoke.install.test.cjs with createTempDir() and a new runNpm() helper in tests/helpers.cjs. Cleanup is now automatic via the helper. Addresses CodeRabbit Major refactor at https://github.com/gsd-build/get-shit-done/pull/3692#discussion_r3260433892. --------- Co-authored-by: Claude Opus 4.7 (1M context) --- .github/workflows/install-smoke.yml | 19 + scripts/release-tarball-smoke.cjs | 677 ++++++++++++++++++ tests/helpers.cjs | 27 +- tests/release-tarball-smoke-workflow.test.cjs | 187 +++++ tests/release-tarball-smoke.install.test.cjs | 187 +++++ 5 files changed, 1096 insertions(+), 1 deletion(-) create mode 100644 scripts/release-tarball-smoke.cjs create mode 100644 tests/release-tarball-smoke-workflow.test.cjs create mode 100644 tests/release-tarball-smoke.install.test.cjs diff --git a/.github/workflows/install-smoke.yml b/.github/workflows/install-smoke.yml index d90813a6d..7723b7543 100644 --- a/.github/workflows/install-smoke.yml +++ b/.github/workflows/install-smoke.yml @@ -23,6 +23,8 @@ on: - 'sdk/**' - 'package.json' - 'package-lock.json' + - 'scripts/release-tarball-smoke.cjs' + - 'tests/release-tarball-smoke.install.test.cjs' - '.github/workflows/install-smoke.yml' - '.github/workflows/release.yml' push: @@ -201,6 +203,23 @@ jobs: gsd-sdk --version || gsd-sdk --help echo "✓ gsd-sdk is executable" + - name: Lifecycle smoke + if: steps.skip.outputs.skip != 'true' + id: lifecycle-smoke + shell: bash + run: | + set -euo pipefail + node scripts/release-tarball-smoke.cjs --json | tee /tmp/release-smoke.json + jq -e '.code == "ok"' /tmp/release-smoke.json + + - name: Upload lifecycle smoke result on failure + if: steps.skip.outputs.skip != 'true' && failure() && steps.lifecycle-smoke.outcome == 'failure' + uses: actions/upload-artifact@ea165f8d65b6e75b540449e92b4886f43607fa02 # v4 + with: + name: release-smoke-${{ matrix.os }}-node${{ matrix.node-version }} + path: /tmp/release-smoke.json + retention-days: 7 + # --------------------------------------------------------------------------- # Job 2: unpacked-dir install — reproduces the mode-644 failure class (#2453) # diff --git a/scripts/release-tarball-smoke.cjs b/scripts/release-tarball-smoke.cjs new file mode 100644 index 000000000..d5e68ff08 --- /dev/null +++ b/scripts/release-tarball-smoke.cjs @@ -0,0 +1,677 @@ +#!/usr/bin/env node +/** + * scripts/release-tarball-smoke.cjs + * + * Release tarball smoke test for issue #3686. + * + * Guards against the class of bugs that can't be caught by working-tree tests: + * - #3684: maskIfSecret import/export mismatch shipped in v1.42.3 (runtime + * crash on installed package, invisible to unit tests) + * - #3668: 75/78 workflows call bare `gsd-sdk` without fallback; --local users + * see `command not found` + * + * Strategy: pack the working tree, install into a temp prefix, invoke the + * installed binary, assert the version matches package.json. Exercises the + * INSTALLED package, not the working tree. + * + * Exports: + * SMOKE — frozen enum of result codes + * runSmoke({ tarballPath, installPrefix, expectedVersion, fixtureDir, + * lifecycleCommands, dryRun }) + * → { code: SMOKE.*, details: { version, tarball, ... } } + * + * CLI entry: node scripts/release-tarball-smoke.cjs --json + * Packs working tree, installs to a temp prefix, checks version. + * Exits 0 on SMOKE.OK, 1 otherwise. + * Always prints JSON to stdout when --json flag is present. + * + * Lifecycle command checks (Cycle 2): + * For each command name (other than 'init') in lifecycleCommands: + * - Assert commands/gsd/.md exists in the installed package + * - Parse the .md for a workflow @-import or inline reference + * - Assert the referenced workflow .md exists in the installed package + * If 'init' is in lifecycleCommands, runs `get-shit-done-cc --local --claude` + * in fixtureDir to verify the installer is callable (INIT_FAILED on crash). + * Non-interactive: --local --claude flags skip all prompts. + * + * Workflow-body checks (Cycle 3 — informational until #3668 is fixed): + * - Calls `gsd-sdk "query" state.json --project-dir ` to verify + * the SDK binary is callable and produces parseable JSON (SDK_BINARY_NOT_CALLABLE). + * - Scans all installed get-shit-done/workflows/*.md for: + * (a) /gsd: colon-namespace leaks (WORKFLOW_BODY_COLON_LEAK) + * (b) bare `gsd-sdk` query invocations in shell fences without a `command -v gsd-sdk` + * guard in the same fence (WORKFLOW_MISSING_SDK_FALLBACK — #3668). + * Both checks populate result.details with counters but do NOT return a failure + * code by default; they are informational until the upstream fixes land. + */ + +'use strict'; + +const { execFileSync, spawnSync } = require('child_process'); +const fs = require('fs'); +const os = require('os'); +const path = require('path'); +const CHILD_TIMEOUT_MS = 120000; + +// --------------------------------------------------------------------------- +// Frozen result-code enum +// --------------------------------------------------------------------------- + +const SMOKE = Object.freeze({ + OK: 'ok', + VERSION_MISMATCH: 'version_mismatch', + PACK_FAILED: 'pack_failed', + INSTALL_FAILED: 'install_failed', + BIN_NOT_CALLABLE: 'bin_not_callable', + // Cycle 2 codes + COMMAND_FILE_MISSING: 'command_file_missing', + WORKFLOW_FILE_MISSING: 'workflow_file_missing', + INIT_FAILED: 'init_failed', + // Cycle 3 codes + SDK_BINARY_NOT_CALLABLE: 'sdk_binary_not_callable', + WORKFLOW_BODY_COLON_LEAK: 'workflow_body_colon_leak', + WORKFLOW_MISSING_SDK_FALLBACK: 'workflow_missing_sdk_fallback', +}); + +// --------------------------------------------------------------------------- +// Internal helpers +// --------------------------------------------------------------------------- + +/** + * Locate the lib/node_modules/get-shit-done-cc package root inside an + * npm --prefix install directory. + */ +function pkgRoot(installPrefix) { + // POSIX: /lib/node_modules/get-shit-done-cc + // Windows: /node_modules/get-shit-done-cc + const posix = path.join(installPrefix, 'lib', 'node_modules', 'get-shit-done-cc'); + const win = path.join(installPrefix, 'node_modules', 'get-shit-done-cc'); + return fs.existsSync(posix) ? posix : win; +} + +/** + * Locate the installed gsd-sdk binary (symlink in /bin/). + */ +function findGsdSdkBin(installPrefix) { + const binDir = process.platform === 'win32' + ? path.join(installPrefix, 'node_modules', '.bin') + : path.join(installPrefix, 'bin'); + + const candidates = process.platform === 'win32' + ? [path.join(binDir, 'gsd-sdk.cmd'), path.join(binDir, 'gsd-sdk')] + : [path.join(binDir, 'gsd-sdk')]; + + for (const c of candidates) { + if (fs.existsSync(c)) return c; + } + return null; +} + +/** + * Locate the get-shit-done-cc installer binary (the symlink in /bin/). + */ +function findInstallerBin(installPrefix) { + const binDir = process.platform === 'win32' + ? path.join(installPrefix, 'node_modules', '.bin') + : path.join(installPrefix, 'bin'); + + const candidates = process.platform === 'win32' + ? [path.join(binDir, 'get-shit-done-cc.cmd'), path.join(binDir, 'get-shit-done-cc')] + : [path.join(binDir, 'get-shit-done-cc')]; + + for (const c of candidates) { + if (fs.existsSync(c)) return c; + } + return null; +} + +/** + * Parse a command .md file and return the first workflow path it references. + * + * Structured parser — only inspects individual lines; never regexes on the + * whole-file string. Two recognised forms (in priority order): + * + * 1. @-import line: `@~/.claude/get-shit-done/workflows/.md` + * 2. Inline mention: any line containing `~/.claude/get-shit-done/workflows/.md` + * (takes the LAST occurrence so conditional-dispatch files resolve to the + * default / unconditional branch, e.g. discuss-phase.md) + * + * Returns the bare workflow filename (e.g. `"discuss-phase.md"`) or null. + */ +function parseWorkflowRef(mdContent) { + const WORKFLOW_PREFIX = 'get-shit-done/workflows/'; + let atImportResult = null; + let lastInlineResult = null; + + const lines = mdContent.split(/\r?\n/); + for (const line of lines) { + const trimmed = line.trim(); + + // Form 1: @-import + if (trimmed.startsWith('@') && trimmed.includes(WORKFLOW_PREFIX)) { + const idx = trimmed.indexOf(WORKFLOW_PREFIX); + const rest = trimmed.slice(idx + WORKFLOW_PREFIX.length); + // rest is like "discuss-phase.md" or "discuss-phase.md end-to-end." + const name = rest.split(/[\s`"]/)[0]; + if (name.endsWith('.md')) { + atImportResult = name; + break; // @-imports are authoritative; stop on first + } + } + + // Form 2: inline mention (collect last) + if (trimmed.includes(WORKFLOW_PREFIX)) { + const idx = trimmed.indexOf(WORKFLOW_PREFIX); + const rest = trimmed.slice(idx + WORKFLOW_PREFIX.length); + const name = rest.split(/[\s`"]/)[0]; + if (name.endsWith('.md')) { + lastInlineResult = name; + } + } + } + + return atImportResult !== null ? atImportResult : lastInlineResult; +} + +/** + * Read the list of known GSD command names from the installed package. + * Returns an array of strings like `['init', 'discuss-phase', ...]`. + */ +function readInstalledCmdNames(pkg) { + const commandsDir = path.join(pkg, 'commands', 'gsd'); + if (!fs.existsSync(commandsDir)) return []; + return fs.readdirSync(commandsDir) + .filter((f) => f.endsWith('.md')) + .map((f) => f.slice(0, -3)); // strip .md +} + +/** + * Scan a single workflow .md file for /gsd: colon-namespace leaks. + * + * Uses the word-boundary-safe regex shape from scripts/fix-slash-commands.cjs: + * /gsd-(||...)(?=[^a-zA-Z0-9_-]|$)/g — forward + * We check the colon form: /gsd: leaking in installed workflow bodies. + * + * Returns the first leaking { line, lineNumber } or null. + */ +function scanWorkflowColonLeak(filePath, cmdNames) { + if (!cmdNames || cmdNames.length === 0) return null; + const sorted = [...cmdNames].sort((a, b) => b.length - a.length); + const pattern = new RegExp(`/gsd:(${sorted.join('|')})(?=[^a-zA-Z0-9_-]|$)`, 'g'); + + const content = fs.readFileSync(filePath, 'utf-8'); + const lines = content.split(/\r?\n/); + for (let i = 0; i < lines.length; i++) { + pattern.lastIndex = 0; + if (pattern.test(lines[i])) { + return { line: i + 1, content: lines[i].trim() }; + } + } + return null; +} + +/** + * Scan a single workflow .md file for bare `gsd-sdk` query invocations inside + * shell fences that lack a `command -v gsd-sdk` guard in the same fence. + * + * Structured check: walks lines, tracks open/close shell fences (```bash / + * ```sh / ``` alone), collects `gsd-sdk` query lines and the fence's guard + * state, then emits findings per-fence. + * + * Returns the first unguarded { line, lineNumber } or null. + */ +function scanWorkflowMissingSdkFallback(filePath) { + const content = fs.readFileSync(filePath, 'utf-8'); + const lines = content.split(/\r?\n/); + + const FENCE_OPEN = /^```(?:bash|sh)?\s*$/; + const FENCE_CLOSE = /^```\s*$/; + const SDK_QUERY = /\bgsd-sdk\s+query\b/; + const COMMAND_V = /\bcommand\s+-v\s+gsd-sdk\b/; + + let inFence = false; + let fenceHasGuard = false; + let firstSdkQueryLineInFence = null; + let firstSdkQueryLineNumInFence = null; + + for (let i = 0; i < lines.length; i++) { + const line = lines[i]; + const trimmed = line.trim(); + + if (!inFence) { + if (FENCE_OPEN.test(trimmed)) { + inFence = true; + fenceHasGuard = false; + firstSdkQueryLineInFence = null; + firstSdkQueryLineNumInFence = null; + } + } else { + if (FENCE_CLOSE.test(trimmed)) { + // Closing the fence — check if there were bare sdk query calls without a guard + if (firstSdkQueryLineInFence !== null && !fenceHasGuard) { + return { line: firstSdkQueryLineNumInFence, content: firstSdkQueryLineInFence.trim() }; + } + inFence = false; + fenceHasGuard = false; + firstSdkQueryLineInFence = null; + firstSdkQueryLineNumInFence = null; + } else { + if (COMMAND_V.test(line)) { + fenceHasGuard = true; + } + if (SDK_QUERY.test(line) && firstSdkQueryLineInFence === null) { + firstSdkQueryLineInFence = line; + firstSdkQueryLineNumInFence = i + 1; + } + } + } + } + + return null; +} + +// --------------------------------------------------------------------------- +// Pure function: runSmoke +// --------------------------------------------------------------------------- + +/** + * @param {object} opts + * @param {string} opts.tarballPath - Absolute path to a pre-packed .tgz + * @param {string} opts.installPrefix - Temp directory to use as npm --prefix + * @param {string} opts.expectedVersion - semver string to assert (e.g. "1.50.0") + * @param {string} [opts.fixtureDir] - Temp dir to run `init` into (must NOT be HOME) + * @param {string[]} [opts.lifecycleCommands] - Commands to file-check (default: see below) + * @param {boolean} [opts.dryRun=false] - If true, skip actual npm install; validate input only + * @returns {{ code: string, details: object }} + */ +function runSmoke({ + tarballPath, + installPrefix, + expectedVersion, + fixtureDir, + lifecycleCommands = ['init', 'discuss-phase', 'plan-phase', 'execute-phase'], + dryRun = false, +}) { + const details = { + tarball: tarballPath, + prefix: installPrefix, + expectedVersion, + }; + + if (dryRun) { + return { code: SMOKE.OK, details: { ...details, version: expectedVersion, dryRun: true } }; + } + + // --- Install the tarball into the temp prefix ---------------------------- + const npmCmd = process.platform === 'win32' ? 'npm.cmd' : 'npm'; + const installResult = spawnSync( + npmCmd, + ['install', '-g', '--prefix', installPrefix, tarballPath], + { encoding: 'utf-8', shell: process.platform === 'win32', timeout: CHILD_TIMEOUT_MS }, + ); + + if (installResult.status !== 0) { + return { + code: SMOKE.INSTALL_FAILED, + details: { + ...details, + stderr: installResult.stderr, + stdout: installResult.stdout, + }, + }; + } + + // --- Locate the installed gsd-sdk binary --------------------------------- + const actualBin = findGsdSdkBin(installPrefix); + + if (!actualBin) { + const binDir = process.platform === 'win32' + ? path.join(installPrefix, 'node_modules', '.bin') + : path.join(installPrefix, 'bin'); + return { + code: SMOKE.BIN_NOT_CALLABLE, + details: { ...details, binDir, searched: [] }, + }; + } + + // --- Invoke `gsd-sdk --version` ------------------------------------------ + const versionResult = spawnSync( + process.execPath, + [actualBin, '--version'], + { encoding: 'utf-8', timeout: CHILD_TIMEOUT_MS }, + ); + + if (versionResult.status !== 0) { + return { + code: SMOKE.BIN_NOT_CALLABLE, + details: { + ...details, + bin: actualBin, + stderr: versionResult.stderr, + stdout: versionResult.stdout, + }, + }; + } + + // Output format: "gsd-sdk v1.50.0-canary.0\n" + const rawOutput = (versionResult.stdout || '').trim(); + const versionMatch = rawOutput.match(/v(.+)$/); + const installedVersion = versionMatch ? versionMatch[1] : rawOutput; + + details.version = installedVersion; + details.rawVersionOutput = rawOutput; + details.bin = actualBin; + + if (installedVersion !== expectedVersion) { + return { + code: SMOKE.VERSION_MISMATCH, + details: { ...details, installedVersion, expectedVersion }, + }; + } + + // ───────────────────────────────────────────────────────────────────────── + // Cycle 2: lifecycle command file-resolution checks + // ───────────────────────────────────────────────────────────────────────── + + const pkg = pkgRoot(installPrefix); + const shouldRunInit = lifecycleCommands.includes('init'); + const commandsToCheck = lifecycleCommands.filter((c) => c !== 'init'); + + // --- Run init if requested ----------------------------------------------- + if (shouldRunInit && fixtureDir) { + const installerBin = findInstallerBin(installPrefix); + + if (!installerBin) { + return { + code: SMOKE.INIT_FAILED, + details: { + ...details, + reason: 'get-shit-done-cc binary not found in installPrefix', + installPrefix, + }, + }; + } + + // Non-interactive: --local --claude installs to .claude/ in cwd (fixtureDir). + // GSD_TEST_MODE must be cleared — install.js skips its main() block when + // GSD_TEST_MODE is set, which would cause the installer to exit 0 silently + // without actually creating any files. + const initEnv = { ...process.env }; + delete initEnv.GSD_TEST_MODE; + + const initResult = spawnSync( + process.execPath, + [installerBin, '--local', '--claude'], + { + encoding: 'utf-8', + cwd: fixtureDir, + // Ensure no TTY so the installer's non-interactive fallback fires + stdio: ['pipe', 'pipe', 'pipe'], + env: initEnv, + timeout: CHILD_TIMEOUT_MS, + }, + ); + + if (initResult.status !== 0) { + return { + code: SMOKE.INIT_FAILED, + details: { + ...details, + fixtureDir, + stderr: initResult.stderr, + stdout: initResult.stdout, + }, + }; + } + + // Verify expected dirs were created + const expectedDirs = [ + path.join(fixtureDir, '.claude', 'commands'), + path.join(fixtureDir, '.claude', 'get-shit-done'), + ]; + for (const dir of expectedDirs) { + if (!fs.existsSync(dir) || !fs.statSync(dir).isDirectory()) { + return { + code: SMOKE.INIT_FAILED, + details: { + ...details, + fixtureDir, + reason: `expected dir not created: ${dir}`, + }, + }; + } + } + } + + // --- Check command files and workflow references ------------------------- + const lifecycleResolved = []; + + for (const cmd of commandsToCheck) { + const cmdFilePath = path.join(pkg, 'commands', 'gsd', `${cmd}.md`); + + if (!fs.existsSync(cmdFilePath) || !fs.statSync(cmdFilePath).isFile()) { + return { + code: SMOKE.COMMAND_FILE_MISSING, + details: { + ...details, + command: cmd, + path: cmdFilePath, + }, + }; + } + + // Parse workflow reference + const mdContent = fs.readFileSync(cmdFilePath, 'utf-8'); + const workflowName = parseWorkflowRef(mdContent); + + let workflowPath = null; + if (workflowName) { + // Workflow files live at get-shit-done/workflows/ in the package. + // Some live in subdirectories; try flat first then scan once. + const flat = path.join(pkg, 'get-shit-done', 'workflows', workflowName); + workflowPath = fs.existsSync(flat) ? flat : null; + + if (!workflowPath) { + return { + code: SMOKE.WORKFLOW_FILE_MISSING, + details: { + ...details, + command: cmd, + path: flat, + }, + }; + } + } + + lifecycleResolved.push({ + command: cmd, + commandPath: cmdFilePath, + workflowPath, + }); + } + + details.lifecycleResolved = lifecycleResolved; + + // ───────────────────────────────────────────────────────────────────────── + // Cycle 3: SDK binary callable + workflow-body validation (informational) + // ───────────────────────────────────────────────────────────────────────── + + // --- Verify `gsd-sdk` query is callable and returns parseable JSON ------- + const sdkQueryDir = fixtureDir || os.tmpdir(); + const sdkQueryResult = spawnSync( + process.execPath, + [actualBin, 'query', 'state.json', '--project-dir', sdkQueryDir], + { encoding: 'utf-8', timeout: CHILD_TIMEOUT_MS }, + ); + + if (sdkQueryResult.status !== 0) { + return { + code: SMOKE.SDK_BINARY_NOT_CALLABLE, + details: { + ...details, + sdkBin: actualBin, + sdkQueryStderr: sdkQueryResult.stderr, + sdkQueryStdout: sdkQueryResult.stdout, + }, + }; + } + + let sdkQueryParsed = false; + try { + JSON.parse(sdkQueryResult.stdout); + sdkQueryParsed = true; + } catch { + // leave sdkQueryParsed = false + } + + if (!sdkQueryParsed) { + return { + code: SMOKE.SDK_BINARY_NOT_CALLABLE, + details: { + ...details, + sdkBin: actualBin, + reason: 'gsd-sdk query-state output is not valid JSON', + sdkQueryStdout: sdkQueryResult.stdout, + }, + }; + } + + details.sdkQueryResult = sdkQueryResult.stdout; + details.sdkQueryParsed = true; + + // --- Workflow-body checks (informational — #3668 not yet fixed) ---------- + const workflowsDir = path.join(pkg, 'get-shit-done', 'workflows'); + const installedCmdNames = readInstalledCmdNames(pkg); + + let workflowsScanned = 0; + let colonLeakCount = 0; + let missingFallbackCount = 0; + // Store first finding per check type (for future enforcement mode) + let firstColonLeak = null; + let firstMissingFallback = null; + + if (fs.existsSync(workflowsDir)) { + // Collect all .md files (flat only — subdirs contain sub-workflows that + // follow the same contract, but the top-level .md files are the primary surface) + const entries = fs.readdirSync(workflowsDir, { withFileTypes: true }); + for (const entry of entries) { + if (!entry.isFile() || !entry.name.endsWith('.md')) continue; + const filePath = path.join(workflowsDir, entry.name); + workflowsScanned++; + + const leak = scanWorkflowColonLeak(filePath, installedCmdNames); + if (leak) { + colonLeakCount++; + if (!firstColonLeak) { + firstColonLeak = { file: filePath, line: leak.line }; + } + } + + const missingFallback = scanWorkflowMissingSdkFallback(filePath); + if (missingFallback) { + missingFallbackCount++; + if (!firstMissingFallback) { + firstMissingFallback = { file: filePath, line: missingFallback.line }; + } + } + } + } + + details.workflowsScanned = workflowsScanned; + details.colonLeakCount = colonLeakCount; + details.missingFallbackCount = missingFallbackCount; + if (firstColonLeak) details.firstColonLeak = firstColonLeak; + if (firstMissingFallback) details.firstMissingFallback = firstMissingFallback; + + // NOTE: colonLeakCount and missingFallbackCount are informational here. + // They will be non-zero against current main per #3668 and the /gsd: leak + // backlog. Once those issues are fixed, a future enforcement mode can be + // enabled (e.g. SMOKE_ENFORCE_WORKFLOW_BODY=1) to fail here. + + return { code: SMOKE.OK, details }; +} + +// --------------------------------------------------------------------------- +// CLI entry +// --------------------------------------------------------------------------- + +function cliMain() { + const args = process.argv.slice(2); + const isJson = args.includes('--json'); + + const pkgPath = path.join(__dirname, '..', 'package.json'); + const pkg = JSON.parse(fs.readFileSync(pkgPath, 'utf-8')); + const expectedVersion = process.env.SMOKE_FORCE_EXPECTED_VERSION || pkg.version; + + // Pack the working tree into a temp directory + const packDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-smoke-pack-')); + const installPrefix = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-smoke-prefix-')); + const fixtureDir = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-smoke-fixture-')); + + let tarballPath; + try { + const npmCmd = process.platform === 'win32' ? 'npm.cmd' : 'npm'; + const packOutput = execFileSync( + npmCmd, + ['pack', '--pack-destination', packDir], + { + cwd: path.join(__dirname, '..'), + encoding: 'utf-8', + shell: process.platform === 'win32', + timeout: CHILD_TIMEOUT_MS, + }, + ).trim(); + // npm pack outputs the filename on stdout (last line when verbose) + const lines = packOutput.split(/\r?\n/).filter(Boolean); + const tgzName = lines[lines.length - 1]; + tarballPath = path.join(packDir, tgzName); + if (!fs.existsSync(tarballPath)) { + // npm 7+ may print just the filename without .tgz extension on some platforms + const found = fs.readdirSync(packDir).find((f) => f.endsWith('.tgz')); + if (found) { + tarballPath = path.join(packDir, found); + } else { + const result = { + code: SMOKE.PACK_FAILED, + details: { packDir, packOutput, reason: 'no .tgz in pack destination' }, + }; + if (isJson) process.stdout.write(JSON.stringify(result) + '\n'); + cleanup(packDir, installPrefix, fixtureDir); + process.exit(1); + } + } + } catch (err) { + const result = { + code: SMOKE.PACK_FAILED, + details: { error: err.message, stderr: err.stderr }, + }; + if (isJson) process.stdout.write(JSON.stringify(result) + '\n'); + cleanup(packDir, installPrefix, fixtureDir); + process.exit(1); + } + + const result = runSmoke({ tarballPath, installPrefix, expectedVersion, fixtureDir }); + if (isJson) process.stdout.write(JSON.stringify(result) + '\n'); + cleanup(packDir, installPrefix, fixtureDir); + process.exit(result.code === SMOKE.OK ? 0 : 1); +} + +function cleanup(...dirs) { + for (const dir of dirs) { + try { + fs.rmSync(dir, { recursive: true, force: true }); + } catch { + // best-effort + } + } +} + +// --------------------------------------------------------------------------- +// Exports +// --------------------------------------------------------------------------- + +module.exports = { SMOKE, runSmoke }; + +if (require.main === module) { + cliMain(); +} diff --git a/tests/helpers.cjs b/tests/helpers.cjs index f1cacb109..e07e8c662 100644 --- a/tests/helpers.cjs +++ b/tests/helpers.cjs @@ -230,4 +230,29 @@ function toPosixPath(p) { return p == null ? p : p.split(path.sep).join('/'); } -module.exports = { runGsdTools, createTempDir, createTempProject, createTempGitProject, cleanup, parseFrontmatter, isUsageOutput, captureConsole, toPosixPath, TOOLS_PATH }; +/** + * Run an npm command via execFileSync with cross-platform portability. + * + * Handles the Windows `npm.cmd` vs POSIX `npm` distinction and the + * `shell: true` requirement on Windows so tests do not need to + * re-implement platform detection inline. + * + * @param {string[]} args - npm subcommand and flags (e.g. ['pack', '--pack-destination', dir]). + * @param {object} [options] - execFileSync options merged with platform defaults. + * `cwd`, `encoding`, `timeout`, and `env` are the commonly overridden keys. + * @returns {string} trimmed stdout string (encoding: 'utf-8'). + * @throws {Error} re-throws the execFileSync error on non-zero exit so callers + * get the full stderr in the error message. + */ +function runNpm(args, options = {}) { + const isWindows = process.platform === 'win32'; + const npmCmd = isWindows ? 'npm.cmd' : 'npm'; + const defaults = { + encoding: 'utf-8', + shell: isWindows, + timeout: 55000, + }; + return execFileSync(npmCmd, args, { ...defaults, ...options }).trim(); +} + +module.exports = { runGsdTools, createTempDir, createTempProject, createTempGitProject, cleanup, parseFrontmatter, isUsageOutput, captureConsole, toPosixPath, runNpm, TOOLS_PATH }; diff --git a/tests/release-tarball-smoke-workflow.test.cjs b/tests/release-tarball-smoke-workflow.test.cjs new file mode 100644 index 000000000..de36e0c0f --- /dev/null +++ b/tests/release-tarball-smoke-workflow.test.cjs @@ -0,0 +1,187 @@ +// allow-test-rule: source-text-is-the-product +// The GitHub Actions YAML is the deployed runtime contract. These tests assert +// on the parsed IR (line-tokenised YAML structure) — not on prose or rendered +// output — to lock the wiring without testing GHA execution semantics. + +'use strict'; + +const { describe, test } = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); + +const WORKFLOW_PATH = path.join(__dirname, '..', '.github', 'workflows', 'install-smoke.yml'); + +/** + * Minimal structural extractor for this specific YAML shape. + * Returns an object with: + * { onTriggers: string[], steps: Array<{ name?: string, run?: string }> } + * + * Strategy: + * 1. Collect top-level `on:` keys by scanning lines after `^on:` until the + * next top-level key. + * 2. Collect step `run:` blocks by scanning all ` - name:` / ` run:` + * entries. No full YAML parse needed — we only need substring presence. + */ +function parseWorkflowStructure(src) { + const lines = src.split('\n'); + + // --- Extract on: trigger keys --- + const onTriggers = []; + let inOn = false; + for (const line of lines) { + if (/^on:/.test(line)) { inOn = true; continue; } + if (inOn) { + // A new top-level key ends the `on:` block + if (/^[a-zA-Z]/.test(line) && !line.startsWith(' ')) { inOn = false; continue; } + // Direct children of `on:` are trigger names at 2-space indent + const m = line.match(/^ ([a-zA-Z_][a-zA-Z0-9_]*):/); + if (m) onTriggers.push(m[1]); + } + } + + // --- Extract PR path filters --- + const prPaths = []; + let inPrPaths = false; + let inPullRequest = false; + for (const line of lines) { + if (/^ pull_request:/.test(line)) { inPullRequest = true; continue; } + if (inPullRequest) { + if (/^ paths:/.test(line)) { inPrPaths = true; continue; } + if (inPrPaths) { + const m = line.match(/^ - '(.+)'/); + if (m) { prPaths.push(m[1]); continue; } + // End of paths list + if (/^ [a-zA-Z]/.test(line)) inPrPaths = false; + } + // End of pull_request block + if (/^ [a-zA-Z]/.test(line) && !line.startsWith(' ')) inPullRequest = false; + } + } + + // --- Extract all step names + run blocks --- + const steps = []; + let currentStep = null; + let inRun = false; + let runLines = []; + + for (const line of lines) { + // Step boundary: 6-space "- name:" or "- uses:" + if (/^ - name:/.test(line)) { + if (currentStep && runLines.length) { + currentStep.run = runLines.join('\n'); + } + if (currentStep) steps.push(currentStep); + currentStep = { name: line.replace(/^ - name:\s*/, '').trim() }; + inRun = false; + runLines = []; + continue; + } + if (/^ - uses:/.test(line)) { + if (currentStep && runLines.length) { + currentStep.run = runLines.join('\n'); + } + if (currentStep) steps.push(currentStep); + currentStep = { uses: line.replace(/^ - uses:\s*/, '').trim() }; + inRun = false; + runLines = []; + continue; + } + // uses: field inside a named step (e.g. "- name: Foo\n uses: actions/...") + if (currentStep && /^ uses:\s/.test(line)) { + currentStep.uses = line.replace(/^ uses:\s*/, '').trim(); + continue; + } + // run: block inside a step + if (currentStep && /^ run:\s*\|/.test(line)) { + inRun = true; + runLines = []; + continue; + } + if (currentStep && /^ run:\s*(?!\|)/.test(line)) { + // Inline run (no |) + currentStep.run = line.replace(/^ run:\s*/, '').trim(); + inRun = false; + continue; + } + if (inRun) { + // Lines deeper than 8 spaces belong to the run block + if (/^ /.test(line) || line.trim() === '') { + runLines.push(line); + } else { + inRun = false; + } + } + } + // Flush final step + if (currentStep) { + if (runLines.length) currentStep.run = runLines.join('\n'); + steps.push(currentStep); + } + + return { onTriggers, prPaths, steps }; +} + +describe('install-smoke.yml structural wiring', () => { + const src = fs.readFileSync(WORKFLOW_PATH, 'utf-8'); + const { onTriggers, prPaths, steps } = parseWorkflowStructure(src); + + test('workflow_call trigger is present (release.yml integration preserved)', () => { + assert.ok( + onTriggers.includes('workflow_call'), + `Expected 'workflow_call' in on: triggers. Found: ${JSON.stringify(onTriggers)}` + ); + }); + + test('lifecycle smoke step calls release-tarball-smoke.cjs', () => { + const smokeStep = steps.find(s => s.run && s.run.includes('release-tarball-smoke.cjs')); + assert.ok( + smokeStep, + 'Expected a step whose run block includes "release-tarball-smoke.cjs". ' + + 'Steps found: ' + JSON.stringify(steps.map(s => s.name || s.uses)) + ); + }); + + test('lifecycle smoke step invokes script with --json flag', () => { + const smokeStep = steps.find(s => s.run && s.run.includes('release-tarball-smoke.cjs')); + assert.ok(smokeStep, 'No lifecycle smoke step found'); + assert.ok( + smokeStep.run.includes('--json'), + `Expected --json in lifecycle smoke run block. Got: ${smokeStep.run}` + ); + }); + + test('lifecycle smoke result is checked via jq', () => { + const smokeStep = steps.find(s => s.run && s.run.includes('release-tarball-smoke.cjs')); + assert.ok(smokeStep, 'No lifecycle smoke step found'); + assert.ok( + smokeStep.run.includes('jq'), + `Expected jq invocation in lifecycle smoke run block. Got: ${smokeStep.run}` + ); + }); + + test('PR path filter includes scripts/release-tarball-smoke.cjs', () => { + assert.ok( + prPaths.includes('scripts/release-tarball-smoke.cjs'), + `Expected 'scripts/release-tarball-smoke.cjs' in PR paths filter. Found: ${JSON.stringify(prPaths)}` + ); + }); + + test('PR path filter includes tests/release-tarball-smoke.install.test.cjs', () => { + assert.ok( + prPaths.includes('tests/release-tarball-smoke.install.test.cjs'), + `Expected 'tests/release-tarball-smoke.install.test.cjs' in PR paths filter. Found: ${JSON.stringify(prPaths)}` + ); + }); + + test('artifact upload step is present for failure debugging', () => { + const uploadStep = steps.find( + s => s.uses && s.uses.startsWith('actions/upload-artifact') + ); + assert.ok( + uploadStep, + 'Expected an actions/upload-artifact step for lifecycle smoke failure debugging. ' + + 'Steps (name + uses): ' + JSON.stringify(steps.map(s => ({ name: s.name, uses: s.uses }))) + ); + }); +}); diff --git a/tests/release-tarball-smoke.install.test.cjs b/tests/release-tarball-smoke.install.test.cjs new file mode 100644 index 000000000..a3d0b4fcd --- /dev/null +++ b/tests/release-tarball-smoke.install.test.cjs @@ -0,0 +1,187 @@ +// allow-test-rule: integration-test-input +// The script under test (scripts/release-tarball-smoke.cjs) is the system +// under test. We exercise it via its exported pure function, not by reading +// source text. The tarball fixture is produced by npm pack in before(). + +'use strict'; + +process.env.GSD_TEST_MODE = '1'; + +const { describe, test, before, after } = require('node:test'); +const assert = require('node:assert/strict'); +const fs = require('node:fs'); +const path = require('node:path'); + +const { cleanup, createTempDir, runNpm } = require('./helpers.cjs'); +const { SMOKE, runSmoke } = require('../scripts/release-tarball-smoke.cjs'); + +const PKG_PATH = path.join(__dirname, '..', 'package.json'); +const pkg = JSON.parse(fs.readFileSync(PKG_PATH, 'utf-8')); + +describe('release-tarball-smoke', () => { + // Shared fixture state: pack the tarball once, install it once, reuse for all tests. + let packDir; + let installPrefix; + let tarballPath; + // fixtureDir for lifecycle / init tests; created once in before(), cleaned in after(). + let fixtureDir; + + before(async () => { + // Pack once into a temp dir. + packDir = createTempDir('gsd-smoke-pack-'); + installPrefix = createTempDir('gsd-smoke-prefix-'); + fixtureDir = createTempDir('gsd-smoke-fixture-'); + + const packOutput = runNpm( + ['pack', '--pack-destination', packDir], + { cwd: path.join(__dirname, '..') }, + ); + + // npm pack prints the filename as the last line of stdout. + const lines = packOutput.split(/\r?\n/).filter(Boolean); + const tgzName = lines[lines.length - 1]; + tarballPath = path.join(packDir, tgzName); + if (!fs.existsSync(tarballPath)) { + const found = fs.readdirSync(packDir).find((f) => f.endsWith('.tgz')); + if (!found) throw new Error(`npm pack produced no .tgz in ${packDir}; output: ${packOutput}`); + tarballPath = path.join(packDir, found); + } + + // Install once into installPrefix. All tests share this install. + runNpm(['install', '-g', '--prefix', installPrefix, tarballPath]); + }); + + after(() => { + cleanup(packDir); + cleanup(installPrefix); + cleanup(fixtureDir); + }); + + // ── Test A — happy path ──────────────────────────────────────────────────── + test('A: happy path — installed version matches package.json', () => { + const result = runSmoke({ + tarballPath, + installPrefix, + expectedVersion: pkg.version, + fixtureDir, + }); + + assert.equal(result.code, SMOKE.OK); + assert.equal(result.details.version, pkg.version); + }); + + // ── Test B — version mismatch detected ──────────────────────────────────── + test('B: version mismatch detected — returns VERSION_MISMATCH', () => { + const result = runSmoke({ + tarballPath, + installPrefix, + expectedVersion: '99.99.99', + fixtureDir, + }); + + assert.equal(result.code, SMOKE.VERSION_MISMATCH); + }); + + // ── Test C — happy lifecycle ─────────────────────────────────────────────── + // Verifies that the installed package has all expected command .md files and + // that each command resolves a workflow .md file that also exists. + // Also verifies that `get-shit-done-cc --local --claude` (init) succeeds in + // the fixtureDir and creates the expected .claude/ directories. + test('C: happy lifecycle — command + workflow files resolve OK', () => { + const result = runSmoke({ + tarballPath, + installPrefix, + expectedVersion: pkg.version, + fixtureDir, + lifecycleCommands: ['init', 'discuss-phase', 'plan-phase'], + }); + + assert.equal(result.code, SMOKE.OK); + + // Each non-init command must be in lifecycleResolved with both paths populated + const resolved = result.details.lifecycleResolved; + assert.ok(Array.isArray(resolved)); + + for (const entry of resolved) { + assert.ok( + typeof entry.commandPath === 'string' && entry.commandPath.length > 0, + `expected commandPath for ${entry.command}`, + ); + assert.ok( + fs.existsSync(entry.commandPath) && fs.statSync(entry.commandPath).isFile(), + `commandPath must be an existing file: ${entry.commandPath}`, + ); + assert.ok( + typeof entry.workflowPath === 'string' && entry.workflowPath.length > 0, + `expected workflowPath for ${entry.command}`, + ); + assert.ok( + fs.existsSync(entry.workflowPath) && fs.statSync(entry.workflowPath).isFile(), + `workflowPath must be an existing file: ${entry.workflowPath}`, + ); + } + }); + + // ── Test D — missing command detected ───────────────────────────────────── + // Passes a nonexistent command name; expects the smoke to detect the missing + // command .md file and return COMMAND_FILE_MISSING with the right details. + test('D: missing command detected — returns COMMAND_FILE_MISSING', () => { + const result = runSmoke({ + tarballPath, + installPrefix, + expectedVersion: pkg.version, + fixtureDir, + lifecycleCommands: ['init', 'nonexistent-phase-xyz'], + }); + + assert.equal(result.code, SMOKE.COMMAND_FILE_MISSING); + assert.equal(result.details.command, 'nonexistent-phase-xyz'); + assert.ok(typeof result.details.path === 'string' && result.details.path.length > 0); + }); + + // ── Test E — SDK binary callable ────────────────────────────────────────── + // Verifies that `gsd-sdk query state.json` exits 0 and returns parseable JSON. + // This is the primary guard for SDK_BINARY_NOT_CALLABLE regressions. + test('E: sdk binary callable — gsd-sdk query produces parseable JSON', () => { + const result = runSmoke({ + tarballPath, + installPrefix, + expectedVersion: pkg.version, + fixtureDir, + lifecycleCommands: [], // skip lifecycle checks; isolate SDK check + }); + + // If the binary can't be called, code would be SDK_BINARY_NOT_CALLABLE + assert.notEqual(result.code, SMOKE.SDK_BINARY_NOT_CALLABLE); + assert.equal(result.details.sdkQueryParsed, true); + }); + + // ── Test F — workflow-body checks run (informational) ───────────────────── + // Asserts that the workflow-body scanning machinery ran (structural assertion). + // Does NOT assert colonLeakCount or missingFallbackCount are zero — they will + // be non-zero against current main per #3668 and the /gsd: leak backlog. + // When those issues are fixed, this test continues to pass unchanged. + test('F: workflow-body checks run — scan counts are present integers', () => { + const result = runSmoke({ + tarballPath, + installPrefix, + expectedVersion: pkg.version, + fixtureDir, + lifecycleCommands: [], + }); + + // Structural: the scan ran and populated the counters + assert.ok( + Number.isInteger(result.details.workflowsScanned) && result.details.workflowsScanned >= 1, + `expected workflowsScanned >= 1, got ${result.details.workflowsScanned}`, + ); + assert.ok( + Number.isInteger(result.details.colonLeakCount), + `expected colonLeakCount to be an integer, got ${result.details.colonLeakCount}`, + ); + assert.ok( + Number.isInteger(result.details.missingFallbackCount), + `expected missingFallbackCount to be an integer, got ${result.details.missingFallbackCount}`, + ); + }); +});