Files

Tom Boucher d24e22b156 enhance(#3912 ): gsd-tools declares outcomes, pinned at v1 (#3983 )

* enhance(#3912): gsd-tools declares outcomes, pinned at v1

ADR-3889 §4. Phase 6 already moved error()'s terminator onto the seam, so what
remained was the declaration — and the pin that makes it invisible today.

The census corrected two documented figures before any code changed.
ERROR_REASON has exactly 25 members (the ADR and epic were right; an earlier
note of mine claiming 23 was wrong and is corrected). And output({error}) is
**64 sites across 9 files, not the 60 ADR-2980 ratified** — the module shape
holds but the total drifted +4: frontmatter 7 not 6, phase 4 not 2, roadmap 3
not 2. That matters because this phase's criterion demands the pin be asserted
over the enumerated population rather than sampled; asserting over a stale 60
would leave four sites unpinned while claiming full coverage, which is the
shape of failure this epic exists to remove.

The issue does not state the fact that shapes the design: output() never
touches the exit code. Confirmed by reading it — it writes fd 1 and returns.
So a declared outcome for those 64 sites had nowhere to be READ. The mapping
was never the work; wiring somewhere for the declaration to land was.

The seam already existed twice over. cli-exit.cts holds two globalThis-Symbol
cells, each because the module is emitted to three locations and a module-level
`let` would let instances disagree, and runMain already maps a code returned by
main(). A third cell inherits that solution. output() records DEGRADED for any
{error} payload — key-order agnostic, which is exactly why the "42 sites"
figure undercounts — and runMain projects the cell only when main() returns
nothing, so an explicit return still wins.

error() maps its reason through a table over the closed 25-member enum, leaving
all 278 call sites untouched; 226 of them pass no reason at all. The version
gate lives in error(), NOT in projectOutcome: registered names are
version-invariant there, so mapping a reason straight through would make USAGE
project to 64 under v1 and break the pin on its first line. projectOutcome is
left exactly as Phase 2 shipped it, DEGRADED's 0/80 asymmetry included.

Proven rather than asserted. v1 is byte-identical across three real CLI paths —
config-get plain, config-get --json-errors, and an output({error}) path —
matching exit code and exact bytes against the pre-change build. Under
GSD_EXIT_CONTRACT=v2 the same commands now exit 66 (CONFIG_KEY_NOT_FOUND ->
NO_INPUT) and 80 (DEGRADED), both looked up through the registry. An
anti-vacuity test pins that v1 and v2 genuinely differ for at least one reason,
because without it a mapping where everything projects to 1 under both versions
would satisfy every other assertion and the declaration would be theatre.

A1 iterates all 25 enum members and A3 asserts over the measured 64-site
population, so a 26th reason or a 65th site fails until it is given a mapping —
the drift guard this phase needs, given ADR-2980's own count had drifted +4
unnoticed.

Verification runs on the remote runner.

Refs #3912

* fix(#3912): the outcome cell must never lower an exit code

The remote run caught a fail-open that this phase introduced, in the phase
whose entire purpose is removing fail-opens.

`state validate --strict` on a missing STATE.md exited **0** where it must exit
1. Mechanism: `runMain` projected the pending outcome whenever `main()` returned
void, and under v1 DEGRADED projects to 0 — so a `process.exitCode` already set
non-zero by the command was clobbered down to success. Confirmed live against a
fixture, before and after.

This refutes a review conclusion recorded earlier in this phase, that the cell
was "fail-closed and can never mask a failure as success". It could, and did.
Recording that plainly so the assumption is not repeated: the cell's danger was
never only that it might add a failure — it was that projecting it
unconditionally overwrites whatever decision came before.

Projection is now guarded: it may set a code only when none is set, and an
already-non-zero exit code always wins. The full precedence — explicit `main()`
return, then an existing non-zero exitCode, then the declared outcome — is
written at the projection site. A regression test drives a void return with a
pre-set non-zero code and a pending DEGRADED, and fails against the pre-fix
build.

The second failure was my test encoding the wrong contract, not a code defect.
It asserted `output({found:false, error: undefined})` records DEGRADED because
the KEY is present. `JSON.stringify` drops undefined, so the payload the user
receives is `{"found":false}` — carrying no error at all, and calling that
degraded would hand back exit 80 under v2 for output that reads as clean. The
discriminator is a serializable error VALUE, not key presence. The test now
pins `{error: undefined}` as explicitly NOT degraded, and the design doc's
wording is tightened to match.

Verification runs on the remote runner.

Refs #3912

* docs(#3912): the versioned exit contract, and a flag defect the docs found

Diataxis pass for Phase 8, plus a real fix that only surfaced because writing
the how-to meant running its own examples.

The docs. ADR-2980's "Revisit if" clause asked for exactly the versioned
projection this phase provides, so it gets an amendment naming #3912 /
ADR-3889 section 4 as that boundary: v1 stays 0 byte-for-byte, v2 projects
DEGRADED to 80. The amendment also records the count drift rather than
restating a stale figure — the ADR ratified 60 output({error}) sites in 9
modules; the AST-measured population is 64 across the same 9 (frontmatter 7
not 6, phase 4 not 2, roadmap 3 not 2). The pin is asserted over the
enumerated 64. json-errors.md gains the outcome-declaration reference,
including the precedence order a review pass got wrong and the suite refuted:
an explicit main() return, then an already-set non-zero process.exitCode, then
the declared outcome. Projection may only ever set a code, never lower one.

A how-to is owed here and is written, not skipped. Under v1 nothing changes,
so the audience is an operator opting into v2 and needing to know what the
codes mean for a CI gate — a migration, which is how-to shaped. It covers
turning v2 on, the code table, why 80 is "ran and reported a condition" rather
than a crash, and how to split a gate that treats any non-zero as fatal. No
tutorial: there is no new entry point to learn, and under the default contract
a reader would be walked through observing nothing.

The defect. Running the how-to's own Step 1 example returned

    $ gsd-tools --exit-contract=v2 state validate --strict
    Error: Unknown command: --exit-contract=v2          (exit 64)

while the same flag trailing the subcommand worked and exited 80. The flag
half-worked, by argv position. resolveContractVersion scans argv
non-destructively, so the token survived into the dispatcher, which treats
argv[2] as the command name. --json-errors had already solved precisely this
at gsd-tools.cjs:4455, under a comment naming the hazard verbatim: "The argv
splice must happen here too, otherwise the dispatcher below sees
--json-errors as an unknown command." The later flag never got the same
treatment.

Fixed rather than documented around: the version is resolved first — which
memoizes the cell and makes an invalid value throw early — and then every
--exit-contract= token is spliced out of the dispatcher's argv copy.
--exit-contract is now listed in TOP_LEVEL_USAGE, where it never was. The
regression test pins leading position, trailing position, agreement between
the two, and a loud failure on v3 rather than a silent fall back to v1.

Neither review engine would have caught this: the defect is invisible in the
diff, because the diff does not touch argv handling. It surfaced only from
running the documentation's own example. Writing a how-to is an execution pass.

Verification runs on the remote runner.

Refs #3912

* fix(#3912): the flag splice has to run before the run-with-timeout return

An isolated review of the previous commit found that the fix did not deliver
what it claimed, and that two of its own tests were weak. All three findings
reproduced by execution before any change was made.

The fix was placed below a return. main() intercepts `run-with-timeout` at
gsd-tools.cjs:4436 and returns from there — above both the --json-errors block
and the --exit-contract splice added in the previous commit. So the flag still
died in leading position for that one command:

    $ gsd-tools --exit-contract=v2 run-with-timeout 5 -- node -e "..."
    Error: Unknown command: run-with-timeout        (exit 64, child never ran)

The previous commit message and the test's describe-block both claimed
position-independence unconditionally. That was an overclaim, not a gap left
open, and it is the part worth naming: the fix was verified by hand on the
commands I happened to think of, and `run-with-timeout` returns before the
code I was verifying.

Both global-flag blocks now run above the interception, with a comment naming
it so a later edit cannot slide them back down. Moving --json-errors up fixes
the identical pre-existing bug for that flag, verified failing beforehand
(exit 1, sdk_unknown_command). Fixing the sibling is deliberate: same defect,
same block, and a known-broken twin next to a fixed one is not a resting state.

Two tests were not pulling their weight. The invalid-value test was vacuous —
it passed against the pre-fix build, because `--exit-contract=v3` already
exited 1 there and already printed the resolve error lazily through
error() -> getContractVersion. Both its assertions held before the fix, so it
pinned nothing. The real discriminator is that the pre-fix build emits BOTH
"Unknown command: --exit-contract=v3" and the resolve error, while the fixed
build emits only the latter; the test now asserts that absence.

The leading-position and leading==trailing tests asserted proxies — "not 64",
"no Unknown command", "the two agree" — none of which pin a value, and all of
which would survive both positions being identically broken. With a .planning
directory and no STATE.md, state-snapshot exits exactly 80 under v2 and 0
under v1 in both positions. Those numbers are pinned now. The multi-token case
the descending splice loop exists for is covered too, and run-with-timeout has
regression tests for both flags.

The lesson is narrower than "test more". Hand-verifying the production
behavior does not verify that the test would have caught its absence. The
pre-fix binary has to be run against the test's own assertions.

Investigated and deliberately not changed: splicing before --cwd parsing
degrades one diagnostic from "Missing value for --cwd" to "Invalid --cwd:
<path>", but that is pre-existing — verified on the pre-fix build via
--json-errors, which already did it. This change joins the pattern rather than
creating it, and both forms exit 64 on malformed input either way.

Verification runs on the remote runner.

Refs #3912

* chore(#3912): backfill changeset pr numbers to 3983

* test(#3912): pin the reason-table invariant as set equality, not a count

A graph-backed review flagged the unchecked lookup in
expectedErrorCode3912. Investigated by execution: the drift guard DOES
hold — for an unmapped reason under v2 the production error() yields 1
while the table yields undefined, so the assertion fails. Not a
correctness defect, and deliberately NOT made tolerant, since a tolerant
lookup would destroy the guard.

Two real problems remained. The guard asserted the wrong invariant: it
counted the TABLE's keys at 25 rather than checking they match the
ENUM's values, so a renamed member keeps the count at 25 and slips past,
and a 26th member leaves the table at 25 and slips past too. Both were
then caught only indirectly, by an undefined mismatch producing 'must
exit undefined'. It is now a sorted set equality, so the failure names
the specific missing or extra reason.

And the comment above it described a '?? FAIL' fallback that does not
exist anywhere in the function. It now states what the code actually
does, verified by running it rather than by reading it.

Refs #3912

---------

Co-authored-by: sim <sim@local>

2026-08-28 08:09:05 -04:00

12 KiB

Raw Blame History

ADR-2980: A payload-carried `error` key is a degraded result, not a fault

Status: Accepted
Date: 2026-08-09
Issue: #2980
Supersedes: —
Relationship to prior work: Resolves the decision ADR-2966 explicitly deferred when its soft-error-exit-zero smell first fired. Constrained by ADR-1411 (resolution must report its provenance) and ADR-227.

Context

gsd-tools reports failure through two different idioms, and they disagree about the exit code.

Idiom	Stream	Exit	Honors `--json-errors`
`error(message, reason)`	stderr	1	yes → `{ok:false,reason,message}`
`output({ error: … })`	stdout	0	no — it is a payload, not an error envelope

The second idiom is used at 60 call sites across nine modules:

Module	Sites	Module	Sites
`src/state.cts`	25	`src/template.cts`	3
`src/verify.cts`	8	`src/gsd2-import.cts`	2
`src/workstream.cts`	7	`src/phase.cts`	2
`src/frontmatter.cts`	6	`src/roadmap.cts`	2
`src/commands.cts`	5	Total	60

On the number 42. #2966 and #2980 both record this population as 42 sites across six modules. That figure counts only the sites where error is the object literal's first key — the shape a line regex such as output\(\{\s*error: matches. Eighteen further sites put another key first (src/roadmap.cts:260 is output({ found: false, error: 'ROADMAP.md not found' }, raw, '')) and are identical in contract. 42 is a real subset, not the size of the contract; the count was re-derived here by brace-matching the first argument rather than by line regex. Do not "correct" 60 back to 42.

Observable today:

$ gsd-tools state-snapshot          # in a project with no STATE.md
{
  "error": "STATE.md not found"
}
$ echo $?
0

docs/json-errors.md described only the first idiom. A reader consulting it would conclude that every gsd-tools error goes to stderr with exit 1, and would write the obvious shell caller:

if ! gsd-tools state-snapshot > snap.json; then
  echo "failed"        # never reached — the process exited 0
fi

The failure is visible only to a caller that parses the payload and knows to look for an error key. Workflows invoke gsd_run <cmd> and branch on exit status, so the gap is not hypothetical.

This surfaced from the loop QA walk (ADR-2966) as a soft-error-exit-zero smell — legal under today's implementation but structurally questionable. That ADR measured the blast radius, declined to act inside a QA-harness ADR, and said the question "warrants a separate, deliberate decision". This is that decision.

Why it is not simply a bug

Many sites show deliberate intent — an error key returned alongside a valid result:

// src/roadmap.cts:310 — cmdRoadmapAnalyze
output({ error: 'ROADMAP.md not found', milestones: [], phases: [], current_phase: null }, raw, undefined);

// src/roadmap.cts:260 — cmdRoadmapGetPhase
output({ found: false, error: 'ROADMAP.md not found' }, raw, '');

Under that reading exit 0 is correct: the command succeeded in determining that the artifact is absent. That matches the project's existing guidance for bounded subprocesses — CONTEXT.md's DEFECT.UNBOUNDED-SUBPROCESS.fix-forward prescribes "on timeout return degraded result + structured warning rather than throw" — and it is the same instinct ADR-1411 encodes: a resolution miss is reported, not thrown.

Decision

The payload-carried error key is a ratified contract, not an accident. It stays. All 60 call sites are unchanged; no code moves.

Precisely, the contract now documented in docs/json-errors.md:

A JSON result on stdout carrying an error key, with exit 0, means the command ran to completion and is reporting a condition through its result. It is not a process failure. A caller that needs to detect it must inspect the payload; the exit code will not tell it, and --json-errors does not apply.

A fault keeps the other path: error(message, reason) → stderr, exit 1, structured envelope under --json-errors. Usage errors keep the third: ExitError → plain text on stderr, its own exit code.

New code should prefer the fault path, or a named-field result. This ADR ratifies an existing population; it is not a license to add a 61st site. Where a verb genuinely needs to report a non-fatal condition in its payload, the richer shape state update-progress already uses — {"updated": false, "reason": "Progress field not found in STATE.md"}, with no overloaded error key — is the better model.

Options declined

Option 2 — split the vocabulary (status: "absent" in place of error). Declined. It buys a cleaner vocabulary at the same compatibility cost as Option 3: any caller already reading .error stops seeing it, and it still requires sweeping every site.

Option 3 — normalize every site to exit 1. Declined on measured blast radius. get_impact rates cmdStateSnapshot — the function this idiom threads through — CRITICAL: ADR-2966 recorded 55 affected symbols across 23 processes, and a re-measure on 2026-08-09 at depth 5 reports ≥200 affected symbols across 41 files and 21 processes. output itself has 170 direct callers. Flipping 0 → 1 across that seam would break every caller currently treating exit 0 as a soft signal.

That is a textbook Hyrum's Law break: the exit-0 behavior is observable, has been in production across 60 sites, and is therefore depended upon whether or not anything promised it. A CLI's exit code has no versioning escape hatch — there is no /v2/ for $?. Hyrum's own prescription for a long-lived observable behavior is to document what is stable, which is what this ADR does.

Consequences

Good.

The idiom is a chosen contract with a written rule, so a caller can be correct on purpose rather than by accident. The gap that made the obvious shell caller wrong is closed at the documentation layer, which is where it existed.
Zero risk. No call site, exit code, or payload shape changes.
ADR-2966's ratchet rule — a smell must terminate in either an assigned defect or a corrected detector — is satisfied through the assigned-defect branch, resolved as "keep".

Costs, stated plainly.

The 60 sites are not a uniform population, and the contract is broader than the motivating example. The issue framed the idiom as "an error key alongside a valid empty result". That shape is real and common — the 18 sites that put another key first are largely it ({found:false, error}) — but it does not describe the whole population. The rest divide into:
- absent artifact — the largest group; a bare {error} or {error, <echo of the input>}, e.g. {"error":"STATE.md not found"};
- missing required argument — at least seven sites, all verified within the error-first subset: src/state.cts 599, 834, 902, 969, 1080 and 2814 ('text required', 'summary required', 'phase, plan, and duration required', 'milestone required (--milestone <vX.Y>)') plus src/template.cts:269 ('File already exists'). These are faults wearing the degraded shape. Under this ADR they are correctly shaped but arguably wrongly classified. "At least" is deliberate — the 18 error-not-first sites were not individually classified;
- unusable input — cmdStateAdvancePlan (src/state.cts:553-585) reports an unparseable STATE.md through the same channel as a missing one.
Reclassifying any of them is a code change and is out of scope here.
Absent and unusable are not distinguishable by exit code. ADR-1411's 2026-07-26 amendment ("corrupt is not absent") requires those classes to stay distinguishable. Today only the message text separates them — and the message is explicitly documented as unstable, so a caller cannot depend on it. This ADR does not close that gap; it names it.
--raw is not uniform on the error path. output(result, raw, rawValue) prints rawValue only when it is not undefined. Most sites pass undefined (eight in src/verify.cts omit the argument entirely), so --raw still hands the caller the JSON error object. Eleven sites pass something else and therefore behave differently under --raw: src/commands.cts 1481, 1546, 1553, 1569, src/phase.cts 246, 692, src/roadmap.cts:260, src/state.cts:436 and :2566 pass '' or 'false'; src/state.cts:2436 passes the message text; src/template.cts:100 passes a template path. A caller cannot assume either behavior from --raw alone.
The soft-error-exit-zero smell keeps firing on state-snapshot and roadmap get-phase, now against behavior this ADR ratifies. Per ADR-2966 §5 a smell never fails a build and never folds into .failed, so this costs nothing but noise — but a detector reporting a ratified contract is reporting a decision, not a finding. Re-pointing it is a code change and is out of scope here.

Revisit if

A caller needs to distinguish absent from unusable programmatically. That is ADR-1411's open edge, and it is the most likely reason this decision gets reopened. The in-band mechanism that ADR already prescribes — naming the cause in a provenance field — would fit these payloads without touching a single exit code, which makes it strictly cheaper than Options 2 and 3.
The missing-required-argument sites (cost 1 above) produce a real caller bug. Those seven are the subset with the weakest claim to exit 0, and they could be moved to error() on their own, with a far smaller radius than the full 60.
A future gsd-tools major version provides a compatibility boundary that a CLI exit code otherwise lacks.

Amendment — 2026-08-27: the compatibility boundary now exists (#3912, ADR-3889 §4)

The third bullet above is answered. ADR-3889 §4 gives gsd-tools a versioned exit-contract projection (v1/v2, selected by --exit-contract=/ GSD_EXIT_CONTRACT=), and #3912 is the phase that lands the projection this idiom's population feeds.

output() now declares an outcome. A payload carrying a serializable error key — any key order, error: undefined excluded because JSON.stringify drops it before the wire — records DEGRADED into the pending-outcome cell runMain reads (src/cli-exit.cts, src/io.cts).
Under v1 — today's default — nothing here changes anything. Every ratified site still exits 0 byte-for-byte; the declaration is recorded but projectOutcome pins DEGRADED -> 0 under v1 specifically so this ADR's Consequences ("no call site, exit code, or payload shape changes") stays true.
Under v2, a caller that opts in gets a real signal. DEGRADED projects to exit code 80 (exitCodeFor('DEGRADED'), ADR-3889's registry) — non-zero, so if ! cmd; then finally trips on this idiom, without moving a single call site or changing the payload shape Option 2/3 above declined to touch.
The population this ADR ratifies is 64, not 60. This ADR's own site count was measured by brace-matching in 2026-08-09; an AST-based re-measure for #3912 finds the same nine modules but 64 output({error}) call sites, not 60 — the drift concentrates in src/frontmatter.cts (7, not 6), src/phase.cts (4, not 2), and src/roadmap.cts (3, not 2). Do not restate 60 as current: the v2 DEGRADED projection above is asserted over the enumerated 64, and any future re-measure that finds a different count should amend this note rather than silently correct the number back.

This does not reopen the Decision above. The 64 sites are still unchanged, still exit 0 under the default contract, and the richer named-field shape (state update-progress's {"updated": false, "reason": …}) is still the recommended shape for new code. What changes is that a caller who explicitly asks for v2 no longer has to parse the payload to notice a degraded result — the exit code says so.

12 KiB Raw Blame History

ADR-2980: A payload-carried error key is a degraded result, not a fault