Files
msd-core/docs/json-errors.md
Tom Boucher a638ca4332 enhance(#3882): stop sentinel phases skewing estimation calibration (#3893)
* test(#3882): failing-first rows for sentinel phases skewing calibration

Adds A1a/A1b/A2/A3 to tests/estimate-calibrate.test.cjs, the module's
existing test file, rather than a new bug-NNNN file. collectCalibrationSamples
(src/estimate-cli.cts:206) does a raw readdirSync over .planning/phases and
never applies isSentinelPhaseId, so a sentinel phase (milestone 0 or 999)
carrying a PLAN estimate / SUMMARY actuals pair contributes a phantom
calibration sample.

computeCalibration is median-based, so a single 50x outlier among three
samples leaves the factor unmoved — asserting "the factor is unchanged"
against one sentinel would pass on the broken code for the wrong reason.
Each row instead asserts the WHOLE computed CalibrationResult object
(factor, applied, confidence, sampleCount, clamped) for a sentinel-free
project against its sentinel-injected twin:

- A1a: one sentinel flips applied false->true and confidence low->med on
  phantom evidence (calibration switches on with zero real signal).
- A1b: two sentinels corrupt the factor itself (1 -> 3, clamped false->true).
- A2: the sentinel's own sample is verified absent from the returned list.
- A3: the two genuine phases still contribute their own unchanged samples
  (regression pin — stops A1/A2 passing by filtering everything).

Verified RED on today's code (node tests/estimate-calibrate.test.cjs):
A1a/A1b/A2 fail with the exact differing objects; A3 and all pre-existing
rows in the file remain green (no collateral).

Refs #3882

* feat(#3882): route phase enumeration through its owner and name the sentinel axis

Task 1: collectCalibrationSamples (src/estimate-cli.cts) hand-rolled a raw readdirSync over .planning/phases, treating every directory (including sentinel phases, milestone 0/999) as a completed phase and feeding phantom PLAN/SUMMARY samples into the estimation calibration factor. Routed through the existing owner, listMilestonePhaseDirs(phasesRoot) with no cwd -- already 'all milestones, sentinels excluded', exactly the combination this caller needs; no new API was required for this half. It now also surfaces the scope discriminator: an unreadable phases directory throws PhasesUnreadableError instead of silently returning zero samples, and cmdEstimateCalibrate reports it via a new ERROR_REASON.ESTIMATE_PHASES_UNREADABLE instead of persisting a phantom empty calibration document.

Task 2: added listAllPhaseDirs(phasesDir, { includeSentinels }) to src/phase-locator.cts -- the one genuinely missing axis: 'physical set, sentinels INCLUDED'. includeSentinels has no default and is required, so a call site cannot obtain sentinel-inclusion by omission (compile-time refusal, not just documentation). Mirrors listMilestonePhaseDirs's absent/unreadable scope handling.

Task 3: migrated the two exemptions whose written reason maps cleanly onto 'physical set, sentinels included' -- cmdRoadmapAnalyze's _phaseDirNames (src/roadmap.cts) and cmdInitMilestoneOp's diskPhaseDirs (src/init.cts), both heading->directory lookup indexes. Left the rest: archivePhaseDirectories's own body has no readdirSync to migrate (its callers already resolve dirs before calling it, and both current callers deliberately EXCLUDE sentinels -- migrating it would be an unauthorized behavior change, not an API swap); cmdValidateHealth's exemption is vestigial (its actual physical-set sweep already lives in planning-snapshot.cts's buildAllPhaseDirNamesField, a pre-existing near-duplicate of the new axis, flagged as a finding, not restructured); cmdPhasesClear/cmdMilestoneComplete/cmdVerifySchemaDrift/detectHasPriorPhases/detectUiPhaseActive want a different combination (sentinels excluded, or a single-phase lookup) and are unaffected.

Task 4: detector 2 (sentinel literal) is untouched and retained. Removed exemption entries only for the two migrated call sites; every other function-scoped exemption is preserved. Guard exits 0.

Refs #3882

* refactor(#3882): delegate the snapshot phase-dir scan to its owner

buildAllPhaseDirNamesField duplicated listAllPhaseDirs's own
readdirSync + directory-filter + absent/unreadable handling — the
'one implementation per rule' defect ADR-3473 SS8.3 names, introduced
by this branch's own #3882 work. Delegate to listAllPhaseDirs and
re-apply the field's existing lexicographic sort on top, since W007's
observable order must not change.

Refs #3882

* docs(#3882): document the sentinel axis and the enumeration consolidation

Records listAllPhaseDirs in the Phase Locator glossary entry, and the fact
that the owner already answers the all-milestones sentinel-free question when
called without a cwd -- the call collectCalibrationSamples was missing.

Also notes that buildAllPhaseDirNamesField now delegates rather than carrying a
second readdir, and that exactly one readdirSync over the phases directory
remains across the two modules.

Refs #3882

* test(#3882): close review findings — real order proof, unreadable coverage, collision fixtures

Refs #3882

* chore(#3882): backfill changeset PR number

Refs #3882

---------

Co-authored-by: sim <sim@local>
2026-08-26 15:03:12 -04:00

9.8 KiB

JSON Error Mode — gsd-tools Structured Errors

Overview

gsd-tools supports a JSON error mode that emits most errors as structured JSON objects on stderr instead of free-form text. This is the recommended surface for tests and tooling that need to assert on error types without grepping raw text (see CONTRIBUTING.md — "Prohibited: Raw Text Matching on Test Outputs"). Usage errors are an intentional exception — see the ExitError carve-out below.

This page describes one of two failure channels. A second, equally intentional one reports conditions in the result payload on stdout with exit 0. A caller that branches on exit status alone will not see it. Read Degraded results vs faults before writing anything that consumes gsd-tools output.

Activating

Either flag or env var activates the mode:

# Flag (preferred in test code):
node gsd-tools.cjs --json-errors <command> [args]

# Env var (preferred for shell wrappers and CI):
GSD_JSON_ERRORS=1 node gsd-tools.cjs <command> [args]

Wire format

On any error, exactly one JSON line is written to stderr and the process exits with code 1:

{ "ok": false, "reason": "<error_code>", "message": "<human text>" }

Fields:

Field Type Description
ok false Always false for error objects.
reason string Typed reason code from the taxonomy below.
message string Human-readable description (may change; do not assert on it).

ExitError carve-out (plain text, not JSON)

Usage errors and explicit exit-code signals take a different path: they throw ExitError (src/cli-exit.cts), which runMain catches before the JSON-envelope branch. An ExitError writes its message as plain text to stderr (not a JSON object) and exits with the error's own code (which may differ from 1). This is intentional — usage messages are operator-facing prose, not structured failures.

If you are testing a usage/flag error, do not parse stderr as JSON; assert on the exit code and (if needed) the plain-text message. The "parse stderr as JSON" guidance below applies only to the structured-envelope branch (non-ExitError failures).

Degraded results vs faults — read this before writing a caller

gsd-tools has two ways of telling you something went wrong, and they use different exit codes. The wire format above describes only one of them. If you write a caller that branches on exit status alone, you will silently miss the other.

Fault Degraded result
Produced by error(message, reason) output({ error: … })
Stream stderr stdout
Exit code 1 0
Shape { "ok": false, "reason": …, "message": … } the command's ordinary result object, with an added error key
Honors --json-errors yes no — it is a payload, not an error envelope
How a caller detects it exit code inspect the payload

A degraded result means: the command ran to completion and is reporting a condition through its result. It is not a process failure. The command succeeded at the job of determining that, for example, the artifact you asked about is absent.

$ gsd-tools state-snapshot          # in a project with no STATE.md
{
  "error": "STATE.md not found"
}
$ echo $?
0

Some verbs return a companion result alongside the key, which is the shape that makes the intent clearest:

$ gsd-tools roadmap get-phase --phase 1      # no ROADMAP.md
{
  "found": false,
  "error": "ROADMAP.md not found"
}
$ echo $?
0

This is a ratified contract, not an accident — see ADR-2980 for the decision and the blast radius that drove it. It applies to 60 call sites across nine modules — state, verify, workstream, frontmatter, commands, template, phase, roadmap, and gsd2-import. (Issues #2966 and #2980 record this as "42 sites"; that figure counts only the sites where error happens to be the object's first key. See ADR-2980 for why the real number is 60.)

Writing a correct caller

The obvious shell form is wrong for a degraded result:

# WRONG — the process exits 0, so this branch never runs
if ! gsd-tools state-snapshot > snap.json; then
  echo "failed"
fi

Check both channels — the exit code for faults, the payload for degraded results:

if ! out=$(gsd-tools state-snapshot); then
  echo "fault (exit non-zero)" >&2      # error() path
  exit 1
fi
if err=$(printf '%s' "$out" | jq -er '.error // empty'); then
  echo "degraded: $err" >&2             # output({error}) path
fi

Four things that will surprise you

  1. --json-errors does nothing here. It governs error() only. A degraded result is byte-identical with and without the flag, and still exits 0.
  2. --raw is not uniform on this path. Most sites pass no raw value, so --raw still yields the JSON object rather than bare text — but eleven sites do pass one and behave differently. Do not infer either behavior from --raw alone; check the verb.
  3. Not every degraded result is an absent artifact. A missing required argument is reported the same way — gsd-tools state add-blocker with no --text returns {"error":"text required"} and exits 0. So is unusable input: gsd-tools state advance-plan against a STATE.md it cannot parse returns {"error":"Cannot parse Current Plan or Total Plans in Phase from STATE.md"}, also exit 0. The exit code does not distinguish absent from malformed from misinvoked — see ADR-2980's Consequences, where this is recorded as a known cost.
  4. message/error text is not stable. Assert on structure and on typed reason codes, never on prose. The rule in "Writing tests" below applies to both paths.

Which one should new code use?

Prefer the fault path, or a result with a named field. ADR-2980 ratifies an existing population; it is not a license to add a 61st output({ error: … }) site. Where a verb needs to report a non-fatal condition in its payload, prefer the shape state update-progress already uses — a named field plus a reason, with no overloaded error key:

$ gsd-tools state update-progress            # STATE.md present, no Progress field
{
  "updated": false,
  "reason": "Progress field not found in STATE.md"
}

Error code taxonomy

Codes are frozen constants in gsd-core/bin/lib/core.cjs under ERROR_REASON. Tests must assert on reason values (stable), not message text (unstable).

Dispatch errors (gsd-tools routing layer)

Code When emitted
sdk_unknown_command Unknown top-level command (gsd-tools bogus-cmd)
sdk_unknown_command Unknown dotted command (gsd-tools foo.bar where foo is not a known command)
sdk_unknown_command Unknown subcommand within a domain (e.g. gsd-tools intel bogus-sub)
sdk_missing_arg Required argument omitted by an SDK-level guard
sdk_fail_fast SDK fail-fast policy triggered

Usage / flag errors

Code When emitted
usage --pick flag used without a following value
usage Version flag (--version, -v) which gsd-tools never accepts
usage Top-level no-args invocation (usage text)

Config errors (config-get, config-set, config-ensure-section)

Code When emitted
config_key_not_found config-get for a key that is absent from the config file
config_no_file Config operation when .planning/config.json does not exist
config_parse_failed Config file exists but is not valid JSON
config_invalid_key config-set for a key outside the allowed whitelist

Phase / workflow errors

Code When emitted
phase_not_found Phase directory lookup returns no match
summary_no_planning Summary operation when no .planning/ directory exists

Estimate errors

Code When emitted
estimate_phases_unreadable estimate-calibrate when .planning/phases/ exists but could not be read (EACCES/EIO) — refused rather than silently rebuilding calibration from a phantom empty sample set (#3882, ADR-3473 §8.5)

Graphify errors

Code When emitted
graphify_no_graph Graphify query or diff when no graph has been built
graphify_invalid_query Graphify query with a malformed query string

Hook / security errors

Code When emitted
hooks_opt_out Hooks are disabled via opt-out config
security_scan_failed Security scan produced a finding that blocks the operation

Fallback

Code When emitted
unknown All other errors without a specific reason code assigned

Writing tests

For non-usage errors (the structured-envelope branch), parse stderr with JSON.parse and assert on typed fields. Never use .includes(), .match(), or regex on the raw error string.

// CORRECT: parse then assert on typed field
const result = runGsdTools(['--json-errors', 'bogus-command'], tmpDir);
assert.strictEqual(result.success, false);
const err = JSON.parse(result.error);
assert.strictEqual(err.ok, false);
assert.strictEqual(err.reason, 'sdk_unknown_command');

// WRONG: text matching (banned by lint-no-source-grep policy)
// assert.ok(result.error.includes('Unknown command'));

Adding a new error code

  1. Add the constant to ERROR_REASON in gsd-core/bin/lib/core.cjs (snake_case, prefixed by subsystem).
  2. Pass it as the second argument to error() at the call site.
  3. Add a row to this document.
  4. Add a test asserting the new reason code via JSON.parse.