* test(#3882): failing-first rows for sentinel phases skewing calibration Adds A1a/A1b/A2/A3 to tests/estimate-calibrate.test.cjs, the module's existing test file, rather than a new bug-NNNN file. collectCalibrationSamples (src/estimate-cli.cts:206) does a raw readdirSync over .planning/phases and never applies isSentinelPhaseId, so a sentinel phase (milestone 0 or 999) carrying a PLAN estimate / SUMMARY actuals pair contributes a phantom calibration sample. computeCalibration is median-based, so a single 50x outlier among three samples leaves the factor unmoved — asserting "the factor is unchanged" against one sentinel would pass on the broken code for the wrong reason. Each row instead asserts the WHOLE computed CalibrationResult object (factor, applied, confidence, sampleCount, clamped) for a sentinel-free project against its sentinel-injected twin: - A1a: one sentinel flips applied false->true and confidence low->med on phantom evidence (calibration switches on with zero real signal). - A1b: two sentinels corrupt the factor itself (1 -> 3, clamped false->true). - A2: the sentinel's own sample is verified absent from the returned list. - A3: the two genuine phases still contribute their own unchanged samples (regression pin — stops A1/A2 passing by filtering everything). Verified RED on today's code (node tests/estimate-calibrate.test.cjs): A1a/A1b/A2 fail with the exact differing objects; A3 and all pre-existing rows in the file remain green (no collateral). Refs #3882 * feat(#3882): route phase enumeration through its owner and name the sentinel axis Task 1: collectCalibrationSamples (src/estimate-cli.cts) hand-rolled a raw readdirSync over .planning/phases, treating every directory (including sentinel phases, milestone 0/999) as a completed phase and feeding phantom PLAN/SUMMARY samples into the estimation calibration factor. Routed through the existing owner, listMilestonePhaseDirs(phasesRoot) with no cwd -- already 'all milestones, sentinels excluded', exactly the combination this caller needs; no new API was required for this half. It now also surfaces the scope discriminator: an unreadable phases directory throws PhasesUnreadableError instead of silently returning zero samples, and cmdEstimateCalibrate reports it via a new ERROR_REASON.ESTIMATE_PHASES_UNREADABLE instead of persisting a phantom empty calibration document. Task 2: added listAllPhaseDirs(phasesDir, { includeSentinels }) to src/phase-locator.cts -- the one genuinely missing axis: 'physical set, sentinels INCLUDED'. includeSentinels has no default and is required, so a call site cannot obtain sentinel-inclusion by omission (compile-time refusal, not just documentation). Mirrors listMilestonePhaseDirs's absent/unreadable scope handling. Task 3: migrated the two exemptions whose written reason maps cleanly onto 'physical set, sentinels included' -- cmdRoadmapAnalyze's _phaseDirNames (src/roadmap.cts) and cmdInitMilestoneOp's diskPhaseDirs (src/init.cts), both heading->directory lookup indexes. Left the rest: archivePhaseDirectories's own body has no readdirSync to migrate (its callers already resolve dirs before calling it, and both current callers deliberately EXCLUDE sentinels -- migrating it would be an unauthorized behavior change, not an API swap); cmdValidateHealth's exemption is vestigial (its actual physical-set sweep already lives in planning-snapshot.cts's buildAllPhaseDirNamesField, a pre-existing near-duplicate of the new axis, flagged as a finding, not restructured); cmdPhasesClear/cmdMilestoneComplete/cmdVerifySchemaDrift/detectHasPriorPhases/detectUiPhaseActive want a different combination (sentinels excluded, or a single-phase lookup) and are unaffected. Task 4: detector 2 (sentinel literal) is untouched and retained. Removed exemption entries only for the two migrated call sites; every other function-scoped exemption is preserved. Guard exits 0. Refs #3882 * refactor(#3882): delegate the snapshot phase-dir scan to its owner buildAllPhaseDirNamesField duplicated listAllPhaseDirs's own readdirSync + directory-filter + absent/unreadable handling — the 'one implementation per rule' defect ADR-3473 SS8.3 names, introduced by this branch's own #3882 work. Delegate to listAllPhaseDirs and re-apply the field's existing lexicographic sort on top, since W007's observable order must not change. Refs #3882 * docs(#3882): document the sentinel axis and the enumeration consolidation Records listAllPhaseDirs in the Phase Locator glossary entry, and the fact that the owner already answers the all-milestones sentinel-free question when called without a cwd -- the call collectCalibrationSamples was missing. Also notes that buildAllPhaseDirNamesField now delegates rather than carrying a second readdir, and that exactly one readdirSync over the phases directory remains across the two modules. Refs #3882 * test(#3882): close review findings — real order proof, unreadable coverage, collision fixtures Refs #3882 * chore(#3882): backfill changeset PR number Refs #3882 --------- Co-authored-by: sim <sim@local>
9.8 KiB
JSON Error Mode — gsd-tools Structured Errors
Overview
gsd-tools supports a JSON error mode that emits most errors as structured
JSON objects on stderr instead of free-form text. This is the recommended
surface for tests and tooling that need to assert on error types without
grepping raw text (see CONTRIBUTING.md — "Prohibited: Raw Text Matching on
Test Outputs"). Usage errors are an intentional exception — see the
ExitError carve-out below.
This page describes one of two failure channels. A second, equally intentional one reports conditions in the result payload on stdout with exit 0. A caller that branches on exit status alone will not see it. Read Degraded results vs faults before writing anything that consumes
gsd-toolsoutput.
Activating
Either flag or env var activates the mode:
# Flag (preferred in test code):
node gsd-tools.cjs --json-errors <command> [args]
# Env var (preferred for shell wrappers and CI):
GSD_JSON_ERRORS=1 node gsd-tools.cjs <command> [args]
Wire format
On any error, exactly one JSON line is written to stderr and the process exits with code 1:
{ "ok": false, "reason": "<error_code>", "message": "<human text>" }
Fields:
| Field | Type | Description |
|---|---|---|
ok |
false |
Always false for error objects. |
reason |
string | Typed reason code from the taxonomy below. |
message |
string | Human-readable description (may change; do not assert on it). |
ExitError carve-out (plain text, not JSON)
Usage errors and explicit exit-code signals take a different path: they
throw ExitError (src/cli-exit.cts), which runMain catches before the
JSON-envelope branch. An ExitError writes its message as plain text
to stderr (not a JSON object) and exits with the error's own code (which
may differ from 1). This is intentional — usage messages are operator-facing
prose, not structured failures.
If you are testing a usage/flag error, do not parse stderr as JSON;
assert on the exit code and (if needed) the plain-text message. The
"parse stderr as JSON" guidance below applies only to the structured-envelope
branch (non-ExitError failures).
Degraded results vs faults — read this before writing a caller
gsd-tools has two ways of telling you something went wrong, and they use different exit
codes. The wire format above describes only one of them. If you write a caller that branches on
exit status alone, you will silently miss the other.
| Fault | Degraded result | |
|---|---|---|
| Produced by | error(message, reason) |
output({ error: … }) |
| Stream | stderr | stdout |
| Exit code | 1 | 0 |
| Shape | { "ok": false, "reason": …, "message": … } |
the command's ordinary result object, with an added error key |
Honors --json-errors |
yes | no — it is a payload, not an error envelope |
| How a caller detects it | exit code | inspect the payload |
A degraded result means: the command ran to completion and is reporting a condition through its result. It is not a process failure. The command succeeded at the job of determining that, for example, the artifact you asked about is absent.
$ gsd-tools state-snapshot # in a project with no STATE.md
{
"error": "STATE.md not found"
}
$ echo $?
0
Some verbs return a companion result alongside the key, which is the shape that makes the intent clearest:
$ gsd-tools roadmap get-phase --phase 1 # no ROADMAP.md
{
"found": false,
"error": "ROADMAP.md not found"
}
$ echo $?
0
This is a ratified contract, not an accident — see
ADR-2980 for the decision and the blast
radius that drove it. It applies to 60 call sites across nine modules — state, verify,
workstream, frontmatter, commands, template, phase, roadmap, and gsd2-import.
(Issues #2966 and #2980 record this as "42 sites"; that figure counts only the sites where error
happens to be the object's first key. See ADR-2980 for why the real number is 60.)
Writing a correct caller
The obvious shell form is wrong for a degraded result:
# WRONG — the process exits 0, so this branch never runs
if ! gsd-tools state-snapshot > snap.json; then
echo "failed"
fi
Check both channels — the exit code for faults, the payload for degraded results:
if ! out=$(gsd-tools state-snapshot); then
echo "fault (exit non-zero)" >&2 # error() path
exit 1
fi
if err=$(printf '%s' "$out" | jq -er '.error // empty'); then
echo "degraded: $err" >&2 # output({error}) path
fi
Four things that will surprise you
--json-errorsdoes nothing here. It governserror()only. A degraded result is byte-identical with and without the flag, and still exits 0.--rawis not uniform on this path. Most sites pass no raw value, so--rawstill yields the JSON object rather than bare text — but eleven sites do pass one and behave differently. Do not infer either behavior from--rawalone; check the verb.- Not every degraded result is an absent artifact. A missing required argument is reported the
same way —
gsd-tools state add-blockerwith no--textreturns{"error":"text required"}and exits 0. So is unusable input:gsd-tools state advance-planagainst a STATE.md it cannot parse returns{"error":"Cannot parse Current Plan or Total Plans in Phase from STATE.md"}, also exit 0. The exit code does not distinguish absent from malformed from misinvoked — see ADR-2980's Consequences, where this is recorded as a known cost. message/errortext is not stable. Assert on structure and on typedreasoncodes, never on prose. The rule in "Writing tests" below applies to both paths.
Which one should new code use?
Prefer the fault path, or a result with a named field. ADR-2980 ratifies an existing population;
it is not a license to add a 61st output({ error: … }) site. Where a verb needs to report a
non-fatal condition in its payload, prefer the shape state update-progress already uses — a named
field plus a reason, with no overloaded error key:
$ gsd-tools state update-progress # STATE.md present, no Progress field
{
"updated": false,
"reason": "Progress field not found in STATE.md"
}
Error code taxonomy
Codes are frozen constants in gsd-core/bin/lib/core.cjs under
ERROR_REASON. Tests must assert on reason values (stable), not message
text (unstable).
Dispatch errors (gsd-tools routing layer)
| Code | When emitted |
|---|---|
sdk_unknown_command |
Unknown top-level command (gsd-tools bogus-cmd) |
sdk_unknown_command |
Unknown dotted command (gsd-tools foo.bar where foo is not a known command) |
sdk_unknown_command |
Unknown subcommand within a domain (e.g. gsd-tools intel bogus-sub) |
sdk_missing_arg |
Required argument omitted by an SDK-level guard |
sdk_fail_fast |
SDK fail-fast policy triggered |
Usage / flag errors
| Code | When emitted |
|---|---|
usage |
--pick flag used without a following value |
usage |
Version flag (--version, -v) which gsd-tools never accepts |
usage |
Top-level no-args invocation (usage text) |
Config errors (config-get, config-set, config-ensure-section)
| Code | When emitted |
|---|---|
config_key_not_found |
config-get for a key that is absent from the config file |
config_no_file |
Config operation when .planning/config.json does not exist |
config_parse_failed |
Config file exists but is not valid JSON |
config_invalid_key |
config-set for a key outside the allowed whitelist |
Phase / workflow errors
| Code | When emitted |
|---|---|
phase_not_found |
Phase directory lookup returns no match |
summary_no_planning |
Summary operation when no .planning/ directory exists |
Estimate errors
| Code | When emitted |
|---|---|
estimate_phases_unreadable |
estimate-calibrate when .planning/phases/ exists but could not be read (EACCES/EIO) — refused rather than silently rebuilding calibration from a phantom empty sample set (#3882, ADR-3473 §8.5) |
Graphify errors
| Code | When emitted |
|---|---|
graphify_no_graph |
Graphify query or diff when no graph has been built |
graphify_invalid_query |
Graphify query with a malformed query string |
Hook / security errors
| Code | When emitted |
|---|---|
hooks_opt_out |
Hooks are disabled via opt-out config |
security_scan_failed |
Security scan produced a finding that blocks the operation |
Fallback
| Code | When emitted |
|---|---|
unknown |
All other errors without a specific reason code assigned |
Writing tests
For non-usage errors (the structured-envelope branch), parse stderr with
JSON.parse and assert on typed fields. Never use .includes(), .match(),
or regex on the raw error string.
// CORRECT: parse then assert on typed field
const result = runGsdTools(['--json-errors', 'bogus-command'], tmpDir);
assert.strictEqual(result.success, false);
const err = JSON.parse(result.error);
assert.strictEqual(err.ok, false);
assert.strictEqual(err.reason, 'sdk_unknown_command');
// WRONG: text matching (banned by lint-no-source-grep policy)
// assert.ok(result.error.includes('Unknown command'));
Adding a new error code
- Add the constant to
ERROR_REASONingsd-core/bin/lib/core.cjs(snake_case, prefixed by subsystem). - Pass it as the second argument to
error()at the call site. - Add a row to this document.
- Add a test asserting the new
reasoncode viaJSON.parse.