fix(#2615): document the effortSurface axis in the host-integration matrix (#2698)

* fix(#2615): document the effortSurface axis in the host-integration matrix

#2481 added `effortSurface` as the ninth negotiated `hostIntegration` axis and
wrote documentation-sourced values into 18 descriptors, but never touched
`docs/reference/host-integration-capability-matrix.md`. The matrix that ADR-1239
designates the cited source of truth had zero occurrences of the axis: no entry in
the axes legend, and no row in any of the per-runtime tables. `src/host-integration.cts`
states "every value is documented or explicitly 'undocumented'" — for this axis
that was false for every runtime.

Adds the legend entry (the `argv` / `none` / `undocumented` vocabulary, plus why
there is deliberately no config-file member) and an `effortSurface` row to all 19
per-runtime tables. Every citation is carried over from #2481's own commit message,
where the values were sourced:

- claude   argv -- `claude --help` documents `--effort <level>`
- opencode argv -- `opencode run --help` documents `--variant`
- codex    argv -- `model_reasoning_effort` is a config.toml key, not a dedicated
                   flag, so the generic `-c key=value` override is the only argv
                   route (still argv)
- 15 hosts undocumented -- their docs state no reasoning setting

kimi-code is the nineteenth section (added by #2603 after #2481) and is the one
runtime with no declared value. Its row and a Documentation-gaps entry record why
rather than inventing one: Kimi Code documents `/effort` (alias `/thinking`), but
only as an INTERACTIVE slash command — `-m, --model` is the only model-adjacent
argv. Neither vocabulary member is accurate (`none` would deny a mechanism the host
has, `argv` would claim one it does not expose), so closing that gap needs a
vocabulary decision, which is a negotiation change and not a documentation one. The
absent value already degrades closed exactly as the sentinel does.

The regression test derives its runtime list from the registry rather than
hardcoding it, so a runtime added later fails until its matrix row exists — the
ratchet whose absence let #2481 add an axis with nothing catching the missing docs.

Closes #2615

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QT3ibz5qJuDuGqpTGRYVGf

* docs(#2615): honest citations for the undocumented rows; fix the four stale 8-axis lists

Two findings from the orthogonal review of the first commit.

1. The 15 `undocumented` rows shared byte-identical text — "searched the runtime's
   official docs (see Sources consulted above)" — which is weaker than this file's
   own convention ("no authoritative doc — searched: <url>") and, worse, implies a
   per-host targeted search that did not happen: each section's Sources-consulted
   list was gathered for OTHER axes and contains no CLI-reference or
   reasoning-effort source. The rows now say plainly what the finding is — an
   ABSENCE established by #2481's cross-host survey — and cite that survey rather
   than implying a URL was checked per host.

2. Four normative docs still described "the eight negotiated axes" and omitted
   effortSurface entirely. The worst of them is
   docs/how-to/add-or-update-a-host-integration.md — the maintainer's own guide for
   onboarding a host, whose Step 2 axis table would have a maintainer reproduce
   exactly the gap #2615 exists to close. Also fixed:
   docs/reference/host-integration-interface.md (which calls itself the normative
   reference and had no effortSurface row at all),
   docs/how-to/author-a-host-plugin.md, docs/registries/README.md ("**exactly** the
   eight … axes keys"), and CONTEXT.md's matching EoS-registry sentence.

Deliberately NOT changed, because they are historical records rather than current
contract: docs/whats-new-1.7.0.md and docs/FEATURES.md's 1.7.0 entry (effortSurface
shipped in 1.8.0 via #2481 — rewriting them would falsify the release history),
ADR-1239's pre-amendment body (already superseded by its own
"Amendment (2026-07-21): effortSurface axis (#2481)"), and ADR-1016's "original
eight axes", which refers to a different axis set entirely.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01QT3ibz5qJuDuGqpTGRYVGf

* chore(#2615): backfill changeset PR number (#2698)

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
2026-07-27 09:02:20 -04:00
committed by GitHub
parent 28e486faf7
commit 1008aabd31
8 changed files with 134 additions and 7 deletions

View File

@@ -0,0 +1,99 @@
// allow-test-rule: runtime-contract-is-the-product #2615 — the host-integration matrix
// IS the cited source of truth for every descriptor axis (ADR-1239); asserting that a
// shipped axis value appears there, and matches, is a contract assertion.
/**
* Regression test for #2615 — `effortSurface` was a shipped `hostIntegration` axis
* with NO presence in the matrix that is supposed to be its cited source of truth.
*
* #2481 added the axis and wrote docs-sourced values into 18 descriptors
* (`claude`/`codex`/`opencode` -> `argv`, 15 others -> `undocumented`) but never
* touched `docs/reference/host-integration-capability-matrix.md`: the axes legend
* omitted it and not one per-runtime table carried a row. `src/host-integration.cts`
* states "every value is documented or explicitly 'undocumented'" — for this axis
* that was false for every runtime.
*
* This test is deliberately GENERIC rather than a hardcoded list: it derives the
* runtimes from the registry, so a runtime added later fails here until its matrix
* row exists. That is the ratchet the original gap needed — #2481 added an axis and
* nothing caught the missing documentation.
*/
'use strict';
process.env.GSD_TEST_MODE = '1';
const { describe, test } = require('node:test');
const assert = require('node:assert/strict');
const fs = require('node:fs');
const path = require('node:path');
const ROOT = path.join(__dirname, '..');
const MATRIX = path.join(ROOT, 'docs', 'reference', 'host-integration-capability-matrix.md');
const registry = require(path.join(ROOT, 'gsd-core', 'bin', 'lib', 'capability-registry.cjs'));
// Normalize CRLF so the row regexes hold on a Windows autocrlf checkout.
const MATRIX_TEXT = fs.readFileSync(MATRIX, 'utf-8').replace(/\r\n/g, '\n');
/** Extract a `## <host>` section body, stopping at the next top-level host heading. */
function section(host) {
const start = MATRIX_TEXT.indexOf(`\n## ${host}\n`);
if (start === -1) return null;
const rest = MATRIX_TEXT.slice(start + 1);
const end = rest.indexOf('\n## ');
return end === -1 ? rest : rest.slice(0, end);
}
/** Read the value cell of a `| <axis> | <value> | …` row. */
function axisValue(body, axis) {
const row = body.split(/\r?\n/).find((l) => l.startsWith(`| ${axis} |`));
return row ? row.split('|')[2].trim() : null;
}
const RUNTIMES = Object.keys(registry.runtimes).filter(
(id) => registry.runtimes[id]?.runtime?.hostIntegration,
);
describe('#2615: the matrix documents the effortSurface axis', () => {
test('the axes legend defines effortSurface and its vocabulary', () => {
const legendRow = MATRIX_TEXT.split(/\r?\n/).find((l) => l.startsWith('| `effortSurface` |'));
assert.ok(legendRow, 'the axes legend must define effortSurface (#2615)');
for (const member of ['`argv`', '`none`', '`undocumented`']) {
assert.ok(legendRow.includes(member),
`the legend must document the ${member} vocabulary member (#2615)`);
}
});
test('there is at least one runtime to check', () => {
// Guards the loops below against silently asserting nothing.
assert.ok(RUNTIMES.length >= 18, `expected the full runtime corpus, got ${RUNTIMES.length}`);
});
for (const id of RUNTIMES) {
describe(`runtime: ${id}`, () => {
test('has a matrix section', () => {
assert.ok(section(id), `${id}: every installed runtime needs a matrix section (ADR-1239)`);
});
test('documents effortSurface, and the value matches the descriptor', () => {
const body = section(id);
assert.ok(body, `${id}: missing matrix section`);
const documented = axisValue(body, 'effortSurface');
assert.ok(documented, `${id}: the matrix must carry an effortSurface row (#2615)`);
const declared = registry.runtimes[id].runtime.hostIntegration.effortSurface;
if (declared === undefined) {
// kimi-code declares no value: its mechanism (`/effort`) is interactive-only
// and neither `argv` nor `none` describes it. The matrix must say so rather
// than invent a value.
assert.match(documented, /not declared/i,
`${id}: an absent descriptor value must be documented as absent, not guessed (#2615)`);
} else {
assert.equal(documented, declared,
`${id}: the matrix effortSurface value must match the shipped descriptor`);
}
});
});
}
});