* chore(#2799): resolve reviewer lanes into executable invocation plans Phase 5b of ADR-2782. Adds the resolver and runner that let invoke_reviewers iterate declared lanes instead of hand-authored per-CLI bash. Five additive descriptor amendments, each forced by a lane that ships today: - LaneHandler gains 'opencode' — the lane rebuilds its review from assistant text parts of a --format json stream; a plain stdout copy re-breaks #1936. - modelConfigKey — antigravity's key is review.models.agy, not .antigravity, so resolving by slug silently dropped a configured model. - defaultHost/fallbackModel — Phase 4 federated every *_host with a default of empty string; the real fallback only existed in the bash. - args becomes an argv template with a closed four-placeholder vocabulary. Positional splicing produced 'codex --model M -o F exec --ephemeral', which is not a valid invocation: codex injects in the middle, twice. - kimi-code lane, with the bounded command-capability probe (needle --output-format) that tells Kimi Code from the legacy python kimi-cli. Parity gate re-pointed: the workflow-text families it scanned are the text this phase deletes, so they are replaced by descriptor-to-registry parity plus an anti-parity check that no bespoke leg returns. jq, curl and external timeout/gtimeout all drop out of the review path. Refs #2782 * chore(#2799): add review-lane query surface and widen the manifest vocabulary Adds the gsd-tools 'review-lane' route (plan/invoke/sections) the workflow loops over, projects all twelve lanes into their capability manifests, and widens capability-validator for the amendments. opencode admitted to VALID_LANE_HANDLERS under the second arm of the enum's own admission rule: one lane, justified by a documented upstream defect data cannot express (#1936 — the agent can end its turn with zero output tokens and --format default then drops the assistant text entirely). Two bugs caught by an end-to-end stub run and fixed here: - loadConfigResolved returns a provenance wrapper, not the config; using it directly resolved every key to undefined, which reads as 'nothing configured' and silently dropped every model override. - hasBinary used shell:true with an args array (Node 26 DEP0190). Replaced with a PATH scan that spawns nothing at all. Refs #2782 * chore(#2799): iterate declared lanes in invoke_reviewers and write_reviews Replaces the eleven hand-authored per-CLI bash legs with a loop over resolved lanes, and renders REVIEWS.md sections from each lane's declared reviewsSection instead of thirteen hardcoded headings. review.md drops from 1104 lines to 507 (61KB to 28.7KB). Parity gate re-pointed, as agreed: the leg-marker and section-heading families scanned exactly the text this phase deletes, so they are replaced by descriptor-to-registry parity in both directions, plus an anti-parity check that fires if a bespoke leg is ever re-added. Enum, emitting sites and the Object.keys lock moved together. The budget-trim helper is hoisted out of the Ollama leg: it was always lane-agnostic, and any lane may now declare a promptBudgetKey. Refs #2782 * feat(#2799): bind the consented egress host and re-verify it at invocation Completes ADR-2782 D5. Rule 1 was recorded in the ADR as delivered by Phase 3 but was not implemented: ConsentRecord had no host field and nothing in the tree bound one, so this phase's rule-4 comparison had no baseline. ConsentRecord gains an OPTIONAL reviewerHost. Optional is the whole design: isValidConsentRecord does not require it, so every record already on disk stays valid and no re-consent storm fires (D4 rule 5). It is deliberately excluded from disclosureSignature — the loader has no config resolver, so folding a config-derived value in would make loader and lifecycle compute different signatures for the same manifest and re-prompt forever. Install resolves hostConfigKey (falling back to the lane's declared defaultHost, which is what the invocation path uses) and records it. Invocation re-resolves and blocks on mismatch rather than silently redirecting. Absence allows: no record, or a record predating the field, means nothing to compare — denying there would break every existing local-model user on upgrade. Refs #2782 * test(#2799): cover the resolver, runner and handlers; retarget the parity suites Adds the golden invocation-plan table (one row per shipped lane, derived from the bash legs rather than the descriptor types) plus runner coverage for the probe, empty-output policy, the three handlers and the egress check. Retargets the existing suites onto the new contract: descriptor-to-registry parity, the anti-parity check, the opencode handler, and the twelfth lane. Two corrections found by running them: - modelConfigKey was required; that breaks D4 rule 2, since a reviewer manifest authored before this phase would fail validation on upgrade. It is optional, read as null when absent. - the antigravity non-zero-exit test pre-seeded the transcript, which asserted that a STALE entry leaks through — the exact bug the watermark prevents. The spawn now appends, as the real tool does. Refs #2782 * fix(#2799): restore agy --add-dir and the self-report prompt in the handler Retargeting the three legacy reviewer suites off the deleted bash surfaced two real regressions in the port, both #2176: - --add-dir was dropped. Without it agy's permission context never receives the cwd repo, so the agent anchors on its own scratch dir and reviews the plan text in isolation — the exact failure the Review Instructions forbid. It is capability-probed, because an older agy rejects the unknown flag outright and a lane that fails to start is worse than one running on the prompt anchor. - the prompt lost the clause mandating a REVIEWED-WITHOUT-REPO-ACCESS self-report, which is what makes a blind review distinguishable from a grounded one. antigravity now builds its own prompt variant. Also ports the #2073 mode-2 cli.log diagnostic, which was dropped: a pinned model that 404s exits 0 with empty stdout AND an empty transcript, so agy's own log is the only evidence that anything failed. The three suites now assert against the plan and the handler instead of matching fence text, so they no longer need allow-test-rule exemptions. Refs #2782 * docs(#2799): document the declared lanes, the new flag, and dropped prerequisites COMMANDS.md gains --kimi-code and replaces the jq-prerequisite paragraph, which is now false: no lane requires jq, curl or an external timeout. Adds the changed-egress-destination behavior, since a blocked lane is something a user can hit. CONFIGURATION.md records that the model config key is declared per lane rather than derived from the flag — antigravity's is review.models.agy — and adds review.models.kimi-code. reviewer-instances.md now routes an instance through its lane's single invocation seam instead of a copied per-adapter bash block, which is what lets a cross-cutting fix reach instances for free. That required implementing the --model/--agent/--as flags it documents; --model re-resolves through the lane's argv template rather than splicing, so the flag lands where the lane declares it rather than ahead of a subcommand. CONTEXT.md glossary gains both new modules. Refs #2782 * chore(#2799): drop the stale emitted-drift acknowledgment The only entry was #2797's, acknowledging COMMENT-ONLY GROWTH in review.md. That file now shrinks by ~32KB and every emitted hash that moved is attributable to this diff, so the ack no longer explains anything. Removing the last entry means removing the file: its presence is the alarm, and an empty one signals nothing. Verified by deleting it and re-running the attribution and provenance gates plus lint:ci — all green without it. Refs #2782 * docs(#2799): record the Phase 5b vocabulary widenings in ADR-2782 Five additive amendments, each forced by a lane that ships today, plus two corrections the phase had to make rather than work around: - D5 rule 1 was recorded as delivered by Phase 3 and was not implemented, so this phase's rule-4 comparison had no baseline. Recorded because an ADR asserting a rule was delivered is exactly what stops a later phase checking. - The DEFECT.GENERATIVE-FIX gate is re-pointed: its workflow-text families scanned the text this phase deletes. Also records that D7's 'skip the probe where no bounding mechanism exists' carve-out is obsolete — in practice it meant the Antigravity lane ran unbounded on every stock macOS host, which ships neither timeout nor gtimeout. Refs #2782 * fix(#2799): close four defects found by adversarial review Two confirmed bugs, both reproduced before fixing: - resolveLanePlan was not total. An openai-http lane with a missing or non-object invoke dereferenced inv.hostConfigKey and threw, contradicting the module's own documented contract; the spawn branch guarded correctly and the http branch did not. The CLI seam resolves every selected lane in one map, so one malformed overlay manifest would have aborted the whole review rather than dropping its own lane. Guarded, plus a per-lane try/catch at the seam so a throw can never take down siblings. - A reviewer-instance model was silently dropped for any lane declaring modelConfigKey null (cursor, qwen, coderabbit). reviewer_instances validates that cli is a known slug but never that the slug accepts a model, so a user could configure one, get a clean run, and never learn a different model reviewed their plan. Now warns explicitly. Two hardening fixes: - The slug is concatenated into artifact paths, so LANE_SLUG_RE is enforced in the resolver rather than inherited from a validator that does not run on this path — the module documents itself as the overlay-manifest trust boundary, so it should not depend on someone else having checked. - normalizeHost mangled a scheme-less value: new URL('localhost:11434') parses with an empty hostname, so it became 'localhost://11434' and was compared and requested as if real. An empty hostname now means not-a-URL. Also documents the one gap that cannot be closed here: the antigravity watermark is keyed by workspace, so two concurrent reviews of the same repo share a transcript. agy exposes no per-invocation id to filter on, so the handler now states which half of its never-stale guarantee actually holds. Refs #2782 * test(#2799): retarget the remaining eight review.md-asserting suites The remote runner found 37 failures the local sweep missed (it hit the shell's two-minute cap before reaching these). All eight extract per-CLI bash from review.md that this phase deletes; each protects a real invariant, so each is retargeted onto the plan, the runner or the handler rather than removed. Three real defects surfaced by doing so: - effort args never reached ANY lane. model-resolver.cjs exports no resolveExecution, so effortFor silently returned [] every time. Restored by calling the same bounded resolve-execution query the bash legs used — and NOT with --raw, which prints the resolved effort rather than the picked field, so claude got 'low' instead of '--effort low'. - the timeout guidance lost 'a silent empty output is a timeout kill, not a crash' — the operator note that exists because of the Codex 0xc0000142 misdiagnosis. Restored. - the opencode handler dropped EMPTY assistant text parts. The shipped jq was , and only substitutes for false/null — an empty string is truthy in jq and contributed a blank line. Found by a property test shrinking to ['', '']. The opencode property suite no longer spawns jq at all, which deletes the #2099 hang mechanism it was architected around rather than mitigating it. Refs #2782 * fix(#2799): register the two new generated modules, and untrack them The remote runner caught build output committed to git. Both new modules compile from src/*.cts into gsd-core/bin/lib/*.cjs, and every sibling generated that way is gitignored and eslint-ignored (ADR-457) - including Phase 1's own review-lane-descriptor.cjs. Mine were neither, so repo-invariants' "each bin/lib/*.cjs is linted xor ignored according to migration state" failed. Registered both in .gitignore and eslint.config.mjs alongside the Phase 1 module, and dropped them from the index. Nothing about the shipped behaviour changes; the artifacts are rebuilt by build:lib. This is the new-.cts-module registration ripple, and it is the one part of it I had not completed - the CONTEXT.md glossary and the inventory manifest were already done. Refs #2782 * chore(#2799): backfill changeset pr number to 2861 * chore(#2799): backfill changeset pr number to 2861 --------- Co-authored-by: Test <test@example.com>
This commit is contained in:
Tom Boucher
@@ -1,224 +1,125 @@
|
||||
// allow-test-rule: source-text-is-the-product (see #1936)
|
||||
// The OpenCode reviewer reconstructs its review from opencode's --format json
|
||||
// event stream using two embedded jq programs in gsd-core/workflows/review.md.
|
||||
// Those programs ARE the runtime contract; this test extracts them verbatim from
|
||||
// the workflow and exercises the real jq (not a reimplementation) so the shipped
|
||||
// reconstruction logic is what gets property-tested.
|
||||
//
|
||||
// ARCHITECTURE (#2099): the shipped jq program is run over a WHOLE fast-check corpus
|
||||
// in ONE jq process, not once per generated case. Each generated event stream is
|
||||
// written as one compact-JSON array per line to a temp file, then `jq -c <PROGRAM>`
|
||||
// (no `-s`) applies PROGRAM to each array — `.` is that array, exactly what
|
||||
// production's `jq -rs` sees after slurping opencode's one-value-per-line stream —
|
||||
// emitting one compact-JSON result per line. This is empirically identical to the
|
||||
// per-stream `-rs` form (verified across embedded-newline/empty/quote/unicode/
|
||||
// null-drop cases) AND reads from a file like production (`jq -rs '…' <file>`), so
|
||||
// there is no stdin pipe to deadlock on large I/O. The prior design spawned ~600
|
||||
// synchronous jq subprocesses (numRuns × 3 properties); a single one freezing on a
|
||||
// contended CI runner hung the whole unit-test chunk to its 600s kill (macOS CI,
|
||||
// #2099). `node --test`'s --test-force-exit cannot interrupt a synchronous
|
||||
// execFileSync, so the cure is to stop spawning per case — not just to time-bound it.
|
||||
/**
|
||||
* OpenCode review reconstruction — property tests (#1936).
|
||||
*
|
||||
* The OpenCode lane rebuilds its review from the assistant `text` parts of a `--format json` event
|
||||
* stream. `--format json` is the PRIMARY invocation, not a fallback: the default formatter drops
|
||||
* the assistant text when the agent ends its turn with no final message, silently losing the
|
||||
* reviewer.
|
||||
*
|
||||
* This suite used to extract two embedded `jq` programs from `review.md` and exercise the real jq
|
||||
* so the shipped logic was what got property-tested. Phase 5b (#2799) replaced those programs with
|
||||
* a named first-party handler in JavaScript, so the property now runs directly against the shipped
|
||||
* function.
|
||||
*
|
||||
* That deletes the entire #2099 problem this file was architected around. The old design spawned
|
||||
* ~600 synchronous jq subprocesses (numRuns × 3 properties), and one freezing on a contended CI
|
||||
* runner hung the whole unit-test chunk to its 600s kill — `--test-force-exit` cannot interrupt a
|
||||
* synchronous `execFileSync`. The batching workaround (one jq process over a whole corpus) existed
|
||||
* solely to avoid that. There is now no subprocess at all, so the hazard is gone by construction
|
||||
* rather than mitigated.
|
||||
*/
|
||||
|
||||
'use strict';
|
||||
|
||||
const { describe, test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const { execFileSync } = require('node:child_process');
|
||||
const fs = require('node:fs');
|
||||
const os = require('node:os');
|
||||
const path = require('node:path');
|
||||
const fc = require('./helpers/fast-check-setup.cjs');
|
||||
const { cleanup } = require('./helpers.cjs');
|
||||
const fc = require('fast-check');
|
||||
|
||||
const reviewPath = path.resolve(__dirname, '..', 'gsd-core', 'workflows', 'review.md');
|
||||
const workflow = fs.readFileSync(reviewPath, 'utf-8');
|
||||
const { handleOpencodeOutput } = require('../gsd-core/bin/lib/review-lane-runner.cjs');
|
||||
|
||||
// Extract the two shipped jq programs verbatim. If review.md changes their shape,
|
||||
// these throw and the test fails loudly (intended coupling — #1936).
|
||||
function extractJqProgram(varName) {
|
||||
const re = new RegExp(`${varName}=\\$\\(jq -rs '([^']*)'`);
|
||||
const m = workflow.match(re);
|
||||
assert.ok(m, `review.md must define ${varName} via jq -rs '<program>' (#1936)`);
|
||||
return m[1];
|
||||
}
|
||||
const TEXT_PROGRAM = extractJqProgram('OPENCODE_REVIEW'); // review reconstruction
|
||||
const DIAG_PROGRAM = extractJqProgram('OPENCODE_DIAG'); // empty-output diagnostic
|
||||
/** Deterministic: pinned seed, bounded runs, replay data printed on failure. */
|
||||
const FC = { seed: 42, numRuns: 200 };
|
||||
|
||||
// This suite shells out to `jq`. On Windows, Node's child_process argument quoting
|
||||
// mangles the jq program (it embeds quotes) — jq then raises a parse error — and
|
||||
// jq isn't guaranteed on the host regardless. The reconstruction logic is
|
||||
// platform-independent (the review workflow runs jq in its Unix-y runtime), so gate
|
||||
// the suite to jq-present non-Windows hosts, mirroring golden-install-parity's win32
|
||||
// skip. The assertions run in full on every macOS/Linux CI leg.
|
||||
let jqAvailable = false;
|
||||
try { execFileSync('jq', ['--version'], { stdio: 'ignore', timeout: 10000, killSignal: 'SIGKILL' }); jqAvailable = true; } catch { /* no jq on PATH */ }
|
||||
const skipReason = process.platform === 'win32'
|
||||
? 'jq invocation is not portable under Node child_process arg-quoting on Windows; logic is platform-independent and asserted on macOS/Linux'
|
||||
: (jqAvailable ? false : 'jq not on PATH');
|
||||
const opts = { skip: skipReason };
|
||||
const NL = String.fromCharCode(10);
|
||||
|
||||
// Bound each jq subprocess so a frozen spawn on a contended runner fails fast +
|
||||
// diagnosably (ETIMEDOUT) rather than hanging the chunk. jq over this corpus
|
||||
// completes in ~10ms, so 30s is an enormous margin that never trips on a healthy run.
|
||||
const JQ_EXEC_OPTS = { encoding: 'utf8', timeout: 30000, killSignal: 'SIGKILL', maxBuffer: 64 * 1024 * 1024 };
|
||||
/** An assistant text event, as opencode emits it. */
|
||||
const textEvent = (text) => ({ type: 'text', part: { text } });
|
||||
/** A terminal step event carrying the stop reason and token counts. */
|
||||
const stepFinish = (reason, output) => ({ type: 'step_finish', part: { reason, tokens: { output } } });
|
||||
|
||||
// Run a shipped jq program over an array of event streams in a SINGLE jq process.
|
||||
// Returns one result per input stream (order preserved), decoded from jq's compact
|
||||
// (`-c`) JSON output back to the raw string production's `-r` would have captured.
|
||||
// Both shipped programs yield exactly one value per stream (join(...) / last|"...");
|
||||
// the length assertion pins that invariant so a future program change that broke it
|
||||
// (0 or >1 outputs) fails loudly instead of silently misaligning results.
|
||||
function runJqBatch(program, streams) {
|
||||
const dir = fs.mkdtempSync(path.join(os.tmpdir(), 'gsd-jq-batch-'));
|
||||
try {
|
||||
const file = path.join(dir, 'streams.jsonl');
|
||||
fs.writeFileSync(file, streams.map((events) => JSON.stringify(events)).join('\n') + '\n');
|
||||
const out = execFileSync('jq', ['-c', program, file], JQ_EXEC_OPTS);
|
||||
const lines = out.split('\n').filter((line) => line.length > 0);
|
||||
assert.equal(
|
||||
lines.length,
|
||||
streams.length,
|
||||
`jq must emit exactly one result per stream (got ${lines.length} for ${streams.length})`,
|
||||
);
|
||||
return lines.map((line) => JSON.parse(line));
|
||||
} finally {
|
||||
cleanup(dir);
|
||||
}
|
||||
}
|
||||
/** Serialize events the way opencode does: one compact JSON value per line. */
|
||||
const streamOf = (events) => events.map((e) => JSON.stringify(e)).join(NL);
|
||||
|
||||
// The intended reconstruction, computed independently of jq. jq is the unit under
|
||||
// test; this JS is the spec it must match on every generated case.
|
||||
function expectedReview(events) {
|
||||
return events
|
||||
.filter((e) => e.type === 'text' && e.part && typeof e.part.text === 'string')
|
||||
.map((e) => e.part.text)
|
||||
.join('\n');
|
||||
}
|
||||
|
||||
// Text values safe to round-trip through JSON → jq (utf8) → string. Excludes lone
|
||||
// surrogates (which don't survive utf8) but keeps the interesting cases: newlines,
|
||||
// quotes, backslashes, braces, unicode.
|
||||
const safeText = fc
|
||||
.string({ minLength: 0, maxLength: 40 })
|
||||
.filter((s) => Buffer.from(s, 'utf8').toString('utf8') === s);
|
||||
|
||||
// A `text` event whose `.part.text` is a string, or null/absent (dropped by `// empty`).
|
||||
const textEvent = fc.record({
|
||||
type: fc.constant('text'),
|
||||
part: fc.oneof(
|
||||
fc.record({ text: safeText }),
|
||||
fc.record({ text: fc.constant(null) }), // null → jq `// empty` drops it
|
||||
fc.record({}), // absent → jq `// empty` drops it
|
||||
),
|
||||
});
|
||||
const stepFinishEvent = fc.record({
|
||||
type: fc.constant('step_finish'),
|
||||
part: fc.record({
|
||||
reason: fc.constantFrom('stop', 'length', 'tool_calls'),
|
||||
tokens: fc.record({ output: fc.integer({ min: 0, max: 100000 }) }),
|
||||
}),
|
||||
});
|
||||
const nonTextEvent = fc.oneof(
|
||||
stepFinishEvent,
|
||||
fc.record({ type: fc.constant('tool_use'), part: fc.record({ tool: safeText }) }),
|
||||
fc.record({ type: fc.constant('step_start'), part: fc.record({}) }),
|
||||
/** Text that survives a JSON round-trip, including the shapes that broke the jq version. */
|
||||
const hostileText = fc.oneof(
|
||||
fc.string(),
|
||||
fc.constantFrom('', ' ', ' ', NL, 'a' + NL + 'b', '"quoted"', '\\backslash', 'emoji 🎉', 'null', '-n'),
|
||||
fc.string({ unit: 'grapheme' }),
|
||||
);
|
||||
// Weight text events higher so streams routinely mix real review text with noise,
|
||||
// but also generate text-free streams (the #1936 zero-output case).
|
||||
const eventStream = fc.array(fc.oneof(textEvent, textEvent, nonTextEvent), {
|
||||
minLength: 1,
|
||||
maxLength: 30,
|
||||
});
|
||||
|
||||
// Corpus size per property. Matches the prior fast-check-setup numRuns:200 so
|
||||
// coverage is unchanged; distinct seeds give each property an independent corpus,
|
||||
// and fixed seeds keep the corpus deterministic across CI runs/OS legs.
|
||||
const CORPUS = 200;
|
||||
|
||||
describe('#1936 OpenCode review reconstruction — jq properties', () => {
|
||||
test('review == the newline-join of every assistant text part, over a generated corpus (order preserved)', opts, () => {
|
||||
const streams = fc.sample(eventStream, { numRuns: CORPUS, seed: 42 });
|
||||
const actual = runJqBatch(TEXT_PROGRAM, streams); // one jq process for the whole corpus
|
||||
streams.forEach((events, i) => {
|
||||
assert.equal(
|
||||
actual[i],
|
||||
expectedReview(events),
|
||||
`case ${i}: shipped jq review must equal the spec for events=${JSON.stringify(events)}`,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
test('a stream with no assistant text part reconstructs to empty (drives the #1936 stub), over a corpus', opts, () => {
|
||||
// The exact failure the bug describes: the agent runs tool calls and ends with
|
||||
// step_finish, emitting no text. Reconstruction must be empty so the content-gate
|
||||
// (`[ -n "$OPENCODE_REVIEW" ]`) falls through to the stub.
|
||||
const streams = fc.sample(fc.array(nonTextEvent, { minLength: 1, maxLength: 20 }), { numRuns: CORPUS, seed: 43 });
|
||||
const actual = runJqBatch(TEXT_PROGRAM, streams);
|
||||
streams.forEach((events, i) => {
|
||||
assert.equal(actual[i], '', `case ${i}: text-free stream must reconstruct to '' for ${JSON.stringify(events)}`);
|
||||
});
|
||||
});
|
||||
|
||||
test('text parts that are null/absent are dropped, never rendered as "null", over a corpus', opts, () => {
|
||||
const nullish = fc.array(
|
||||
fc.oneof(
|
||||
fc.record({ type: fc.constant('text'), part: fc.record({ text: fc.constant(null) }) }),
|
||||
fc.record({ type: fc.constant('text'), part: fc.record({}) }),
|
||||
),
|
||||
{ minLength: 1, maxLength: 10 },
|
||||
describe('opencode reconstruction — properties', () => {
|
||||
test('every assistant text part appears, in order, joined by newlines', () => {
|
||||
fc.assert(
|
||||
fc.property(fc.array(hostileText, { minLength: 1, maxLength: 8 }), (texts) => {
|
||||
const stream = streamOf(texts.map(textEvent));
|
||||
return handleOpencodeOutput(stream).review === texts.join(NL);
|
||||
}),
|
||||
FC,
|
||||
);
|
||||
const streams = fc.sample(nullish, { numRuns: CORPUS, seed: 44 });
|
||||
const actual = runJqBatch(TEXT_PROGRAM, streams);
|
||||
streams.forEach((events, i) => {
|
||||
assert.equal(actual[i], '', `case ${i}: null/absent text must drop to ''`);
|
||||
assert.doesNotMatch(actual[i], /null/, `case ${i}: must never render "null"`);
|
||||
});
|
||||
});
|
||||
|
||||
// Explicit boundary + happy examples (deterministic, not sampled) — all in one
|
||||
// batched jq spawn. Pins the exact contract the corpus only covers probabilistically.
|
||||
test('boundary + happy example streams reconstruct exactly (batched)', opts, () => {
|
||||
const cases = [
|
||||
{ events: [{ type: 'text', part: { text: 'only' } }], expect: 'only' }, // single text part
|
||||
{ events: [{ type: 'text', part: { text: '' } }], expect: '' }, // empty-string text is kept
|
||||
{ events: [{ type: 'text', part: { text: 'a' } }, { type: 'tool_use', part: { tool: 'r' } }, { type: 'text', part: { text: 'b' } }], expect: 'a\nb' }, // text interleaved with noise
|
||||
{ events: [{ type: 'text', part: { text: 'x\ny' } }], expect: 'x\ny' }, // embedded newline preserved
|
||||
{ events: [{ type: 'tool_use', part: { tool: 'read' } }, { type: 'step_finish', part: { reason: 'stop', tokens: { output: 3 } } }], expect: '' }, // no text at all
|
||||
{ events: Array.from({ length: 30 }, (_v, i) => ({ type: 'text', part: { text: `p${i}` } })), expect: Array.from({ length: 30 }, (_v, i) => `p${i}`).join('\n') }, // max-size all-text stream
|
||||
];
|
||||
const actual = runJqBatch(TEXT_PROGRAM, cases.map((c) => c.events));
|
||||
cases.forEach((c, i) => assert.equal(actual[i], c.expect, `example ${i}: ${JSON.stringify(c.events)}`));
|
||||
test('non-text events never contribute to the review', () => {
|
||||
fc.assert(
|
||||
fc.property(
|
||||
fc.array(hostileText, { minLength: 1, maxLength: 5 }),
|
||||
fc.array(fc.record({ reason: fc.string(), output: fc.nat() }), { maxLength: 4 }),
|
||||
(texts, steps) => {
|
||||
const events = [...texts.map(textEvent), ...steps.map((s) => stepFinish(s.reason, s.output))];
|
||||
return handleOpencodeOutput(streamOf(events)).review === texts.join(NL);
|
||||
},
|
||||
),
|
||||
FC,
|
||||
);
|
||||
});
|
||||
|
||||
// Diagnostic path (empty-output stub). The finding calls out `missing .tokens.output`
|
||||
// and no-step_finish as real edges — pin them with examples against the shipped jq,
|
||||
// all in one batched spawn.
|
||||
describe('diagnostic reconstruction (stop reason + output tokens)', () => {
|
||||
test('reports reason/tokens from the LAST step_finish and degrades missing fields to "?"', opts, () => {
|
||||
const cases = [
|
||||
{ events: [
|
||||
{ type: 'step_finish', part: { reason: 'tool_calls', tokens: { output: 5 } } },
|
||||
{ type: 'tool_use', part: {} },
|
||||
{ type: 'step_finish', part: { reason: 'stop', tokens: { output: 0 } } },
|
||||
], expect: 'stop reason=stop, output tokens=0' }, // LAST step_finish wins
|
||||
{ events: [{ type: 'step_finish', part: { reason: 'stop', tokens: {} } }], expect: 'stop reason=stop, output tokens=?' }, // missing .tokens.output → "?"
|
||||
{ events: [{ type: 'tool_use', part: { tool: 'read' } }], expect: 'stop reason=?, output tokens=?' }, // no step_finish at all → both "?"
|
||||
];
|
||||
const actual = runJqBatch(DIAG_PROGRAM, cases.map((c) => c.events));
|
||||
cases.forEach((c, i) => assert.equal(actual[i], c.expect, `diag ${i}: ${JSON.stringify(c.events)}`));
|
||||
});
|
||||
test('a malformed line is skipped without losing the surrounding review', () => {
|
||||
// Losing an entire review to one unparseable line would be strictly worse than the bug this
|
||||
// handler exists to fix, so a partial stream must still yield its usable text.
|
||||
fc.assert(
|
||||
fc.property(
|
||||
fc.array(hostileText, { minLength: 1, maxLength: 4 }),
|
||||
fc.constantFrom('NOT JSON', '{"truncated":', '}{', '[', 'null', ' '),
|
||||
(texts, garbage) => {
|
||||
const lines = [...texts.map((t) => JSON.stringify(textEvent(t))), garbage];
|
||||
return handleOpencodeOutput(lines.join(NL)).review === texts.join(NL);
|
||||
},
|
||||
),
|
||||
FC,
|
||||
);
|
||||
});
|
||||
|
||||
// The primary reconstruction runs before any content gate; on non-JSON stdout
|
||||
// (e.g. an opencode crash that printed a plain-text error) jq must fail rather
|
||||
// than emit that text as a "review" — the workflow's `2>/dev/null` + empty
|
||||
// capture then routes to the diagnostic stub.
|
||||
test('non-JSON stdout does not masquerade as a reconstructed review', opts, () => {
|
||||
let threw = false;
|
||||
try {
|
||||
execFileSync('jq', ['-rs', TEXT_PROGRAM], { ...JQ_EXEC_OPTS, input: 'auth token expired\n' });
|
||||
} catch {
|
||||
threw = true;
|
||||
}
|
||||
assert.ok(threw, 'jq must reject non-JSON input so it cannot be captured as a review');
|
||||
test('is total — never throws on arbitrary input', () => {
|
||||
fc.assert(
|
||||
fc.property(fc.anything(), (anything) => {
|
||||
try {
|
||||
const r = handleOpencodeOutput(anything);
|
||||
return typeof r.review === 'string' && typeof r.diagnostic === 'string';
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}),
|
||||
FC,
|
||||
);
|
||||
});
|
||||
|
||||
test('CRLF streams reconstruct identically to LF', () => {
|
||||
fc.assert(
|
||||
fc.property(fc.array(fc.string({ minLength: 1 }), { minLength: 1, maxLength: 5 }), (texts) => {
|
||||
const lf = streamOf(texts.map(textEvent));
|
||||
const crlf = lf.split(NL).join(String.fromCharCode(13) + NL);
|
||||
return handleOpencodeOutput(lf).review === handleOpencodeOutput(crlf).review;
|
||||
}),
|
||||
FC,
|
||||
);
|
||||
});
|
||||
|
||||
test('a stream with no assistant text yields an empty review and a diagnostic', () => {
|
||||
fc.assert(
|
||||
fc.property(fc.string({ minLength: 1 }), fc.nat(), (reason, output) => {
|
||||
const r = handleOpencodeOutput(streamOf([stepFinish(reason, output)]));
|
||||
return r.review === '' && r.diagnostic.includes(String(output));
|
||||
}),
|
||||
FC,
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
Reference in New Issue
Block a user