golem15/msd-core
3
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

enhance(#2295): record the resolved model per reviewer in REVIEWS.md frontmatter (#3649)

Browse Source 
* test(#2295): failing-first coverage for per-lane resolved-model recording

* feat(#2295): record the resolved model per reviewer lane

* docs(#2295): document the recorded reviewer model and its provenance

* fix(#2295): refuse control characters in a recorded model value

* test(#2295): correct watermark assertions for the widened mark shape

* fix(#2295): anchor the role-manipulation injection pattern at a word boundary

* feat(#2295): record the applied reasoning effort in the model value

* chore(#2295): backfill changeset pr number

* chore(#2295): restore em-dash in changeset body

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-08-19 08:59:37 -04:00
committed by GitHub
parent 1adf6d2245
commit 1bf73d957b
15 changed files with 1328 additions and 75 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
docs/COMMANDS.md
View File

@@ -1614,6 +1614,8 @@ Reviewers reached through `--all` or `review.default_reviewers` behave different
**Produces:** `{phase}-REVIEWS.md` — consumable by `/gsd-plan-phase --reviews`
Its frontmatter records the model each reviewer resolved to, as `models:` (the model id, or `unknown`, with a `(reasoning=<level>)` suffix when GSD applied a reasoning effort to that lane) and `model_sources:` (how each value was determined — `pinned`, `served`, `requested`, `banner`, `transcript`, or `unknown`). See [Resolved model recording](CONFIGURATION.md#resolved-model-recording-2295).
```bash
# set project default reviewers for no-flag /gsd-review runs
gsd config-set review.default_reviewers '["gemini","codex"]'

19
docs/CONFIGURATION.md
View File

@@ -240,6 +240,25 @@ The key suffix is **not** always the lane slug. Each lane declares the config ke
| `review.models.opencode` | string | `null` | Model id for OpenCode review (injected into --model), e.g. `"claude-sonnet-4"` |
| `review.models.kimi-code` | string | `null` | Model id for Kimi Code review (injected into -m) |
### Resolved model recording (#2295)
Every `/gsd-review` run records the resolved model per reviewer in the `REVIEWS.md` frontmatter as `models:` and `model_sources:`, whether or not the lane was pinned via the keys above.
| `model_sources` value | Meaning |
|---|---|
| `pinned` | `review.models.<slug>` (or an ADR-1517 reviewer-instance `--model`) that really reached the invocation |
| `served` | An OpenAI-compatible server echoed the model it actually ran. Most authoritative |
| `requested` | openai-http: discovered from `/v1/models`, or the lane's declared `fallbackModel`; the server did not echo one |
| `banner` | The CLI's own startup banner named it. File-output lanes only (`codex` today) |
| `transcript` | The lane handler's own on-disk session log named it (`agy`'s `transcript_full.jsonl`) |
| `unknown` | Nothing recoverable |
A `models:` value reads `unknown` if and only if its `model_sources:` entry is `unknown`.
When GSD applies a reasoning effort to a lane, the recorded value carries it as a
`(reasoning=<level>)` suffix (for example `gpt-5.6-sol (reasoning=high)`) — the level is GSD's
own resolved effort, not the CLI's default.
**Ownership.** These keys are owned by their reviewer-lane capabilities rather than the central
config schema — `review.models.ollama` belongs to the `ollama` capability, `review.ollama_host`
to the same, and so on. Key names and existing `.planning/config.json` files are unchanged; only

6
docs/FEATURES.md
View File

@@ -1270,6 +1270,12 @@ Each reviewer is a **declared lane**: its binary, prompt and output channels, ti
- Use `--all` for a full pre-merge sweep without changing project defaults.
- For local model servers with small context windows, set `review.max_prompt_tokens_per_reviewer` to auto-trim prompts per reviewer — see [Prompt budgets for small-context reviewers](../docs/CONFIGURATION.md#prompt-budgets-for-small-context-reviewers) in CONFIGURATION.md.
**Why record which model produced a review (#2295):** `reviewers:` in the frontmatter recorded which CLIs ran, but not which model each one resolved to. Without a pin, the model is whatever the CLI's own config or internal default happens to pick, so a "Codex vs Antigravity" comparison could quietly be a frontier model against a cheap-tier default with nothing in the record to say so — and a CLI update, or an unrelated config edit, could silently make past and future reviews incomparable.
The fix records the model *and its provenance*. Provenance is what makes the value trustworthy: `pinned` (from `review.models.<slug>`) is certain, while `banner` and `transcript` are recovered from third-party CLI output this project does not own — a startup banner or an undocumented session log.
That third-party dependence is a real trade-off, held honestly rather than papered over: the `banner` and `transcript` arms read formats GSD does not control, so they are best-effort by design and degrade to `unknown` rather than guessing or failing the run. A recorded `unknown` is a real answer — a wrong model name attributed to a review would be worse than none.
---
### 43. Backlog Parking Lot

11
docs/how-to/set-up-cross-ai-review.md
View File

@@ -99,6 +99,17 @@ The `{padded_phase}-REVIEWS.md` file contains:
- Individual reviews from each reviewer with severity-classified concerns
- A **Consensus Summary** section that synthesises concerns raised by two or more reviewers — start here for the highest-priority signal
- A **Divergent Views** section for areas where reviewers disagreed
- `models:` and `model_sources:` frontmatter maps — the resolved model each reviewer actually ran under, and how that value was determined
### Which model produced a review
Compare two reviewers' verdicts only after checking what actually produced each one — `models:` in the frontmatter gives the model per reviewer, and `model_sources:` gives the mechanism that recovered it.
If a reviewer's entry reads `unknown`, pin it: set `review.models.<slug>` for that lane (the key suffix is not always the lane's slug — Antigravity's is `review.models.agy`) so the next run records `pinned`. See [Code-review CLI routing](../CONFIGURATION.md#code-review-cli-routing) for the full key table.
Some `unknown` values are expected, not a bug to chase: lanes that accept no model at all (`cursor`, `qwen`, `coderabbit`), and any lane whose CLI didn't disclose one on this run. `pinned` is a certain value; `banner` and `transcript` are recovered from third-party CLI output and can degrade to `unknown` after an upstream release changes that output.
A `models:` entry like `gpt-5.6-sol (reasoning=high)` is not a formatting quirk: the `(reasoning=<level>)` suffix reflects a reasoning effort GSD itself applied to that lane, driven by your `effort.*` config — not the CLI's own default.
---