* test(#2943): parity guard against the nonexistent get-library-docs tool Second context7 naming drift after #2017 (which guarded the plugin-marketplace PREFIX). #2017's guard only checks tools: frontmatter lines, not prose bodies — which is where the broken tool NAME (get-library-docs) lived. The context7 MCP server registers only resolve-library-id and query-docs; get-library-docs is a stale copy from upstream's own README. Scans the shipped prose surface (agents/, gsd-core/references|workflows/, commands/gsd/, skills/) and fails if any artifact instructs an agent to call mcp__context7__get-library-docs. Excludes tests/ (a fixture may use the name as a negative input) and CHANGELOG/RELEASE-NOTES-LEGACY (history). Fails-first: 4 offenders today (gsd-executor.md:29, research-documentation-lookup.md:5, discovery-phase.md:68 & :104). * fix(#2943): rename get-library-docs to query-docs and correct the ctx7 fallback rationale The context7 MCP server registers only resolve-library-id and query-docs (verified against upstream packages/mcp/src/index.ts); get-library-docs is a stale name copied from upstream's own README. Four shipped prose sites instructed agents to call a tool the server does not register, so every research path that loaded the canonical reference either errored, fell through to the ctx7 CLI branch, or fabricated a result. - research-documentation-lookup.md, gsd-executor.md, discovery-phase.md (x2): get-library-docs -> query-docs, params context7CompatibleLibraryId/topic -> libraryId/query (the registered contract). - Same files' ctx7 CLI fallback rationale: the cited cause (anthropics/claude-code#13898 'strips MCP tools from agents with a tools: frontmatter restriction') was wrong on two counts — #13898 is closed and was never about tools: frontmatter. Rewritten to describe the real mechanism (custom subagents cannot see project-scoped .mcp.json; they only inherit user-scoped ~/.claude/mcp.json). The fallback itself is kept. - discovery-phase.md 'mode: code/info' dropped — query-docs takes libraryId + query only; the code-vs-concepts intent is now expressed via the query text. resolve-library-id is unchanged (still registered upstream). CHANGELOG and RELEASE-NOTES-LEGACY citations are historical record, left as-is. * chore(#2943): add changeset fragment (pr:0 placeholder) * test(#2943): widen parity-guard scan surface to docs/ (isolated-review finding) The isolated adversarial review flagged that SCAN_DIRS omitted docs/, which ships docs/AGENTS.md — agent-consumed prose carrying 8 mcp__context7__* refs. No false negative today (it uses only the wildcard), but a future banned-name addition there would slip through, recreating the exact drift this guard exists to prevent. Add docs/ to the scan surface, with an EXCLUDED_FILES set for historical record (docs/RELEASE-NOTES-LEGACY.md, CHANGELOG.md) that must not be rewritten to satisfy the guard. * fix(#2943): update shifted PROSE_ALLOWLIST line + acknowledge gsd-executor.md growth The gsd-test gate caught two real consequences of the rationale rewrite in agents/gsd-executor.md (the +2-line corrected mechanism description shifted line numbers below it): 1. tests/no-bare-gsd-tools-command-position.test.cjs: the legitimate 'gsd-tools query commit' descriptive mention moved from line 791 -> 793. Update the PROSE_ALLOWLIST entry to the new line (the mention is unchanged, just relocated by my edit above it). Without this the gate reports both a stale allowlist entry (791) and a new offender (793) for the same mention. 2. tests/emitted-drift-acks/2943-context7-tool-name.json: gsd-executor.md grew 95 bytes (the accurate mechanism rationale is longer than the wrong one-line #13898 attribution it replaces). Acknowledge the growth with the reason. Both are mandated by the gate, not optional. The rename itself (get-library-docs -> query-docs) is byte-neutral-ish; only the rationale rewrite grew the file. * chore(#2943): backfill changeset PR number 2963 --------- Co-authored-by: sim <sim@local>
This commit is contained in:
5
.changeset/vivid-ibex-hop.md
Normal file
5
.changeset/vivid-ibex-hop.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Fixed
|
||||
pr: 2963
|
||||
---
|
||||
**Research agents no longer call a context7 tool that doesn't exist** — four shipped docs instructed agents to call `mcp__context7__get-library-docs`, a tool the context7 MCP server does not register (it exposes only `resolve-library-id` and `query-docs`). Every research workflow that loaded the canonical doc-lookup reference either errored, fell back to the `ctx7` CLI, or fabricated a result. All sites now name `query-docs` with the registered `libraryId`/`query` params, the CLI-fallback rationale now describes the real project-scoped `.mcp.json` mechanism, and a parity guard fails the build if the banned name returns. (#2943)
|
||||
@@ -26,10 +26,12 @@ When you need library or framework documentation, check in this order:
|
||||
|
||||
1. If Context7 MCP tools (`mcp__context7__*, mcp__plugin_context7_context7__*`) are available in your environment, use them:
|
||||
- Resolve library ID: `mcp__context7__resolve-library-id` with `libraryName`
|
||||
- Fetch docs: `mcp__context7__get-library-docs` with `context7CompatibleLibraryId` and `topic`
|
||||
- Fetch docs: `mcp__context7__query-docs` with `libraryId` (the ID from step 1) and `query`
|
||||
|
||||
2. If Context7 MCP is not available (upstream bug anthropics/claude-code#13898 strips MCP
|
||||
tools from agents with a `tools:` frontmatter restriction), use the CLI fallback via Bash:
|
||||
2. If Context7 MCP is not available (custom subagents cannot see project-scoped
|
||||
`.mcp.json` servers — they only inherit user-scoped `~/.claude/mcp.json`, so a
|
||||
context7 server configured at the project scope is invisible to spawned
|
||||
agents), use the CLI fallback via Bash:
|
||||
|
||||
Step 1 — Resolve library ID:
|
||||
```bash
|
||||
|
||||
@@ -2,10 +2,12 @@ When you need library or framework documentation, check in this order:
|
||||
|
||||
1. If Context7 MCP tools (`mcp__context7__*`) are available in your environment, use them:
|
||||
- Resolve library ID: `mcp__context7__resolve-library-id` with `libraryName`
|
||||
- Fetch docs: `mcp__context7__get-library-docs` with `context7CompatibleLibraryId` and `topic`
|
||||
- Fetch docs: `mcp__context7__query-docs` with `libraryId` (the ID from step 1) and `query`
|
||||
|
||||
2. If Context7 MCP is not available (upstream bug anthropics/claude-code#13898 strips MCP
|
||||
tools from agents with a `tools:` frontmatter restriction), use the CLI fallback via Bash:
|
||||
2. If Context7 MCP is not available (custom subagents cannot see project-scoped
|
||||
`.mcp.json` servers — they only inherit user-scoped `~/.claude/mcp.json`, so a
|
||||
context7 server configured at the project scope is invisible to spawned
|
||||
agents), use the CLI fallback via Bash:
|
||||
|
||||
Step 1 — Resolve library ID:
|
||||
```bash
|
||||
|
||||
@@ -65,9 +65,9 @@ For: Single known library, confirming syntax/version still correct.
|
||||
2. Fetch relevant docs:
|
||||
|
||||
```
|
||||
mcp__context7__get-library-docs with:
|
||||
- context7CompatibleLibraryID: [from step 1]
|
||||
- topic: [specific concern]
|
||||
mcp__context7__query-docs with:
|
||||
- libraryId: [from step 1]
|
||||
- query: [specific concern]
|
||||
```
|
||||
|
||||
3. Verify:
|
||||
@@ -101,7 +101,7 @@ For: Choosing between options, new external integration.
|
||||
```
|
||||
For each library/framework:
|
||||
- mcp__context7__resolve-library-id
|
||||
- mcp__context7__get-library-docs (mode: "code" for API, "info" for concepts)
|
||||
- mcp__context7__query-docs (frame the `query` for API usage vs concepts)
|
||||
```
|
||||
|
||||
3. **Official docs** for anything Context7 lacks.
|
||||
|
||||
113
tests/context7-tool-name-parity.test.cjs
Normal file
113
tests/context7-tool-name-parity.test.cjs
Normal file
@@ -0,0 +1,113 @@
|
||||
'use strict';
|
||||
|
||||
// Regression guard for #2943 — the context7 MCP server registers exactly two
|
||||
// tools (`resolve-library-id` and `query-docs`); it does NOT register
|
||||
// `get-library-docs`. That stale name survived in shipped prose (copied from
|
||||
// upstream's own stale README), so agents were instructed to call a tool the
|
||||
// server does not register — they errored, fell through to the ctx7 CLI
|
||||
// fallback, or fabricated a result.
|
||||
//
|
||||
// This is the SECOND context7 naming drift after #2017 (which guarded the
|
||||
// plugin-marketplace PREFIX). #2017's guard (context7-plugin-grant-parity) only
|
||||
// checks `tools:` frontmatter lines; it does not scan prose bodies, which is
|
||||
// where the broken tool NAME lived. This guard scans the shipped prose surface
|
||||
// and fails if any artifact tells an agent to call the nonexistent tool.
|
||||
//
|
||||
// Allowed (registered upstream): mcp__context7__resolve-library-id,
|
||||
// mcp__context7__query-docs, and the mcp__context7__* / plugin-marketplace
|
||||
// grants. Banned: mcp__context7__get-library-docs.
|
||||
|
||||
const { describe, test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('fs');
|
||||
const path = require('path');
|
||||
|
||||
const REPO_ROOT = path.join(__dirname, '..');
|
||||
|
||||
// Shipped prose directories an agent can be instructed by. Tests/, CHANGELOG,
|
||||
// RELEASE-NOTES-LEGACY (historical record), and node_modules are deliberately
|
||||
// excluded — a test fixture may use the banned name as a negative input, and
|
||||
// history must not be rewritten.
|
||||
const SCAN_DIRS = [
|
||||
'agents',
|
||||
'gsd-core/references',
|
||||
'gsd-core/workflows',
|
||||
'commands/gsd',
|
||||
'skills',
|
||||
'docs',
|
||||
];
|
||||
|
||||
// Files that are historical record (must not be rewritten to satisfy this
|
||||
// guard) or generated changelog. RELEASE-NOTES-LEGACY.md carries the old #13898
|
||||
// attribution as shipped history; it is out of scope.
|
||||
const EXCLUDED_FILES = new Set([
|
||||
path.join(REPO_ROOT, 'docs', 'RELEASE-NOTES-LEGACY.md'),
|
||||
path.join(REPO_ROOT, 'CHANGELOG.md'),
|
||||
]);
|
||||
|
||||
const BANNED = 'mcp__context7__get-library-docs';
|
||||
|
||||
function listMarkdown(dir) {
|
||||
const abs = path.join(REPO_ROOT, dir);
|
||||
if (!fs.existsSync(abs)) return [];
|
||||
const out = [];
|
||||
for (const entry of fs.readdirSync(abs, { withFileTypes: true })) {
|
||||
const full = path.join(abs, entry.name);
|
||||
if (entry.isDirectory()) {
|
||||
out.push(...listMarkdown(path.join(dir, entry.name)));
|
||||
} else if (entry.isFile() && entry.name.endsWith('.md')) {
|
||||
out.push(full);
|
||||
}
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
describe('#2943 — no shipped artifact references the nonexistent get-library-docs tool', () => {
|
||||
const files = SCAN_DIRS.flatMap(listMarkdown)
|
||||
.filter((f) => !EXCLUDED_FILES.has(f));
|
||||
assert.ok(files.length > 50, `scan surface sanity check (found ${files.length} markdown files)`);
|
||||
|
||||
const offenders = [];
|
||||
for (const file of files) {
|
||||
const content = fs.readFileSync(file, 'utf8');
|
||||
if (content.includes(BANNED)) {
|
||||
// Report every offending line for actionable failures.
|
||||
const rel = path.relative(REPO_ROOT, file);
|
||||
for (const [i, line] of content.split(/\r?\n/).entries()) {
|
||||
if (line.includes(BANNED)) offenders.push(`${rel}:${i + 1}`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
test(`no shipped prose references ${BANNED}`, () => {
|
||||
assert.deepEqual(offenders, [],
|
||||
`Shipped artifacts must not instruct agents to call ${BANNED} — the ` +
|
||||
`context7 MCP server does not register it (only resolve-library-id and ` +
|
||||
`query-docs exist). Offending sites:\n${offenders.join('\n')}`);
|
||||
});
|
||||
|
||||
test('the canonical reference names the registered query-docs tool', () => {
|
||||
const ref = path.join(REPO_ROOT, 'gsd-core', 'references', 'research-documentation-lookup.md');
|
||||
const content = fs.readFileSync(ref, 'utf8');
|
||||
assert.ok(content.includes('mcp__context7__query-docs'),
|
||||
'research-documentation-lookup.md must name mcp__context7__query-docs');
|
||||
// The canonical param names (upstream query-docs: libraryId + query). The
|
||||
// alias context7CompatibleLibraryID is tolerated upstream but the canonical
|
||||
// spelling removes the drift source.
|
||||
assert.ok(/query-docs` with `libraryId`/.test(content) || /libraryId` and `query`/.test(content),
|
||||
'query-docs must be documented with the libraryId / query params');
|
||||
});
|
||||
|
||||
test('resolve-library-id tool name is preserved (still registered upstream)', () => {
|
||||
// Negative-space guard: this fix renames get-library-docs only.
|
||||
// resolve-library-id is a different, still-valid tool and must survive.
|
||||
let any = false;
|
||||
for (const file of files) {
|
||||
if (fs.readFileSync(file, 'utf8').includes('mcp__context7__resolve-library-id')) {
|
||||
any = true;
|
||||
break;
|
||||
}
|
||||
}
|
||||
assert.ok(any, 'mcp__context7__resolve-library-id must still appear in shipped artifacts');
|
||||
});
|
||||
});
|
||||
6
tests/emitted-drift-acks/2943-context7-tool-name.json
Normal file
6
tests/emitted-drift-acks/2943-context7-tool-name.json
Normal file
@@ -0,0 +1,6 @@
|
||||
{
|
||||
"version": 1,
|
||||
"paths": {
|
||||
"gsd-executor.md": "#2943: the context7 doc-lookup block's ctx7 CLI-fallback rationale was rewritten to describe the real mechanism (custom subagents cannot see project-scoped .mcp.json; they inherit only user-scoped ~/.claude/mcp.json), replacing the wrong anthropics/claude-code#13898 'tools: frontmatter restriction' attribution. The tool name was also corrected (get-library-docs -> query-docs) and the params renamed (context7CompatibleLibraryId/topic -> libraryId/query). The +95 bytes is the longer-but-accurate mechanism description; it is the literal fix for the mis-attribution, not incidental prose growth, and the rationale must be correct because agents read it to decide when to fall back to the CLI."
|
||||
}
|
||||
}
|
||||
@@ -82,7 +82,7 @@ const BARE_COMMAND_RE = new RegExp(
|
||||
// Each entry MUST carry a one-line reason; the test prints the allowlist on
|
||||
// failure so a reviewer can see exactly what is sanctioned.
|
||||
const PROSE_ALLOWLIST = [
|
||||
{ file: 'agents/gsd-executor.md', line: 791, reason: 'describes the SDK return envelope of `gsd-tools query commit`; not an instruction to run the bare word' },
|
||||
{ file: 'agents/gsd-executor.md', line: 793, reason: 'describes the SDK return envelope of `gsd-tools query commit`; not an instruction to run the bare word' },
|
||||
{ file: 'agents/gsd-phase-researcher.md', line: 33, reason: 'package-legitimacy provenance rule names the command as the source of an OK verdict; descriptive' },
|
||||
{ file: 'agents/gsd-roadmapper.md', line: 624, reason: 'parenthetical "e.g." naming SDK queries a user *could* run; not an agent instruction' },
|
||||
{ file: 'agents/gsd-intel-updater.md', line: 40, reason: 'cross-platform note names the `gsd-tools intel <subcommand>` CLI surface descriptively ("CLI invocations go through..."); not an agent instruction' },
|
||||
|
||||
Reference in New Issue
Block a user