Six plans (tracer generator, site UX and checkers, content A, content B,
acme/blog walkthrough, unit tests). SC4/DOCS-04 narrowed to docs/ pages per
D-18; README Go fence conversion logged as a todo.
Per D-14 and the CLAUDE.md lean rule, this sixth and last plan brings full unit test coverage for the phase's code: `go test -cover ./internal/docsite` reports at least 85.0% of statements.
Every problem rule the generator emits (frontmatter variants, section, readme, snippet variants including confinement, identifier, link, anchor, command, forbidden, go fence without src=, callout, heading, output guard) has a planted fixture that produces exactly that problem, and the clean fixture produces none (TestPlantedViolations).
The forbidden-name fixture is generated at test time from split strings, so no committed test file or fixture contains a consuming-application name (D-11).
TestPhase11_1Acceptance in cmd/summer has one subtest per ROADMAP success criterion (SC1 to SC5) that asserts it on the real tree, and all five pass.
scripts/check-phase11.1.sh --all runs --preconditions, --deps, --self-test (including TestPlantedViolations), --docs, --forbidden, --claude, --named and --go (full `go test ./...`, not -short) and prints `phase11.1 all passed`; --named fails when a named test is missing, skipped or matches zero tests.
11.1-VALIDATION.md has every per-task row filled with its command and a green status, `status: validated`, `nyquist_compliant: true` and `wave_0_complete: true`, and keeps the manual-only rows (search, dark mode) for /gsd-verify-work.
No production code changes in this plan except fixes for defects the new tests expose; each such fix lands with the failing-then-passing test in the same commit and is listed in the SUMMARY.
statement
verification
Manual UAT of search and dark mode in `summer docs:serve` is run in /gsd-verify-work (VALIDATION manual-only rows).
backstop
requirement_id
category
status
verification
resolution
reason
statement
DOCS-05
transparency
resolved
judgment
The gate detector requires named tests to PASS; skips, zero matches and 'no tests to run' fail the stage.
A docs gate that goes green because a checker test was skipped or filtered away certifies drift it never measured.
The phase gate must not count a skipped, filtered-out or zero-match test run as a pass.
--self-test and --named run TestPlantedViolations and require PASS
TestPlantedViolations
from
to
via
pattern
cmd/summer/phase11_1_acceptance_test.go
internal/docsite
real-tree Check, Build and page tree assertions per success criterion
docsite.(Check|Build)
Unit tests last (D-14, CLAUDE.md lean rule 3): full coverage of `internal/docsite`, a planted-violation fixture for every checker rule, the final phase gate with named-test detection, one acceptance test per success criterion, and a validated 11.1-VALIDATION.md.
Purpose: make the docs gates fail closed. A checker regression that stops reporting a problem must turn a test red, and the gate must refuse a skipped or empty run.
Output: test files and fixtures under internal/docsite, cmd/summer/phase11_1_acceptance_test.go, the finished scripts/check-phase11.1.sh, and 11.1-VALIDATION.md.
Test conventions (repository): stdlib testing only, no testify in these packages; t.TempDir() fixtures; t.Fatalf("x = %v, want %v"). Stage only each task's files.
Task 1: Tracer: every checker rule has a planted fixture that must produce exactly its problem, and the gate runs them
internal/docsite/violations_test.go, internal/docsite/testdata/clean/, internal/docsite/testdata/violations/, scripts/check-phase11.1.sh
- internal/docsite/*.go (every Problem rule and message format actually emitted; grep for `Rule:`)
- .planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-UI-SPEC.md CLI output table (problem wording)
- scripts/check-phase11.1.sh (current modes and self-test)
- scripts/check-phase11.sh phase11_detect (go test -json detector shape)
1. `internal/docsite/testdata/clean/`: a small root that passes every check: `go.mod` (a throwaway module path), `docs/site.yaml` with two sections, `docs/index.md`, two pages per section with valid frontmatter, one Go fence `src=` into a fixture `modules/demo/example_test.go#ExampleHello` with `// Output:`, a region fence, a link with an anchor, a `summer docs:build` token, a callout, and `modules/demo/` with `demo.go` and a README using `demo.Hello`. Keep the fixture's Go files out of the root module build (put them under `testdata`, which the go tool ignores).
2. `internal/docsite/testdata/violations//`: one directory per case, each a copy of the clean fixture with one planted fault plus a `want.txt` holding the expected rule and message substring. Cases: frontmatter-unknown-field, frontmatter-missing-field, frontmatter-section-mismatch, frontmatter-duplicate-order, frontmatter-first-line, frontmatter-long-description, section-no-pages, readme-missing, snippet-drift, snippet-missing-file, snippet-missing-ident, snippet-missing-region, snippet-absolute-path, snippet-dotdot, snippet-dotfile, snippet-nested-module, snippet-example-no-output, snippet-unrun-region, go-fence-no-src, identifier-unknown, identifier-unknown-member, link-broken, anchor-missing, anchor-cross-page, command-unknown-summer, command-unknown-app, callout-unknown, heading-code, heading-link, heading-non-ascii. Symlink escape and the forbidden name are built at test time (a symlink in `t.TempDir()`, and the forbidden word assembled from split string literals so no committed file contains it).
3. `violations_test.go`: `TestPlantedViolations` copies each case into `t.TempDir()`, runs `Check` with a fixture `Commands` set, and asserts exactly one problem whose rule and message match `want.txt` (the rule proves it failed for its own reason, not another); `TestCleanFixture` asserts zero problems. Add output-guard cases (out equals root, out inside src, out containing src, unmarked non-empty out) asserting `Build` refuses and leaves a sentinel file untouched.
4. Gate: `--self-test` also runs `go test ./internal/docsite -run '^(TestPlantedViolations|TestCleanFixture)$' -count=1 -json` through a detector (Python, like `phase11_detect`) that fails on any FAIL, any SKIP, zero tests, or "no tests to run".
go vet ./... && go test ./internal/docsite -run '^(TestPlantedViolations|TestCleanFixture)$' -count=1 -v
non-zero exit, a "--- FAIL" line, or "no tests to run"
scripts/check-phase11.1.sh --self-test
non-zero exit, a line starting "refuse:", or no "phase11.1 self-test passed" line
- `ls -d internal/docsite/testdata/violations/*/ | wc -l` prints at least 30.
- `go test ./internal/docsite -run '^TestPlantedViolations$' -count=1 -v | grep -c -- '--- PASS: TestPlantedViolations/'` prints at least 30.
- `! grep -rniE 'fonoteka|p(l|ł)ytarium' internal/docsite/testdata internal/docsite/violations_test.go` (no match: the forbidden word is assembled at run time; the checker's own pattern in check_forbidden.go is the only source that names it).
- Deleting the body of the identifier checker's miss branch in a scratch copy makes `TestPlantedViolations/identifier-unknown` fail (checked once by hand during execution and recorded in the SUMMARY).
- `scripts/check-phase11.1.sh --self-test` prints `phase11.1 self-test passed`.
Each rule's failure path is pinned by a fixture that must fail for its own reason, the clean fixture passes, and the gate's self-test runs them fail-closed.
Task 2: internal/docsite reaches at least 85% statement coverage across load, render, emit, snippets, highlighting, checkers and serve
internal/docsite/load_test.go, internal/docsite/render_test.go, internal/docsite/emit_test.go, internal/docsite/snippet_test.go, internal/docsite/highlight_test.go, internal/docsite/serve_test.go, internal/docsite/checks_test.go, internal/docsite/theme_test.go, internal/docsite/docsite_test.go, cmd/summer/docs_test.go
- internal/docsite/*.go and the existing *_test.go files (smoke tests from plans 11.1-01 and 11.1-02)
- `go test -coverprofile` output for internal/docsite (to target uncovered blocks)
- cmd/summer/docs.go (flag handling, problem printing, exit error)
Add branch-level tests (table-driven where natural):
- `load_test.go`: strict site.yaml (unknown key, missing sections, duplicate section names), frontmatter split edge cases (no closing delimiter, CRLF line endings, BOM), index page rules, `_`-prefixed and `examples/` exclusion, README ingestion (summary line after blank lines, missing H1).
- `render_test.go`: slug IDs (Unicode letters kept, emoji and punctuation dropped, `_` kept, duplicates `-1`/`-2`, empty heading), H1 strip, link rewriting for every link class (page .md with and without fragment, module README from guides and from READMEs, external, mailto), callout rendering for the three types, heading anchor markup, TOC threshold (0, 1, 2 headings).
- `emit_test.go`: llms.txt exact shape on a fixture (H1, blockquote, notes, Overview, one H2 per section, API reference), llms-full.txt block format and order, `.md` transform (frontmatter dropped, `# Title` and `> description`, fence info reduced to the language word, links to .md), search-index.json schema and the 300-character excerpt cap, base_url prefixing (empty, path prefix, absolute URL).
- `snippet_test.go`: whole-file, `#Ident` with doc comment, generic declarations, Example body dedent with Output, Go and YAML regions, unterminated and duplicate regions, trailing-newline normalisation, `Sync` idempotence and byte-for-byte preservation of surrounding text.
- `highlight_test.go`: each token class (`tok-kw`, `tok-key`, `tok-str`, `tok-com`, `tok-num`, `tok-prompt`) for go, yaml and sh samples; HTML escaping of `<`, `&` and quotes inside tokens; unknown language falls back to escaped plain text; no inline style attribute in output.
- `serve_test.go`: loopback acceptance for `127.0.0.1`, `::1`, `localhost`; refusal for `0.0.0.0`, `::`, a LAN IP and a non-local hostname; `--allow-remote` override; `Handler` 200, 404 with the 404 body, dot-file refused; rebuild keeps the last good directory when a build returns problems (inject the build function or use a fixture tree that you break between builds).
- `checks_test.go`: identifier index for generic methods, embedded fields, interface methods, sub-package keys and the ambiguous-key problem; the `go doc` fallback accepting a promoted member; command token forms (`$ summer x`, `summer x --flag`, `./bin/app x`, code spans); link checker on ingested READMEs.
- `cmd/summer/docs_test.go`: `docs:build --check` prints problems and `docs:build: N problems, nothing written` and returns an error for a scratch tree with a planted fault; `docs:sync` output lines; `docs:serve` refusal output.
Fix any defect these tests expose in the same commit as the test, and list each fix in the SUMMARY.
go vet ./... && go test ./internal/docsite ./cmd/summer -count=1 && go test ./internal/docsite -count=1 -coverprofile="${TMPDIR:-/tmp}/docsite.cover" && go tool cover -func="${TMPDIR:-/tmp}/docsite.cover" | awk '/^total:/ { sub("%", "", $3); if ($3 + 0 < 85.0) { print "coverage " $3 "% below 85%"; exit 1 } }'
non-zero exit, a "FAIL" line, or a "coverage ... below 85%" line
- `go test -cover ./internal/docsite -count=1` prints `coverage:` with a value of at least 85.0%.
- `go test ./internal/docsite -run '^(TestSlugIDs|TestHighlight.*|TestServe.*|TestLLMS.*|TestSnippet.*)' -count=1 -v | grep -c -- '--- PASS'` prints at least 10.
- `go test ./cmd/summer -run '^TestDocs' -count=1` passes.
internal/docsite has branch-level tests for every stage with at least 85% statement coverage, and any defect found is fixed with its test.
Task 3: One acceptance test per success criterion, the final gate, and a validated VALIDATION.md
cmd/summer/phase11_1_acceptance_test.go, scripts/check-phase11.1.sh, .planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VALIDATION.md
- .planning/ROADMAP.md Phase 11.1 success criteria 1-5
- .planning/REQUIREMENTS.md DOCS-01 to DOCS-08
- cmd/summer/docs_test.go (real-tree helpers to reuse)
- scripts/check-phase11.1.sh and scripts/check-phase11.sh (phase11_detect, run_named)
- .planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VALIDATION.md
- the six 11.1-0N-SUMMARY.md files (test names actually created)
1. `cmd/summer/phase11_1_acceptance_test.go`: `TestPhase11_1Acceptance` with subtests:
- `SC1`: site.yaml sections are exactly setup, architecture, plugins, backend, database, services, console, api; every page's frontmatter is valid (Check has no frontmatter or section problem); every `modules/` with non-test Go files has an `api/` page in the sidebar.
- `SC2`: a real-tree build into `t.TempDir()` contains every UI-SPEC Theme Parts marker on a guide page, `404.html`, the four assets and eight fonts; `internal/docsite` and `cmd/summer/docs.go` contain no `os/exec` call to `node` or `npm` (parse with go/ast and inspect exec.Command first arguments); `docs:serve` is registered.
- `SC3`: pages, `.html`, `.md`, llms.txt links and llms-full.txt Source lines are equal sets in reading order.
- `SC4`: every go fence under docs/ has src=, `Check` reports zero problems (snippets, identifiers, links, anchors, commands, forbidden), and every referenced `Example*` exists with an `// Output:` comment.
- `SC5`: `setup/coming-from-wintercms` and `setup/porting-a-plugin` exist, the walkthrough page references `docs/examples/blog` sources only through src=, and `go list ./docs/examples/...` includes the blog packages.
2. Gate: add `--named` (runs `go test -json` for the named tests of the phase across `./internal/docsite`, `./cmd/summer`, `./docs/examples/...` and the modules' `TestDocs*` and `Example*` tests, and requires each to PASS; FAIL, SKIP, zero tests and "no tests to run" refuse) and make `--all` run `--preconditions --deps --self-test --docs --forbidden --claude --named --go` and print `phase11.1 all passed`. Keep `--go` as full `go vet ./...` and `go test ./...` without `-short`.
3. `11.1-VALIDATION.md` (planning commit, separate from code): fill the per-task verification map with every task of plans 01 to 06 (task id, requirement, test type, the exact automated command, file exists, green status), set `status: validated`, `nyquist_compliant: true`, `wave_0_complete: true`, tick the sign-off list, and keep the two manual-only rows (search, dark mode) for /gsd-verify-work.
4. Run `scripts/check-phase11.1.sh --all` and fix any failure at its source.
go vet ./... && go test ./cmd/summer -run '^TestPhase11_1Acceptance$' -count=1 -v
non-zero exit, a "--- FAIL" line, fewer than five "--- PASS: TestPhase11_1Acceptance/SC" lines, or "no tests to run"
bash -n scripts/check-phase11.1.sh && scripts/check-phase11.1.sh --self-test && scripts/check-phase11.1.sh --all
non-zero exit, a line starting "refuse:", or no "phase11.1 all passed" line
- `go test ./cmd/summer -run '^TestPhase11_1Acceptance$' -count=1 -v | grep -c -- '--- PASS: TestPhase11_1Acceptance/SC'` prints 5.
- `scripts/check-phase11.1.sh --all` prints `phase11.1 all passed`.
- `grep -n '^status: validated$' .planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VALIDATION.md` and `grep -n '^nyquist_compliant: true$' .planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VALIDATION.md` each find a match.
- `grep -c 'pending' .planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VALIDATION.md` counts only the manual-only rows (at most 2).
- Temporarily renaming `TestPlantedViolations` in a scratch copy makes `--named` refuse with a zero-tests or missing-test message (checked once during execution and recorded in the SUMMARY).
Each success criterion has a passing acceptance subtest, the final gate runs every stage fail-closed and passes, and VALIDATION.md is validated with only the manual UAT rows left for verify-work.
<threat_model>
Trust Boundaries
Boundary
Description
gate and tests → phase sign-off
A green gate is the evidence that the docs match the code
test fixtures → published docs
Fixture content must never reach docs/ or the built site
One planted fixture per rule that must fail for its own rule (TestPlantedViolations); go test -json detector refuses FAIL, SKIP, zero tests and "no tests to run"; --go runs without -short
T-11.1-19
Information disclosure (policy)
internal/docsite/testdata
low
mitigate
Fixtures live under internal/docsite/testdata, outside docs/ and the build walker; the forbidden word is assembled at run time and never committed
T-11.1-SC
Tampering
npm/pip/cargo/go installs
high
accept
This plan adds no module or package
</threat_model>
- `scripts/check-phase11.1.sh --all` prints `phase11.1 all passed` (Docker available).
- `go test -cover ./internal/docsite` at least 85%.
- Manual UAT in /gsd-verify-work: `summer docs:serve`, search for a module and a command, toggle theme in all three modes and reload, check a guide and an API page at 1280px, 1024px and 375px, and with JS disabled.
<success_criteria>
All five ROADMAP success criteria asserted by TestPhase11_1Acceptance subtests SC1 to SC5.
DOCS-01 to DOCS-08 each mapped to a green automated command in 11.1-VALIDATION.md.
The phase gate is fail-closed and green.
</success_criteria>
Artifacts this phase produces
Tests: TestPlantedViolations, TestCleanFixture (internal/docsite/violations_test.go); branch tests in load_test.go, render_test.go, emit_test.go, snippet_test.go, highlight_test.go, serve_test.go and extended checks_test.go, theme_test.go, docsite_test.go; TestPhase11_1Acceptance with subtests SC1 to SC5 (cmd/summer/phase11_1_acceptance_test.go).
Fixtures: internal/docsite/testdata/clean/, internal/docsite/testdata/violations/<case>/ with want.txt.
Gate modes: --named added; --all final; --self-test runs the fixture tests through the JSON detector.
11.1-VALIDATION.md validated.
Create `.planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-06-SUMMARY.md` when done
6f57604028