summercms/.planning/phases/11.1-summercms-documentation-for-humans-and-ai-agents/11.1-VALIDATION.md

phase, slug, status, nyquist_compliant, wave_0_complete, created, validated

phase	slug	status	nyquist_compliant	wave_0_complete	created	validated
11.1	summercms-documentation-for-humans-and-ai-agents	validated	true	true	2026-09-28	2026-10-01

Phase 11.1 — Validation Strategy

Per-phase validation contract for feedback sampling during execution.

---

Test Infrastructure

Property	Value
Framework	go test (stdlib testing)
Config file	none. Tests locate docs/ and modules/ via relative paths from internal/docsite
Quick run command	go test -short ./internal/docsite/... ./cmd/summer/... ./docs/examples/...
Full suite command	go vet ./... && go test ./... (Docker required for walkthrough and lagoon DB tests)
Phase gate	scripts/check-phase11.1.sh --all
Estimated runtime	~60 seconds (quick), several minutes (full with Docker)

---

Sampling Rate

• After every task commit: Run go vet ./... && go test -short ./...
• After every plan wave: Run go test ./... plus scripts/check-phase11.1.sh --docs --forbidden
• Before /gsd-verify-work: scripts/check-phase11.1.sh --all must be green, then manual UAT of search and dark mode
• Max feedback latency: 60 seconds

---

Per-Task Verification Map

Filled in by the planner from plans 01 to 06, with plan 07 rows added for the gap-closure checks. Real-tree checks live in cmd/summer (package main owns toolCommands(), which the command checker needs); fixture and unit tests live in internal/docsite. Plan 11.1-06 Task 3 ran the earlier commands on 2026-10-01; plan 11.1-07 ran its five rows the same day and recorded the result. scripts/check-phase11.1.sh --named re-runs every named test by exact name and refuses a failure, a skip, a missing or renamed test and "no tests to run".

Task ID	Plan	Requirement	Behavior	Test Type	Automated Command	File Exists	Status
11.1-01-T1	01	DOCS-01, DOCS-02, DOCS-03	docs:build writes html, .md, llms.txt, llms-full.txt, search index; strict frontmatter; output guard	smoke (real tree)	go test ./internal/docsite ./cmd/summer -run '^(TestDocsTree|TestDocsBuildRealTree|TestToolCommandNames)$' -count=1 -v	✅	✅ green
11.1-01-T2	01	DOCS-01, DOCS-03	Every module is an API page in the sidebar; shared slug IDs; AI outputs match the page tree	unit + smoke	go test ./internal/docsite ./cmd/summer -run '^(TestSlugIDs|TestReadmeIngestion|TestEveryModuleInSidebar|TestDocsAIOutputsInSync|TestDocsTree)$' -count=1 -v	✅	✅ green
11.1-01-T3	01	DOCS-04	src= extraction, confinement, drift, docs:sync; ExampleCall runs	unit + toolchain	go vet ./... && go test ./internal/docsite ./cmd/summer ./modules/bonfire -count=1	✅	✅ green
11.1-02-T1	02	DOCS-05	Identifier checker over docs, module READMEs and root README; gate self-test	unit + gate	go test ./internal/docsite ./cmd/summer -run '^(TestIdentifierChecker|TestDocsTree)$' -count=1 -v && scripts/check-phase11.1.sh --self-test	✅	✅ green
11.1-02-T2	02	DOCS-05, DOCS-08	Links/anchors, command names (collected sets), forbidden names, go-fence policy; CLAUDE.md rule	unit + gate	go test ./internal/docsite ./cmd/summer -count=1 && scripts/check-phase11.1.sh --claude	✅	✅ green
11.1-02-T3	02	DOCS-02	UI-SPEC theme markers, chroma highlighting, search assets, docs:serve loopback and 404; go.mod adds only chroma/v2 and regexp2/v2	unit + gate	go test ./internal/docsite -run '^(TestBuildSiteMarkers|TestServeHandler|TestServeRefusesNonLoopback)$' -count=1 -v && scripts/check-phase11.1.sh --deps	✅	✅ green
11.1-03-T1	03	DOCS-06	Coming from WinterCMS page with checked identifiers and ExamplePlugin	toolchain + smoke	go test ./modules/party -run '^ExamplePlugin$' -count=1 -v && go test ./cmd/summer -run '^(TestDocsTree|TestDocsRequiredPages)$' -count=1	✅	✅ green
11.1-03-T2	03	DOCS-01, DOCS-04	Architecture and Plugins pages with running Examples	toolchain + smoke	go test ./modules/backpack ./modules/pact ./modules/festival ./modules/towel -run '^Example' -count=1 && go test ./cmd/summer -run '^(TestDocsTree|TestDocsRequiredPages)$' -count=1	✅	✅ green
11.1-03-T3	03	DOCS-01, DOCS-04	Setup and Console pages; every command name verified	smoke + gate	go test ./cmd/summer -run '^(TestDocsTree|TestDocsRequiredPages)$' -count=1 && scripts/check-phase11.1.sh --docs	✅	✅ green
11.1-04-T1	04	DOCS-04	Jobs page with a runnable Example and a database-backed region run by TestDocsDispatch	integration (Docker)	go test ./modules/conga -run '^(Example.*|TestDocsDispatch)$' -count=1 -v	✅	✅ green
11.1-04-T2	04	DOCS-01, DOCS-04	Database and core Services pages with running examples	toolchain + integration	go test ./modules/lagoon/... ./modules/compass ./modules/surf ./modules/wire ./modules/bouncer ./modules/wristband ./modules/postcard ./modules/phrasebook -run '^(Example|TestDocs)' -count=1	✅	✅ green
11.1-04-T3	04	DOCS-01, DOCS-06	Backend and remaining Services pages, Frontend and AJAX page, section order, concept-map links	smoke + gate	go test ./cmd/summer -run '^(TestDocsTree|TestDocsRequiredPages)$' -count=1 && scripts/check-phase11.1.sh --forbidden	✅	✅ green
11.1-05-T1	05	DOCS-07	Blog plugin activates, route registered; walkthrough page from src=	unit + smoke	go test -short ./docs/examples/... -count=1 -v	✅	✅ green
11.1-05-T2	05	DOCS-07	Admin controller, command, second migration proven against Postgres	integration (Docker)	go test ./docs/examples/... -count=1 -v	✅	✅ green
11.1-05-T3	05	DOCS-07	Walkthrough file set equals scaffolder output	unit	go test -short ./docs/examples/blog -run '^TestScaffoldLayout$' -count=1 -v	✅	✅ green
11.1-06-T1	06	DOCS-05	Every rule has a planted fixture that fails for its own rule (63 committed cases plus 5 built at run time), and the clean fixture passes	unit + gate	go test ./internal/docsite -run '^(TestPlantedViolations|TestCleanFixture|TestBuildOutputGuard)$' -count=1 -v && scripts/check-phase11.1.sh --self-test	✅	✅ green
11.1-06-T2	06	DOCS-01..05	internal/docsite statement coverage at least 85% (94.8% measured)	unit	go test ./internal/docsite -count=1 -coverprofile="${TMPDIR:-/tmp}/docsite.cover" && go tool cover -func="${TMPDIR:-/tmp}/docsite.cover"	✅	✅ green
11.1-06-T3	06	DOCS-01..08	SC1..SC5 acceptance subtests; final gate with --named	acceptance + gate	go test ./cmd/summer -run '^TestPhase11_1Acceptance$' -count=1 -v && scripts/check-phase11.1.sh --all	✅	✅ green
11.1-07-T1	07	DOCS-04	AST fence discovery refuses a nested src= fence; real tree stays clean	unit + smoke	go vet ./... && go test ./internal/docsite ./cmd/summer -count=1 && d=$(mktemp -d) && cp -r internal/docsite/testdata/clean/. "$d" && go run ./cmd/summer docs:build --root "$d" --check && printf '\n> [!TIP]\n> ```go src=modules/demo/example_test.go#ExampleHello\n> BOGUS\n> ```\n' >> "$d/docs/extras/faq.md" && ! go run ./cmd/summer docs:build --root "$d" --check > "$d/check.log" 2>&1 && grep -F 'modules/demo/example_test.go#ExampleHello: src= code block must be a top-level block' "$d/check.log" && go run ./cmd/summer docs:build --check	✅	✅ green
11.1-07-T2	07	DOCS-04	A golang fence without src= and a module README src= fence are refused	unit + smoke	go vet ./... && go test ./internal/docsite ./cmd/summer -count=1 && d=$(mktemp -d) && cp -r internal/docsite/testdata/clean/. "$d" && printf '\n```golang\nx := 1\n```\n' >> "$d/docs/extras/faq.md" && ! go run ./cmd/summer docs:build --root "$d" --check > "$d/check.log" 2>&1 && grep -F 'snippet: golang code block has no src= reference' "$d/check.log" && d=$(mktemp -d) && cp -r internal/docsite/testdata/clean/. "$d" && printf '\n```go src=modules/demo/example_test.go#ExampleHello\nfmt.Println(demo.Hello("blog"))\n// Output: Hello, blog\n```\n' >> "$d/modules/demo/README.md" && ! go run ./cmd/summer docs:build --root "$d" --check > "$d/check.log" 2>&1 && grep -F 'module README code blocks are rendered as written' "$d/check.log" && go run ./cmd/summer docs:build --check	✅	✅ green
11.1-07-T3	07	DOCS-04, DOCS-05	Build-ignored Go, wrong-case go doc -c, and an env-prefixed command are refused	unit + smoke	go vet ./... && go test ./internal/docsite ./cmd/summer -count=1 && d=$(mktemp -d) && cp -r internal/docsite/testdata/clean/. "$d" && printf '\n```sh\nFOO=1 summer no:such\n```\n' >> "$d/docs/extras/faq.md" && ! go run ./cmd/summer docs:build --root "$d" --check > "$d/check.log" 2>&1 && grep -F '"no:such" is not a summer or application command' "$d/check.log" && d=$(mktemp -d) && cp -r internal/docsite/testdata/clean/. "$d" && printf 'package demo\n\n// OpenFromApp is a fixture.\nfunc OpenFromApp() {}\n' > "$d/modules/demo/open.go" && printf '\nCall demo.Openfromapp.\n' >> "$d/docs/extras/faq.md" && ! go run ./cmd/summer docs:build --root "$d" --check > "$d/check.log" 2>&1 && grep -F 'identifier: demo.Openfromapp does not exist in modules/demo' "$d/check.log" && d=$(mktemp -d) && cp -r internal/docsite/testdata/clean/. "$d" && printf '//go:build ignore\n\npackage demo\n\n// Broken is never built.\nfunc Broken() { undefinedCall() }\n' > "$d/modules/demo/broken.go" && printf '\n```go src=modules/demo/broken.go#Broken\n// Broken is never built.\nfunc Broken() { undefinedCall() }\n```\n' >> "$d/docs/extras/faq.md" && ! go run ./cmd/summer docs:build --root "$d" --check > "$d/check.log" 2>&1 && grep -F 'file is excluded from the default build' "$d/check.log" && go run ./cmd/summer docs:build --check	✅	✅ green
11.1-07-T4	07	DOCS-04, DOCS-05	Eighteen planted fixtures and the fence, caption, root, go doc and commandWord tests; coverage at least 85% (94.1% measured)	unit	go vet ./... && go test ./internal/docsite -run '^(TestPlantedViolations|TestCleanFixture|TestCollectFences|TestFenceCaptionsOnlyVerified|TestGoLang|TestNestedFenceChecks|TestIdentifierGoDocCaseSensitive|TestCommandWord|TestSnippetRootsAndBuild|TestSyncParsedFences)$' -count=1 -v && go test ./internal/docsite -count=1 -coverprofile="${TMPDIR:-/tmp}/docsite.cover" && go tool cover -func="${TMPDIR:-/tmp}/docsite.cover" | awk '/^total:/ { sub("%", "", $3); if ($3 + 0 < 85.0) { print "coverage " $3 "% below 85%"; exit 1 } }'	✅	✅ green
11.1-07-T5	07	DOCS-04, DOCS-05	Acceptance scanner sees blockquoted and aliased fences; the gate plants the holes and --all passes	acceptance + gate	go vet ./... && go test ./cmd/summer -run '^(TestPhase11_1Acceptance|TestAcceptanceFenceScanner)$' -count=1 -v && bash -n scripts/check-phase11.1.sh && scripts/check-phase11.1.sh --self-test && scripts/check-phase11.1.sh --all	✅	✅ green

Status: ✅ green · ❌ red · ⚠️ flaky. Every automated row is green; only the two manual rows below wait for /gsd-verify-work.

---

Wave 0 Requirements

• internal/docsite/ package with test scaffolding and a testdata/ planted-violation corpus (11.1-01, corpus in 11.1-06)
• modules/bonfire/example_test.go — first verified snippet (tracer, 11.1-01)
• docs/site.yaml, docs/index.md, docs/setup/installation.md (11.1-01)
• cmd/summer/docs.go + TestToolCommandNames extension (11.1-01)
• scripts/check-phase11.1.sh with --self-test (11.1-02, corpus and --named in 11.1-06)
• /site/ in .gitignore (11.1-01)

---

Manual-Only Verifications

Behavior	Requirement	Why Manual	Test Instructions
Client-side search returns relevant pages	DOCS-02	No JS test runner without a Node toolchain	summer docs:serve, search for a module name and a CLI command, confirm results link correctly. Status: pending (manual UAT in /gsd-verify-work)
Dark mode toggles and persists	DOCS-02	Visual	summer docs:serve, toggle theme, reload, confirm persistence and contrast per UI-SPEC at 1280px, 1024px and 375px, and the no-JS layout. Status: pending (manual UAT in /gsd-verify-work)

---

Validation Sign-Off

• All tasks have <automated> verify or Wave 0 dependencies
• Sampling continuity: no 3 consecutive tasks without automated verify
• Wave 0 covers all MISSING references
• No watch-mode flags
• Feedback latency < 60s for the quick run; the full gate (--all, Docker) takes several minutes
• nyquist_compliant: true set in frontmatter

Approval: approved 2026-10-01 (plan 11.1-06). Manual UAT of search and dark mode remains for /gsd-verify-work.