docs(11.1-01): complete docs generator tracer plan summary
This commit is contained in:
Jakub Zych
@@ -0,0 +1,243 @@
|
||||
---
|
||||
phase: 11.1-summercms-documentation-for-humans-and-ai-agents
|
||||
plan: 01
|
||||
subsystem: docs
|
||||
tags: [docs, goldmark, html-template, llms-txt, cli, bonfire]
|
||||
|
||||
requires:
|
||||
- phase: 11-core-framework
|
||||
provides: "the 22 framework modules with READMEs that the API reference ingests"
|
||||
provides:
|
||||
- "internal/docsite: Options, Problem, Result, SyncResult, Check, Build, Sync, Pages, Site, Section, Frontmatter, Page, Ref, ParseSrc, Extract, MarkerFile"
|
||||
- "summer docs:build (--root --src --out --base-url --check) and summer docs:sync (--root --src)"
|
||||
- "docs/site.yaml, docs/index.md, docs/setup/installation.md"
|
||||
- "modules/bonfire/example_test.go ExampleCall, the first src= verified example"
|
||||
- "one slug parser.IDs (slugIDs) and a headingIDs helper for the plan 11.1-02 link checker"
|
||||
affects: [11.1-02, 11.1-03, 11.1-04, 11.1-05, 11.1-06]
|
||||
|
||||
actuals:
|
||||
tokens: 24800
|
||||
tasks: 3
|
||||
commits: 3
|
||||
plan_head_before: 63bcbc31b0dabc5fc80b50cdbb2213e440798596
|
||||
plan_head_after: dc6a03c7142c3e075a16e0d15518d2c5bf81c267
|
||||
|
||||
tech-stack:
|
||||
added: []
|
||||
patterns:
|
||||
- "One goldmark pipeline (GFM, safe mode) with AST transformers fed per-page state through parser.Context"
|
||||
- "Render everything into an in-memory map, then guard and write the output directory"
|
||||
- "embedmd-style src= fences: the fence body is a byte copy of real, tested source; docs:sync refreshes it"
|
||||
- "Problems print as file:line: rule: message and block the build"
|
||||
|
||||
key-files:
|
||||
created:
|
||||
- internal/docsite/docsite.go
|
||||
- internal/docsite/load.go
|
||||
- internal/docsite/render.go
|
||||
- internal/docsite/emit.go
|
||||
- internal/docsite/snippet.go
|
||||
- internal/docsite/docsite_test.go
|
||||
- internal/docsite/theme/templates/page.html
|
||||
- internal/docsite/theme/assets/site.css
|
||||
- cmd/summer/docs.go
|
||||
- cmd/summer/docs_test.go
|
||||
- modules/bonfire/example_test.go
|
||||
- docs/site.yaml
|
||||
- docs/index.md
|
||||
- docs/setup/installation.md
|
||||
modified:
|
||||
- cmd/summer/main.go
|
||||
- cmd/summer/main_test.go
|
||||
- .gitignore
|
||||
- README.md
|
||||
|
||||
key-decisions:
|
||||
- "Added exported docsite.Pages(opts) (reading-order pages plus load problems, no rendering) so TestDocsAIOutputsInSync and later checkers read the page tree without re-implementing ordering"
|
||||
- "Module README pages blank the summary line in the body (line count kept) so the lead is not repeated and README problems keep their own line numbers"
|
||||
- "A type, var or const named by #Ident in a _test.go file counts as run when a reachable Test/Example function references it; functions and regions must be reachable from a Test or Example"
|
||||
- "Fence rendering uses a goldmark NodeRenderer at priority 100: goldmark registers lower priority values last, so it overrides the default html renderer (1000)"
|
||||
- "Development commits land on master: the project config uses branching_strategy none and every earlier phase commit is on master"
|
||||
|
||||
patterns-established:
|
||||
- "Content rule: a docs page file starts with frontmatter and its first body line is exactly '# <title>'"
|
||||
- "Example convention: modules/<m>/example_test.go, package <m>_test, Example<Ident> ending in // Output:"
|
||||
|
||||
requirements-completed: [DOCS-01, DOCS-02, DOCS-03, DOCS-04]
|
||||
|
||||
coverage:
|
||||
- id: D1
|
||||
description: "summer docs:build renders docs/ into a static site with .html pages, .md siblings, llms.txt, llms-full.txt, search-index.json, assets and the .summer-docs marker"
|
||||
requirement: DOCS-02
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "cmd/summer/docs_test.go#TestDocsBuildRealTree"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "cmd/summer/docs_test.go#TestDocsTree"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D2
|
||||
description: "Output guard refuses a non-empty --out without the marker and an --out equal to the root, inside --src or containing it"
|
||||
requirement: DOCS-02
|
||||
verification:
|
||||
- kind: other
|
||||
ref: "go run ./cmd/summer docs:build --out <dir with an unrelated file> (exit 1, file kept); --out docs/x, --out ., --out .. (exit 1)"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "internal/docsite/docsite_test.go#TestSyncRewritesDrift (Build writes nothing while a snippet drifts)"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D3
|
||||
description: "Every framework module is an API reference page in the sidebar, llms.txt and search; links to READMEs and pages are rewritten"
|
||||
requirement: DOCS-01
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "cmd/summer/docs_test.go#TestEveryModuleInSidebar"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "internal/docsite/docsite_test.go#TestReadmeIngestion"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "internal/docsite/docsite_test.go#TestSlugIDs"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D4
|
||||
description: "llms.txt, llms-full.txt and the per-page .md files list the same pages in reading order and follow the llms.txt shape"
|
||||
requirement: DOCS-03
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "cmd/summer/docs_test.go#TestDocsAIOutputsInSync"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D5
|
||||
description: "src= fences are verified copies of tested source; drift or a missing source fails Check and Build; docs:sync rewrites drifted copies; bonfire ExampleCall runs under go test"
|
||||
requirement: DOCS-04
|
||||
verification:
|
||||
- kind: unit
|
||||
ref: "internal/docsite/docsite_test.go#TestSnippetForms"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "internal/docsite/docsite_test.go#TestSnippetConfinement"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "internal/docsite/docsite_test.go#TestSyncRewritesDrift"
|
||||
status: pass
|
||||
- kind: unit
|
||||
ref: "modules/bonfire/example_test.go#ExampleCall"
|
||||
status: pass
|
||||
human_judgment: false
|
||||
- id: D6
|
||||
description: "llms.txt and the per-page .md files are useful to an AI agent reading raw files"
|
||||
verification: []
|
||||
human_judgment: true
|
||||
rationale: "Plan marks this truth as a backstop; usefulness to an agent is a judgment call beyond the shape assertions in TestDocsAIOutputsInSync"
|
||||
|
||||
duration: 15min
|
||||
completed: 2026-09-30
|
||||
status: complete
|
||||
---
|
||||
|
||||
# Phase 11.1 Plan 01: Docs generator tracer Summary
|
||||
|
||||
**`summer docs:build` renders docs/ and all 22 module READMEs through goldmark and html/template into HTML, .md siblings, llms.txt, llms-full.txt and a search index. Go fences with `src=` are byte-checked copies of tested source, and `summer docs:sync` refreshes them.**
|
||||
|
||||
## Performance
|
||||
|
||||
- **Duration:** 15 min
|
||||
- **Started:** 2026-09-30T19:12:22Z
|
||||
- **Completed:** 2026-09-30T19:27:06Z
|
||||
- **Tasks:** 3
|
||||
- **Files modified:** 18
|
||||
|
||||
## Accomplishments
|
||||
|
||||
- `internal/docsite` is a stdlib and goldmark generator. It decodes `site.yaml` and frontmatter strictly with goccy/go-yaml `DisallowUnknownField`, reports problems in the UI-SPEC `file:line: rule: message` wording, renders every page in memory and writes nothing when a problem exists.
|
||||
- Module discovery reads `modules/` at run time. Each directory with non-test Go files becomes `api/<name>` from its README, and a directory with no README is a `readme:` problem. The real tree builds 24 pages: the index, installation and 22 API pages.
|
||||
- Heading anchors come from a single GitHub-compatible `parser.IDs` implementation (`slugIDs`). Links to `.md` pages and to `modules/<m>/README.md` are rewritten to `.html` URLs in the HTML output and to `.md` URLs in the raw output. `search-index.json` has one entry per H2.
|
||||
- A `src=` reference can name a whole file, a Go declaration, an Example body with its `// Output:` line, or a `docs:start`/`docs:end` region. The path must stay inside the root: it must be relative and clean, must not follow a symlink out of the root, and must not point at a dotfile, a `.env` file or a nested go.mod module. The referenced code must also run under `go test`: Examples need `// Output:`, `_test.go` fragments must be reachable from a Test or Example, and non-test sources need a package with tests.
|
||||
- `bonfire.ExampleCall` is shown in `docs/setup/installation.md`. Its fence body was written by `docs:sync`.
|
||||
|
||||
## Task Commits
|
||||
|
||||
1. **Task 1: Tracer: `summer docs:build` with every output** - `6dacddc` (feat)
|
||||
2. **Task 2: Every module as an API reference page, shared heading IDs, rewritten links, search entries** - `e433dcf` (feat)
|
||||
3. **Task 3: Verified snippets, `summer docs:sync`, first Example** - `dc6a03c` (feat)
|
||||
|
||||
**Plan metadata:** see the docs(11.1-01) commit that adds this file
|
||||
|
||||
## Files Created/Modified
|
||||
|
||||
- `internal/docsite/docsite.go`: Options, Problem, Result, Check, Build, the output guard and the file writer
|
||||
- `internal/docsite/load.go`: site.yaml and frontmatter decoding, the page walk, README ingestion, reading order and Pages
|
||||
- `internal/docsite/render.go`: the goldmark pipeline, slugIDs, link rewriting, fence annotation and rendering, plain-text extraction and the fence scanner
|
||||
- `internal/docsite/emit.go`: the page template, .md siblings, llms.txt, llms-full.txt, search-index.json and assets
|
||||
- `internal/docsite/snippet.go`: Ref, ParseSrc, Extract, the confinement and run rules, the drift check and Sync
|
||||
- `internal/docsite/theme/templates/page.html`, `theme/assets/site.css`: the tracer shell (sidebar and content) and the light tokens
|
||||
- `cmd/summer/docs.go`: the `docs:build` and `docs:sync` commands
|
||||
- `cmd/summer/docs_test.go`: TestDocsTree, TestDocsBuildRealTree, TestEveryModuleInSidebar and TestDocsAIOutputsInSync
|
||||
- `internal/docsite/docsite_test.go`: TestSlugIDs, TestReadmeIngestion, TestSnippetForms, TestSnippetConfinement and TestSyncRewritesDrift
|
||||
- `modules/bonfire/example_test.go`: ExampleCall
|
||||
- `docs/site.yaml`, `docs/index.md`, `docs/setup/installation.md`: the first content
|
||||
- `cmd/summer/main.go` and `main_test.go` (registration, want list and help flags), `.gitignore` (`/site/`), `README.md` (a `docs/` layout row and a docs:build/docs:sync paragraph in Development)
|
||||
|
||||
## Decisions Made
|
||||
|
||||
See `key-decisions` in the frontmatter. The most visible is the exported `docsite.Pages`, an addition to the planned interface that does not break it.
|
||||
|
||||
## Deviations from Plan
|
||||
|
||||
### Auto-fixed Issues
|
||||
|
||||
**1. [Rule 3 - Blocking] `api` section added to site.yaml in Task 2, not Task 1**
|
||||
- **Found during:** Task 1
|
||||
- **Issue:** With `api` listed in site.yaml before README ingestion existed (Task 2), the empty-section rule failed `TestDocsTree`. That would have broken `go test` at the Task 1 commit.
|
||||
- **Fix:** Task 1 lists only `setup`. Task 2 adds `api` in the same change as its first pages, which is the rule the build enforces.
|
||||
- **Files modified:** docs/site.yaml
|
||||
- **Commit:** 6dacddc, e433dcf
|
||||
|
||||
**2. [Rule 1 - Bug] Code-block renderer priority**
|
||||
- **Found during:** Task 3
|
||||
- **Issue:** At priority 2000 the custom fence renderer never replaced goldmark's default, because goldmark registers lower priority values last. No `<figure class="code">` was emitted.
|
||||
- **Fix:** Priority 100, with a comment explaining it.
|
||||
- **Files modified:** internal/docsite/render.go
|
||||
- **Commit:** dc6a03c
|
||||
|
||||
**3. [Scope note] Snippet check wired in load.go**
|
||||
- Task 3 lists docsite.go and render.go. The call to `checkSnippets` sits in `assemble` in load.go, where the rest of the load pipeline already lives. There is no behaviour difference.
|
||||
|
||||
**4. [Content] installation.md links the bonfire README**
|
||||
- The "Check your install" section links `../../modules/bonfire/README.md`, so the real tree exercises guide-to-README link rewriting.
|
||||
|
||||
---
|
||||
|
||||
**Total deviations:** 2 auto-fixed (1 blocking, 1 bug), 2 notes. **Impact:** none on scope. Every planned output, test and acceptance criterion is met.
|
||||
|
||||
## Issues Encountered
|
||||
|
||||
- `gofmt -l` lists `internal/build/registry.go`. This was already the case before this plan, so it was left alone.
|
||||
- The GSD HEAD assertion treats `master` as protected. The project runs with `branching_strategy: none`, and all earlier phase commits are on master, so this plan's commits are on master too. Setting `git.allow_default_branch_commits: true` would make that explicit.
|
||||
|
||||
## Verification
|
||||
|
||||
- `go vet ./...` is green, and so is `go test -short ./...` across the repository.
|
||||
- `go test ./internal/docsite ./cmd/summer ./modules/bonfire -count=1` is green. `ExampleCall` reports `--- PASS`.
|
||||
- `go run ./cmd/summer docs:build --out <tmp>` writes 24 pages. `docs:sync` reports all snippets up to date. A drifted scratch copy fails `--check` with `snippet: body differs from modules/bonfire/example_test.go#ExampleCall (run: summer docs:sync)`.
|
||||
- The built output contains no consuming-application names. Its only absolute URL is the source_url caption link to the framework's own forge, and it has no third-party asset origins.
|
||||
- `go.mod` is unchanged.
|
||||
|
||||
## User Setup Required
|
||||
|
||||
None. No external service configuration is required.
|
||||
|
||||
## Next Phase Readiness
|
||||
|
||||
Plan 11.1-02 can replace `page.html` with the full theme, reuse `headingIDs`/`slugIDs` for the anchor checker and `docsite.Pages` for the tree checks, and add `docs:serve`. The `Commands` option field is still to be added by 11.1-02, as planned.
|
||||
|
||||
## Self-Check: PASSED
|
||||
|
||||
All 14 created files exist. Commits 6dacddc, e433dcf and dc6a03c are in the log.
|
||||
|
||||
---
|
||||
*Phase: 11.1-summercms-documentation-for-humans-and-ai-agents*
|
||||
*Completed: 2026-09-30*
|
||||
Reference in New Issue
Block a user