feat(docsite): check, caption and sync src= blocks in module READMEs (quick-261118-qsd)
- checkSnippets compares top-level src= fences of module READMEs like docs/ pages; the README src= refusal is gone - Sync also rewrites drifted src= fences in modules/<name>/README.md, still all-or-nothing across pages and READMEs - fenceAnnotator captions top-level src= fences on every page - the snippet-readme-src planted violation now plants a drifted README fence instead of the old refusal - utilities.md, README.md and CLAUDE.md describe src= in module READMEs
This commit is contained in:
@@ -22,7 +22,7 @@ cd examples/hello && go test ./... # separate workspace module, not co
|
||||
go install ./cmd/summer # the CLI
|
||||
go test ./cmd/summer -run TestDocsTree # docs checker
|
||||
summer docs:build --check # docs checker (identifiers, links, anchors, src= snippets, command names)
|
||||
summer docs:sync # refresh src= code copies in docs/ after changing their source
|
||||
summer docs:sync # refresh src= code copies in docs/ and module READMEs after changing their source
|
||||
|
||||
npm --prefix admin ci
|
||||
npm --prefix admin run test | test:browser | typecheck | build | dev
|
||||
|
||||
@@ -165,7 +165,7 @@ npm --prefix admin run gen:api # regenerate TypeScript types from admin/open
|
||||
|
||||
`summer dev` watches an application's sources and rebuilds its binary on change.
|
||||
|
||||
`summer docs:build` renders `docs/` and every module README into a static site under `site/` (use `--out` for another directory, `--check` to validate without writing). Code blocks with a `src=` reference are copies of real source, and `src=` works on top-level code blocks of `docs/` pages only; after changing that source, run `summer docs:sync` to refresh the copies.
|
||||
`summer docs:build` renders `docs/` and every module README into a static site under `site/` (use `--out` for another directory, `--check` to validate without writing). Code blocks with a `src=` reference are copies of real source, and `src=` works on top-level code blocks of `docs/` pages and module READMEs; after changing that source, run `summer docs:sync` to refresh the copies.
|
||||
|
||||
`summer docs:serve` builds the same site into a temporary directory and previews it at `http://127.0.0.1:8088` (`--addr` to change it), rebuilding when the docs, a module or a `src=` source changes; a failed rebuild prints its problems and keeps serving the last good build. It listens only on a loopback address unless you pass `--allow-remote`. Search needs this server: browsers block the search index over `file://`.
|
||||
|
||||
|
||||
@@ -34,7 +34,7 @@ These docs are Markdown files under `docs/`, plus every module README, built int
|
||||
| Command | Flags | Purpose |
|
||||
|---------|-------|---------|
|
||||
| `summer docs:build` | `--root` (default `.`), `--src`, `--out`, `--base-url`, `--site-url`, `--site-label`, `--check` | Checks every page and writes the site to `site/` (or `--out`): HTML pages, a raw `.md` copy of each page, `llms.txt`, `llms-full.txt` and the search index. With `--check` it only reports problems and writes nothing. |
|
||||
| `summer docs:sync` | `--root` (default `.`), `--src` | Rewrites every code block that has a `src=` reference from its source file. |
|
||||
| `summer docs:sync` | `--root` (default `.`), `--src` | Rewrites every code block that has a `src=` reference, in docs pages and module READMEs, from its source file. |
|
||||
| `summer docs:serve` | `--root` (default `.`), `--src`, `--base-url`, `--site-url`, `--site-label`, `--addr` (default `127.0.0.1:8088`), `--allow-remote` | Builds the site into a temporary directory, serves it and rebuilds when a page, a module or a referenced source changes. A failed rebuild prints its problems and keeps serving the last good build. Ctrl+C stops it and exits 0. |
|
||||
|
||||
`docs/site.yaml` accepts two optional keys, `site_url` and `site_label`, that add a link back to the main site to every page header. With `site_url: https://acme.example/` the link reads "acme.example". Without `site_label` the label is the URL's host, or Home when `site_url` is a path such as `/`. `site_url` must be an `http://` or `https://` URL with a host, or a path starting with a single `/`. The `--site-url` and `--site-label` flags override the two keys the way `--base-url` overrides `base_url`.
|
||||
@@ -45,6 +45,6 @@ summer docs:sync
|
||||
summer docs:serve
|
||||
```
|
||||
|
||||
`summer help docs:build` and `summer help docs:sync` show examples. `docs:build` fails, and writes nothing, when a page names an identifier that does not exist, links to a missing page or anchor, shows a command that neither `summer` nor an application binary has, or has a `src=` code block that differs from its source. It also fails on a Go code block (including `golang`) without a `src=` reference, a `src=` code block inside a callout, blockquote or list item (`src=` blocks must be top-level), and a `src=` code block in a module README. A `src=` target must be code `go test ./...` compiles and runs, that is a file in the default build, inside a `Test` function or an `Example` with an `// Output:` comment, or a function one of them calls. After you change code that a page shows, run `summer docs:sync` to refresh the copies.
|
||||
`summer help docs:build` and `summer help docs:sync` show examples. `docs:build` fails, and writes nothing, when a page names an identifier that does not exist, links to a missing page or anchor, shows a command that neither `summer` nor an application binary has, or has a `src=` code block that differs from its source. It also fails on a Go code block (including `golang`) without a `src=` reference and on a `src=` code block inside a callout, blockquote or list item; `src=` works on top-level code blocks of docs pages and module READMEs. A `src=` target must be code `go test ./...` compiles and runs, that is a file in the default build, inside a `Test` function or an `Example` with an `// Output:` comment, or a function one of them calls. After you change code that a page shows, run `summer docs:sync` to refresh the copies.
|
||||
|
||||
`docs:serve` listens only on a loopback address unless you pass `--allow-remote`. Use it to preview search, which browsers block when you open the built files directly from disk.
|
||||
|
||||
@@ -504,6 +504,9 @@ func TestNestedFenceChecks(t *testing.T) {
|
||||
"> ```",
|
||||
"",
|
||||
}, "\n")
|
||||
// A module README's src= fences are checked like a docs/ page's: a
|
||||
// matching body passes, drift, nesting and refused targets are
|
||||
// reported at the README's own lines.
|
||||
readme := strings.Join([]string{
|
||||
"# fixture",
|
||||
"",
|
||||
@@ -515,15 +518,42 @@ func TestNestedFenceChecks(t *testing.T) {
|
||||
"x := 1",
|
||||
"```",
|
||||
"",
|
||||
"```go src=modules/fixture/fixture.go",
|
||||
"package fixture",
|
||||
"```go src=modules/fixture/example_test.go#ExampleHello",
|
||||
"fmt.Println(\"hello\")",
|
||||
"// Output: hello",
|
||||
"```",
|
||||
"",
|
||||
"```go src=modules/fixture/example_test.go#ExampleHello",
|
||||
"stale",
|
||||
"```",
|
||||
"",
|
||||
"- item",
|
||||
"",
|
||||
" ```go src=modules/fixture/example_test.go#ExampleHello",
|
||||
" stale",
|
||||
" ```",
|
||||
"",
|
||||
"> [!NOTE]",
|
||||
"> ```go src=modules/fixture/example_test.go#ExampleHello",
|
||||
"> ```",
|
||||
"",
|
||||
"```text src=modules/fixture/.env",
|
||||
"```",
|
||||
"",
|
||||
"```go src=modules/fixture/example_test.go#ExampleSilent",
|
||||
"```",
|
||||
"",
|
||||
}, "\n")
|
||||
exampleTest := "package fixture\n\nimport \"fmt\"\n\n" +
|
||||
"func ExampleHello() {\n\tfmt.Println(\"hello\")\n\t// Output: hello\n}\n\n" +
|
||||
"func ExampleSilent() {\n\tfmt.Println(\"quiet\")\n}\n"
|
||||
const hello = "modules/fixture/example_test.go#ExampleHello"
|
||||
assertProblems(t, checkFixture(t, fixtureCommands, map[string]string{
|
||||
"docs/index.md": page("Acme docs", "index", 0, "Text.\n"),
|
||||
"docs/setup/start.md": page("Start", "setup", 10, body),
|
||||
"modules/fixture/README.md": readme,
|
||||
"go.mod": "module example.com/acme\n\ngo 1.27\n",
|
||||
"docs/index.md": page("Acme docs", "index", 0, "Text.\n"),
|
||||
"docs/setup/start.md": page("Start", "setup", 10, body),
|
||||
"modules/fixture/README.md": readme,
|
||||
"modules/fixture/example_test.go": exampleTest,
|
||||
}), []string{
|
||||
"docs/setup/start.md:10: snippet: go code block has no src= reference",
|
||||
"docs/setup/start.md:16: snippet: go code block has no src= reference",
|
||||
@@ -531,7 +561,11 @@ func TestNestedFenceChecks(t *testing.T) {
|
||||
"docs/setup/start.md:24: snippet: main.go code block has no src= reference",
|
||||
`docs/setup/start.md:34: command: "bogus:nested" is not a summer or application command`,
|
||||
"docs/setup/start.md:37: snippet: config/app.yaml: " + nestedSrcMessage,
|
||||
"modules/fixture/README.md:11: snippet: modules/fixture/fixture.go: " + readmeSrcMessage,
|
||||
"modules/fixture/README.md:16: snippet: body differs from " + hello + " (run: summer docs:sync)",
|
||||
"modules/fixture/README.md:22: snippet: " + hello + ": " + nestedSrcMessage,
|
||||
"modules/fixture/README.md:27: snippet: " + hello + ": " + nestedSrcMessage,
|
||||
"modules/fixture/README.md:30: snippet: modules/fixture/.env: path names a dotfile or .env file",
|
||||
"modules/fixture/README.md:33: snippet: modules/fixture/example_test.go#ExampleSilent: ExampleSilent has no // Output: comment, so go test compiles it but never runs it",
|
||||
})
|
||||
}
|
||||
|
||||
|
||||
@@ -59,10 +59,11 @@ func (fenceAnnotator) Transform(doc *ast.Document, reader text.Reader, pc parser
|
||||
if !entering || !ok || fc.Info == nil {
|
||||
return ast.WalkContinue, nil
|
||||
}
|
||||
// Captions only on the fences checkSnippets compares: top-level
|
||||
// src= fences of docs/ pages. A nested or module README fence is
|
||||
// not verified, so it must not claim a source.
|
||||
verified := pctx != nil && pctx.page != nil && pctx.page.Module == "" &&
|
||||
// Captions only on the fences checkSnippets compares: the
|
||||
// top-level src= fences of every page, docs/ pages and module
|
||||
// READMEs alike. A nested fence is refused, not verified, so it
|
||||
// must not claim a source.
|
||||
verified := pctx != nil && pctx.page != nil &&
|
||||
fc.Parent() != nil && fc.Parent().Kind() == ast.KindDocument
|
||||
if !verified {
|
||||
return ast.WalkContinue, nil
|
||||
|
||||
@@ -71,7 +71,7 @@ func renderHTML(t *testing.T, s *site, p *Page) renderedPage {
|
||||
return r
|
||||
}
|
||||
|
||||
func TestFenceCaptionsOnlyVerified(t *testing.T) {
|
||||
func TestFenceCaptionsTopLevelSrc(t *testing.T) {
|
||||
guide := page("Start", "setup", 10, strings.Join([]string{
|
||||
"```go src=pkg/lib.go#Greeting",
|
||||
"func Greeting() {}",
|
||||
@@ -83,7 +83,8 @@ func TestFenceCaptionsOnlyVerified(t *testing.T) {
|
||||
"> ```",
|
||||
"",
|
||||
}, "\n"))
|
||||
readme := "# alpha\n\nAlpha does one thing.\n\n```go src=modules/alpha/alpha.go\npackage alpha\n```\n"
|
||||
readme := "# alpha\n\nAlpha does one thing.\n\n```go src=modules/alpha/alpha.go\npackage alpha\n```\n\n" +
|
||||
"> [!TIP]\n> ```go src=modules/alpha/alpha.go#Nested\n> package alpha\n> ```\n"
|
||||
s, pages := renderFixture(t, map[string]string{
|
||||
"docs/setup/start.md": guide,
|
||||
"modules/alpha/README.md": readme,
|
||||
@@ -95,9 +96,14 @@ func TestFenceCaptionsOnlyVerified(t *testing.T) {
|
||||
if !strings.Contains(guideHTML, "<figcaption>pkg/lib.go#Greeting</figcaption>") {
|
||||
t.Fatalf("guide caption missing the top-level reference\n%s", guideHTML)
|
||||
}
|
||||
// A module README captions its top-level src= fence like a docs/
|
||||
// page; the fence nested in the callout gets none.
|
||||
readmeHTML := string(renderHTML(t, s, pages["api/alpha"]).html)
|
||||
if strings.Contains(readmeHTML, "<figcaption>") {
|
||||
t.Fatalf("module README has a caption\n%s", readmeHTML)
|
||||
if got := strings.Count(readmeHTML, "<figcaption>"); got != 1 {
|
||||
t.Fatalf("module README figcaptions = %d, want 1\n%s", got, readmeHTML)
|
||||
}
|
||||
if !strings.Contains(readmeHTML, "<figcaption>modules/alpha/alpha.go</figcaption>") {
|
||||
t.Fatalf("module README caption missing the top-level reference\n%s", readmeHTML)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -530,14 +530,11 @@ type drift struct {
|
||||
// its container marker, so the fence is refused instead of extracted.
|
||||
const nestedSrcMessage = "src= code block must be a top-level block of the page, not inside a callout, blockquote or list item"
|
||||
|
||||
// readmeSrcMessage is the problem for a src= fence in a module README.
|
||||
// Those fences are rendered as written and never compared, so a src=
|
||||
// there would claim a check that does not run.
|
||||
const readmeSrcMessage = "module README code blocks are rendered as written; src= is verified only in docs/ pages"
|
||||
|
||||
// checkFences verifies every src= fence in fences. lines are the body
|
||||
// lines the fences index into. lineBase is the 1-based file line of
|
||||
// lines[0]; file is the display path for problems.
|
||||
// checkFences verifies every src= fence in fences, on a docs/ page or a
|
||||
// module README alike: a top-level fence is extracted and compared with
|
||||
// its source, a nested one is refused. lines are the body lines the
|
||||
// fences index into. lineBase is the 1-based file line of lines[0]; file
|
||||
// is the display path for problems.
|
||||
func checkFences(root, file string, lines []string, lineBase int, fences []mdFence) ([]drift, []Problem, error) {
|
||||
var drifts []drift
|
||||
var problems []Problem
|
||||
@@ -596,9 +593,10 @@ func unindent(l string, n int) string {
|
||||
return l[i:]
|
||||
}
|
||||
|
||||
// checkSnippets verifies the src= fences of every page. A docs/ page is
|
||||
// compared with its source. A module README is rendered as written, so
|
||||
// any src= fence there is refused instead of checked.
|
||||
// checkSnippets verifies the src= fences of every page, docs/ pages and
|
||||
// module READMEs alike: each top-level src= fence is compared with its
|
||||
// source. A README page keeps its file's line indices (BodyLine 1), so
|
||||
// problems cite the README's own lines.
|
||||
func (s *site) checkSnippets(docs []parsedDoc) ([]Problem, error) {
|
||||
var problems []Problem
|
||||
for _, d := range docs {
|
||||
@@ -609,17 +607,6 @@ func (s *site) checkSnippets(docs []parsedDoc) ([]Problem, error) {
|
||||
if err != nil {
|
||||
return nil, err
|
||||
}
|
||||
if d.page.Module != "" {
|
||||
for _, f := range fences {
|
||||
ref, ok := ParseSrc(f.info)
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
problems = append(problems, Problem{File: d.file, Line: d.line + f.line, Rule: "snippet",
|
||||
Message: ref.String() + ": " + readmeSrcMessage})
|
||||
}
|
||||
continue
|
||||
}
|
||||
_, ps, err := checkFences(s.opts.Root, d.file, strings.Split(string(d.body), "\n"), d.line, fences)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
@@ -629,9 +616,10 @@ func (s *site) checkSnippets(docs []parsedDoc) ([]Problem, error) {
|
||||
return problems, nil
|
||||
}
|
||||
|
||||
// Sync rewrites every drifted src= fence body under Src from its source,
|
||||
// preserving everything else byte for byte. When a reference is missing or
|
||||
// breaks a rule, it returns the problems and writes nothing.
|
||||
// Sync rewrites every drifted src= fence body from its source, in the
|
||||
// pages under Src and in every modules/<name>/README.md under Root,
|
||||
// preserving everything else byte for byte. When a reference anywhere is
|
||||
// missing or breaks a rule, it returns the problems and writes nothing.
|
||||
func Sync(opts Options) (SyncResult, []Problem, error) {
|
||||
opts, err := opts.normalize()
|
||||
if err != nil {
|
||||
@@ -642,6 +630,20 @@ func Sync(opts Options) (SyncResult, []Problem, error) {
|
||||
if err != nil {
|
||||
return SyncResult{}, nil, err
|
||||
}
|
||||
// Module READMEs come from Root, as load ingests them. A module
|
||||
// without a README is skipped here; load reports it.
|
||||
names, err := moduleNames(opts.Root)
|
||||
if err != nil {
|
||||
return SyncResult{}, nil, err
|
||||
}
|
||||
for _, name := range names {
|
||||
readme := filepath.Join(opts.Root, "modules", name, "README.md")
|
||||
if _, err := os.Stat(readme); err == nil {
|
||||
files = append(files, readme)
|
||||
} else if !errors.Is(err, fs.ErrNotExist) {
|
||||
return SyncResult{}, nil, fmt.Errorf("docs:sync: %w", err)
|
||||
}
|
||||
}
|
||||
type rewrite struct {
|
||||
path string
|
||||
mode fs.FileMode
|
||||
|
||||
@@ -478,3 +478,94 @@ func TestSyncParsedFences(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
}
|
||||
|
||||
// treeContents reads every file under root, keyed by path.
|
||||
func treeContents(t *testing.T, root string) map[string]string {
|
||||
t.Helper()
|
||||
files := map[string]string{}
|
||||
err := filepath.WalkDir(root, func(p string, d os.DirEntry, err error) error {
|
||||
if err != nil || d.IsDir() {
|
||||
return err
|
||||
}
|
||||
b, err := os.ReadFile(p)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
files[p] = string(b)
|
||||
return nil
|
||||
})
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
return files
|
||||
}
|
||||
|
||||
func TestSyncModuleReadme(t *testing.T) {
|
||||
const alphaTest = "package alpha\n\nimport \"fmt\"\n\n" +
|
||||
"func ExampleHello() {\n\tfmt.Println(\"hello\")\n\t// Output: hello\n}\n"
|
||||
readme := strings.Join([]string{
|
||||
"# alpha",
|
||||
"",
|
||||
"Alpha does one thing.",
|
||||
"",
|
||||
"`import \"example.com/acme/modules/alpha\"`",
|
||||
"",
|
||||
"## Usage",
|
||||
"",
|
||||
"Prose before.",
|
||||
"",
|
||||
"```go src=modules/alpha/alpha_test.go#ExampleHello",
|
||||
"stale",
|
||||
"```",
|
||||
"",
|
||||
"Prose `alpha` after.",
|
||||
"",
|
||||
"```sh",
|
||||
"summer docs:build",
|
||||
"```",
|
||||
"",
|
||||
}, "\n")
|
||||
tree := map[string]string{
|
||||
"modules/alpha/alpha.go": "package alpha\n",
|
||||
"modules/alpha/alpha_test.go": alphaTest,
|
||||
"modules/alpha/README.md": readme,
|
||||
// A module directory without a README is skipped; load reports it.
|
||||
"modules/beta/beta.go": "package beta\n",
|
||||
}
|
||||
root := snippetTree(t, tree)
|
||||
path := filepath.Join(root, "modules/alpha/README.md")
|
||||
res, problems, err := Sync(Options{Root: root})
|
||||
if err != nil || len(problems) > 0 || res != (SyncResult{Snippets: 1, Files: 1}) {
|
||||
t.Fatalf("Sync = %+v %q %v", res, problemLines(problems), err)
|
||||
}
|
||||
got, err := os.ReadFile(path)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
want := strings.Replace(readme, "stale\n", "fmt.Println(\"hello\")\n// Output: hello\n", 1)
|
||||
if string(got) != want {
|
||||
t.Fatalf("synced README =\n%s\nwant\n%s", got, want)
|
||||
}
|
||||
if res, problems, err := Sync(Options{Root: root}); err != nil || len(problems) > 0 || res != (SyncResult{}) {
|
||||
t.Fatalf("second Sync = %+v %q %v", res, problemLines(problems), err)
|
||||
}
|
||||
|
||||
// A refused README fence blocks every write, docs/ pages included.
|
||||
tree["modules/alpha/README.md"] = readme + "\n```text src=modules/alpha/.env\n```\n"
|
||||
tree["docs/setup/start.md"] = page("Start", "setup", 10, "```go src=pkg/lib.go#Greeting\nstale\n```\n")
|
||||
root = snippetTree(t, tree)
|
||||
before := treeContents(t, root)
|
||||
res, problems, err = Sync(Options{Root: root})
|
||||
if err != nil || res != (SyncResult{}) {
|
||||
t.Fatalf("Sync with a refused README fence = %+v, %v", res, err)
|
||||
}
|
||||
assertProblems(t, problemLines(problems), []string{
|
||||
"modules/alpha/README.md:21: snippet: modules/alpha/.env: path names a dotfile or .env file",
|
||||
})
|
||||
after := treeContents(t, root)
|
||||
for p, b := range before {
|
||||
if after[p] != b {
|
||||
t.Errorf("Sync changed %s although a README reference was refused", p)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -19,6 +19,6 @@ go test ./modules/demo
|
||||
```
|
||||
|
||||
```go src=modules/demo/example_test.go#ExampleHello
|
||||
fmt.Println(demo.Hello("blog"))
|
||||
// Output: Hello, blog
|
||||
fmt.Println(demo.Hello("world"))
|
||||
// Output: Hello, world
|
||||
```
|
||||
|
||||
@@ -1,3 +1,3 @@
|
||||
rule: snippet
|
||||
file: modules/demo/README.md
|
||||
message: modules/demo/example_test.go#ExampleHello: module README code blocks are rendered as written
|
||||
message: body differs from modules/demo/example_test.go#ExampleHello (run: summer docs:sync)
|
||||
|
||||
Reference in New Issue
Block a user