feat: rendered comparison through headless Chrome

- github.com/chromedp/chromedp v0.21.0 (decision note 2026-10-28-chromedp-rendered-crawl.md): with golem15.seo.crawl_chrome_path the crawl renders up to golem15.seo.crawl_render_max pages (200) and compares them with the raw HTML; without a path it records skipped - guard: site host and www twin resolved and checked with fetchguard.IsPrivateAddr, pinned through --host-resolver-rules with a final MAP * ~NOTFOUND; Fetch-domain interception fails every request outside the site (third-party scripts never load); fresh temporary profile removed on Close; downloads denied; one tab, 20 s, 5 MiB DOM - a relative or non-executable path, a private address or a Chrome that does not start records render_status failed; the crawl still finishes - config keys documented; README Rendered comparison section - TestChromeRenderer (runs with a local Chromium) and TestRendererGuards
2026-11-04 15:58:36 +01:00
parent 96963c3067
commit b553fc2937
10 changed files with 638 additions and 9 deletions
--- a/README.md
+++ b/README.md
@@ -221,7 +221,9 @@ A crawl is a conga job (`golem15.seo.crawl`, one attempt) started from **Start c

 **Checks** (kind, severity): `page_status` (error), `sitemap_error` (error), `sitemap_noindex` (error), `sitemap_blocked` (error), `sitemap_missing` (notice), `broken_link` (error, with the linking pages), `redirected_link` (notice), `canonical_missing` (warning), `canonical_error` (error), `missing_title` and `duplicate_title` (warning), `missing_description` and `duplicate_description` (notice), `h1_missing` and `h1_multiple` (error), `heading_skipped` and `heading_empty` (warning), `h1_title_mismatch` (notice), `jsonld_invalid` (error), `jsonld_multiple` (warning), `jsonld_required` (error) and `jsonld_recommended` (notice) from the same rules as the structured data preview (`schema.Validate`; FAQPage and other types without Google rules find nothing), `hreflang_reciprocal` (warning), `hreflang_error` (error), `generic_anchor` (warning, the settings' generic link texts, whole text, any case), `image_alt` (warning: an `<img>` without `alt`; `alt=""` is decorative), `image_size` and `featured_lazy` (notice), `content_js` (warning) and `orphan_page` (warning: an indexable sitemap page no crawled page links to, with up to 10 pages whose text mentions its title or focus keyphrase). The CMS-data checks run at the end of every crawl (and when the list is opened before any crawl) and are stored with the crawl's issues as source `cms`; a CMS issue the crawl found too is stored once.

-**Raw HTML against the CMS.** The server-rendered HTML is checked against the record behind the page (`content_js` with `details.check: cms`): the record's title must appear in an H1, the text must reach half of the record's word count, and the record's primary parent (the breadcrumb link) must be linked. The comparison with the JavaScript-rendered page is `details.check: rendered`; without a renderer every crawl records it as skipped (P6 and `seo_get_crawl` say so).
+**Raw HTML against the CMS.** The server-rendered HTML is checked against the record behind the page (`content_js` with `details.check: cms`): the record's title must appear in an H1, the text must reach half of the record's word count, and the record's primary parent (the breadcrumb link) must be linked. The comparison with the JavaScript-rendered page is `details.check: rendered` (next section); without a Chrome path every crawl records it as skipped (P6 and `seo_get_crawl` say so).
+
+**Rendered comparison.** When `golem15.seo.crawl_chrome_path` names a Chrome or Chromium binary the operator installed on the server (none is installed by default), the crawler renders the first `golem15.seo.crawl_render_max` pages (default 200, in crawl order, so the home page and the top-level pages come first) in headless Chrome through `github.com/chromedp/chromedp`, right after each page's raw fetch, and compares the DOM after JavaScript with the raw HTML: same-host links, an H1, or at least 50 words and 20 % more text that appear only after JavaScript are one `content_js` finding with `details.check: rendered` and `details.what` naming them (up to five links). The browser is held to the crawler's limits: before Chrome starts, the site host and its `www` twin are resolved and every address checked like the crawler's dial guard (`fetchguard.IsPrivateAddr`; a private address refuses unless `golem15.seo.crawl_trusted` is on), and Chrome is pinned to those addresses through `--host-resolver-rules` while every other host resolves to nothing; every request is intercepted and continued only when it is https (http only in trusted mode) to the site host or its twin, or a `data:`/`blob:` URL, so third-party scripts, fonts, analytics and metadata addresses are never loaded. Each crawl gets a fresh temporary profile, removed when the crawl ends, with no cookies, credentials or extensions and downloads denied; one tab renders at a time, 20 s per page, the DOM is capped at 5 MiB, Chrome's own sandbox stays on, and Chrome is closed when the crawl finishes, fails or is cancelled. The path is config only (or `SUMMER_GOLEM15__SEO__CRAWL_CHROME_PATH`), never a setting or MCP value. The crawl records `render_status` `done` with the pages compared, `skipped` without a path, or `failed` with the reason (a relative path or one that is not an executable file, a private address, Chrome not starting, a page that cannot be rendered); a renderer failure never fails the crawl, and P6, `GET /_seo/api/v1/crawls/{id}` and `seo_get_crawl` (`render {status, pages, note, message}`) say which.

 **Fix routes.** An issue a CMS value fixes names its tool: `seo_set_url_seo` (titles, descriptions, canonicals, sitemap membership of a record, structured data), `seo_update_robots`, `seo_create_redirect` (a missing page, a page that answers an error), `seo_create_page` (a page missing from the registry), `seo_update_settings` (publisher and site nodes), or the content type's own tool from `pact.ContentDescriptor.Tools` when the problem sits in the record's content: H1 findings prefer `Tools[headings]`, image alt text `Tools[image_alt]`, other content findings `Tools[edit]` (the registry pages name `seo_update_page`). A finding of the site's template code, or of content whose type names no fitting tool, is `fix: code` with `details.what` naming the element to change. No issue has neither.

@@ -304,7 +306,8 @@ The element bundle (`assets/dist/seo.js` and `seo.css`) is built from `assets/sr
 | `crawl.Check`, `crawl.Env`, `crawl.Finding`, `crawl.Route`, `crawl.Fingerprint`, `crawl.Kinds`, `crawl.Severity`, `crawl.Record` | The checks, a finding's fix route (tool or code) and identity, and the kinds with their severities. |
 | `crawl.Start`, `crawl.Run`, `crawl.Cancel`, `crawl.Recheck`, `crawl.Ignore`, `crawl.Unignore`, `crawl.DeleteCrawls`, `crawl.Prune`, `crawl.Current`, `crawl.ScheduledDue` | Starting, running and cancelling a crawl, checking an address again, ignores, deletes and the schedule's decision. |
 | `crawl.CrawlArgs`, `crawl.CrawlKind` | The crawl job's arguments and River kind `golem15.seo.crawl`. |
-| `crawl.Renderer`, `crawl.WithRenderer` | The rendered comparison's seam and its code-only test override. |
+| `crawl.Renderer`, `crawl.WithRenderer`, `crawl.NewRenderer`, `crawl.RenderMax` | The rendered comparison's seam, its code-only test override, the guarded Chrome renderer built from config and the configured page cap. |
+| `crawl.ChromePathConfigKey`, `crawl.RenderMaxConfigKey`, `crawl.TrustedConfigKey`, `crawl.RenderTimeout`, `crawl.MaxDOM` | The renderer's and crawler's config keys and the per-page render limits. |
 | `classes.IssueImageAlt`, `classes.IssueH1InContent` | The CMS-data kinds from the content descriptors (`Service.ContentIssues`). |
 | `classes.Service` | The single entry point for SEO reads and writes: `SaveRecord`, `Record` (history), `GenerateEntries`, `EnqueueGenerate`, `RecordDeleted`, `RecordCreated`, `Resolver`, `SiteURL`, `Staging`. Redirects: `SaveRedirect`, `DeleteRedirect`, `CheckRedirect`, `AddressChanges`, `AddressChanged`. 404 log: `ReportNotFound`, `ResolveNotFound`. robots.txt: `ServedFile`, `TestRobots`, `BlockedNoindex`, `SaveRobots`. `VerificationFileByName`. Management: `FindOwner`, `SetURLSEO`, `SaveRule`, `DeleteRule`, `SaveSettings`, `DeleteNotFound`, `Issues`. |
 | `classes.Owner`, `classes.URLSEOWrite`, `classes.URLSEOResult` | The owner of a URL (`classes.WrittenToRecord` or `classes.WrittenToRule`) and a write addressed by URL. |
@@ -341,6 +344,8 @@ The element bundle (`assets/dist/seo.js` and `seo.css`) is built from `assets/sr
 | `golem15.seo.search_url` | unset | The site search address with `{search_term_string}` (`/szukaj?q={search_term_string}`); set, the WebSite node carries a SearchAction. |
 | `golem15.seo.route_report_token` | `""` | The bearer token of `POST /_seo/api/v1/registry/routes`; empty uses `notfound_token`, and both empty turn the route report off. |
 | `golem15.seo.crawl_trusted` | `false` | Lets the crawler fetch plain http and private addresses (a staging or local site), still only on the site host and its `www` twin. Config only. |
+| `golem15.seo.crawl_chrome_path` | `""` | The absolute path of an operator-installed Chrome or Chromium binary for the rendered comparison; empty skips it. Config only. |
+| `golem15.seo.crawl_render_max` | `200` | The pages rendered per crawl, the first in crawl order. |

 The non-production lock reads the application environment (`SUMMER_ENV`); it has no key of its own. The address rules (host, trailing slash, lowercase) are settings, on the Addresses tab.

@@ -358,6 +363,7 @@ The non-production lock reads the application environment (`SUMMER_ENV`); it has
 - `gorm.io/gorm`, `github.com/go-gormigrate/gormigrate/v2`, `gocloud.dev/blob` (the media bucket for social images), `github.com/goccy/go-yaml` (the per-type settings tabs and robots form documents).
 - `github.com/jimsmart/grobotstxt` v1.0.3, the robots.txt matcher (decision note `.planning/notes/2026-10-27-robots-txt-grobotstxt.md` in the framework repository).
 - `golang.org/x/net/html`, the crawler's HTML parser (already required by the framework).
+- `github.com/chromedp/chromedp` v0.21.0 (with `github.com/chromedp/cdproto`), the optional rendered comparison's DevTools client; it drives an operator-installed Chrome and needs no cgo or Node (decision note `.planning/notes/2026-10-28-chromedp-rendered-crawl.md` in the framework repository).
 - Element toolchain (development only, exact pins): `typescript`, `vite`, `vitest`, `happy-dom`. The bundle has no runtime dependencies.

 ## Testing
@@ -371,4 +377,4 @@ npm ci && npm run typecheck && npm test && bash scripts/check-dist.sh

 The element tests (`tests/elements`, vitest on happy-dom) drive the widget elements through a fake of the admin's widget host (`tests/elements/host.ts`), including a host that drops an action.

-The smoke tests boot the plugin with fake content plugins (`acme.pages`, and `acme.blog` with a category segment for strict URLs) and exercise the SEO tab, the settings tabs, the resolver precedence, history, generation, the URL rule form, redirects (engine, loops and chains, address changes from the admin and from events), the 404 log, normalisation and strict URLs, robots.txt and verification files through the admin and public APIs, and the management API as an MCP client (`TestMCPSetURLSeoTracer`, `TestManagementAPI`, `TestIssues`), batches, undo and CSV (`TestBatch`, `TestUndo`, `TestCSVTransfer`, and the `mcp` and `import` cases of `TestLoopRefused`). The sitemap, the registry and the site-level endpoints are covered by `TestSitemapFilterTracer`, `TestFilterReasons`, `TestNoindexPublishedConflict`, `TestSitemapBlockedIssue`, `TestRegistry`, `TestRouteReport`, `TestSiteEndpoint`, `TestLLMSTxt`, `TestRobotsSitemapLine`, `TestSettingsLinks` and `TestPurgeAfterWrite`; the fixture boots the sitemap plugin next to the SEO plugin. The site audit is covered by `TestCrawlerTracer`, `TestCrawlerChecks` (a planted site with one page per kind and a clean control page, served to the crawler through `fetchguard.WithTransport`), `TestIssueMerge`, `TestStagingCrawl`, `TestRenderSkipped`, `TestCrawlAdmin`, `TestCrawlSchedule` and `TestCrawlTools`, with the fake content plugin `acme.crawl` naming fixing tools. `TestEveryAdminOperationHasTool`, `TestMCPReadToolsAreGET`, `TestIssueKindLabels` and the static parts of `TestCrawlTools` and `TestCrawlSchedule` run without Docker.
+The smoke tests boot the plugin with fake content plugins (`acme.pages`, and `acme.blog` with a category segment for strict URLs) and exercise the SEO tab, the settings tabs, the resolver precedence, history, generation, the URL rule form, redirects (engine, loops and chains, address changes from the admin and from events), the 404 log, normalisation and strict URLs, robots.txt and verification files through the admin and public APIs, and the management API as an MCP client (`TestMCPSetURLSeoTracer`, `TestManagementAPI`, `TestIssues`), batches, undo and CSV (`TestBatch`, `TestUndo`, `TestCSVTransfer`, and the `mcp` and `import` cases of `TestLoopRefused`). The sitemap, the registry and the site-level endpoints are covered by `TestSitemapFilterTracer`, `TestFilterReasons`, `TestNoindexPublishedConflict`, `TestSitemapBlockedIssue`, `TestRegistry`, `TestRouteReport`, `TestSiteEndpoint`, `TestLLMSTxt`, `TestRobotsSitemapLine`, `TestSettingsLinks` and `TestPurgeAfterWrite`; the fixture boots the sitemap plugin next to the SEO plugin. The site audit is covered by `TestCrawlerTracer`, `TestCrawlerChecks` (a planted site with one page per kind and a clean control page, served to the crawler through `fetchguard.WithTransport`), `TestIssueMerge`, `TestStagingCrawl`, `TestRenderSkipped`, `TestCrawlAdmin`, `TestCrawlSchedule` and `TestCrawlTools`, with the fake content plugin `acme.crawl` naming fixing tools. `TestChromeRenderer` renders a page in a real headless Chrome (it runs when `SEO_TEST_CHROME` names a binary or `chromium`, `chromium-browser` or `google-chrome` is on `PATH`, and is skipped otherwise) and `TestRendererGuards` checks the refused paths and private addresses. `TestEveryAdminOperationHasTool`, `TestMCPReadToolsAreGET`, `TestIssueKindLabels` and the static parts of `TestCrawlTools` and `TestCrawlSchedule` run without Docker.
--- a/classes/crawl/crawler.go
+++ b/classes/crawl/crawler.go
@@ -104,6 +104,9 @@ type Options struct {
 	Renderer Renderer
 	// RenderMax caps the rendered pages (default 200, crawl order).
 	RenderMax int
+	// RenderErr is why the renderer could not be built: the comparison is
+	// recorded as failed with it, and the raw checks run.
+	RenderErr error
 	// Canceled is asked between requests; true stops the crawl and keeps
 	// what was fetched.
 	Canceled func(ctx context.Context) bool
@@ -164,8 +167,11 @@ func Crawl(ctx context.Context, f *Fetcher, opt Options) (*Result, error) {
 		opt.RenderMax = DefaultRenderMax
 	}
 	res := &Result{Pages: map[string]*Page{}, Listed: map[string]string{}, Render: RenderState{Status: renderStatusSkipped}, fetcher: f}
-	if opt.Renderer != nil {
+	switch {
+	case opt.Renderer != nil:
 		res.Render.Status = renderStatusDone
+	case opt.RenderErr != nil:
+		res.Render = RenderState{Status: renderStatusFailed, Note: opt.RenderErr.Error()}
 	}
 	pace := &pacer{delay: opt.Delay}
 	canceled := func() bool {
--- a/classes/crawl/job.go
+++ b/classes/crawl/job.go
@@ -196,10 +196,22 @@ func run(ctx context.Context, svc *classes.Service, db *gorm.DB, row models.Craw
 	if app := svc.App(); app != nil {
 		m, _ = conga.From(app)
 	}
+	// The rendered comparison: the test seam's renderer, else Chrome from
+	// golem15.seo.crawl_chrome_path (none: skipped). Chrome is closed when
+	// the crawl finishes, fails or is cancelled.
+	renderer, renderErr := rendererFrom(ctx), error(nil)
+	if renderer == nil {
+		renderer, renderErr = NewRenderer(ctx, svc.Config(), f.Site(), f.IsTrusted(), f.UA())
+		if renderer != nil {
+			defer func() { _ = renderer.Close() }()
+		}
+	}
 	opt := Options{
-		MaxPages: limits.MaxPages,
-		Delay:    limits.Delay,
-		Renderer: rendererFrom(ctx),
+		MaxPages:  limits.MaxPages,
+		Delay:     limits.Delay,
+		Renderer:  renderer,
+		RenderErr: renderErr,
+		RenderMax: RenderMax(svc.Config()),
 		Canceled: func(ctx context.Context) bool {
 			var status string
 			if err := db.Raw(`SELECT status FROM golem15_seo_crawls WHERE id = ?`, row.ID).Scan(&status).Error; err == nil && status == models.CrawlCancelled {
--- a/classes/crawl/render.go
+++ b/classes/crawl/render.go
@@ -1,6 +1,18 @@
 package crawl

-import "context"
+import (
+	"context"
+	"fmt"
+	"net/netip"
+	"net/url"
+	"os"
+	"path/filepath"
+	"sort"
+	"strings"
+	"time"
+
+	"git.golem15.com/golem15/summercms/modules/compass"
+)

 // Renderer renders a page of the site in a browser for the raw versus
 // rendered comparison (D-70, P33-55): Render answers the page's DOM after
@@ -41,3 +53,140 @@ func rendererFrom(ctx context.Context) Renderer {
 	r, _ := ctx.Value(rendererKey{}).(Renderer)
 	return r
 }
+
+// The config keys of the rendered comparison (P33-55). The executable
+// path is Go config only (environment-overridable like every key), never
+// an admin setting or MCP value: an editable path would let any holder of
+// golem15.seo.manage make the server run an arbitrary binary.
+const (
+	// ChromePathConfigKey names an operator-installed Chrome or Chromium
+	// binary; empty skips the comparison.
+	ChromePathConfigKey = "golem15.seo.crawl_chrome_path"
+	// RenderMaxConfigKey caps the pages rendered per crawl.
+	RenderMaxConfigKey = "golem15.seo.crawl_render_max"
+)
+
+// The limits of one rendered page.
+const (
+	RenderTimeout = 20 * time.Second
+	MaxDOM        = 5 << 20
+)
+
+// RenderMax is the configured page cap of the comparison (default 200).
+func RenderMax(cfg *compass.Config) int {
+	if cfg != nil {
+		if n := cfg.Int(RenderMaxConfigKey); n > 0 {
+			return n
+		}
+	}
+	return DefaultRenderMax
+}
+
+// NewRenderer builds the renderer of a crawl of site from config: nil
+// without error when golem15.seo.crawl_chrome_path is empty (the
+// comparison is skipped); an error, recorded as the comparison's failure
+// note, when the path is not absolute, not an executable regular file, the
+// site host resolves to a private address outside trusted mode, or Chrome
+// does not start. ua is the crawler's User-Agent.
+func NewRenderer(ctx context.Context, cfg *compass.Config, site *url.URL, trusted bool, ua string) (Renderer, error) {
+	if cfg == nil || site == nil {
+		return nil, nil
+	}
+	path := strings.TrimSpace(cfg.String(ChromePathConfigKey))
+	if path == "" {
+		return nil, nil
+	}
+	if err := checkExecutable(path); err != nil {
+		return nil, err
+	}
+	hosts, err := resolveHosts(ctx, SiteHosts(site.Hostname()), trusted)
+	if err != nil {
+		return nil, err
+	}
+	r, err := newChromeRenderer(path, hosts, trusted, ua)
+	if err != nil {
+		return nil, err
+	}
+	return r, nil
+}
+
+// checkExecutable refuses a Chrome path that is not absolute or not an
+// executable regular file.
+func checkExecutable(path string) error {
+	if !filepath.IsAbs(path) {
+		return fmt.Errorf("the Chrome path %q is not absolute", path)
+	}
+	info, err := os.Stat(path)
+	if err != nil {
+		return fmt.Errorf("the Chrome path %q cannot be read", path)
+	}
+	if !info.Mode().IsRegular() || info.Mode().Perm()&0o111 == 0 {
+		return fmt.Errorf("the Chrome path %q is not an executable file", path)
+	}
+	return nil
+}
+
+// resolverRules is Chrome's --host-resolver-rules value: one MAP rule per
+// allowed host to one of its checked addresses (IPv4 first), then
+// MAP * ~NOTFOUND for every other host. Chrome applies the first rule
+// that matches a host, so the catch-all comes last.
+func resolverRules(hosts map[string][]netip.Addr) string {
+	names := make([]string, 0, len(hosts))
+	for h := range hosts {
+		names = append(names, h)
+	}
+	sort.Strings(names)
+	var rules []string
+	for _, h := range names {
+		addrs := hosts[h]
+		if len(addrs) == 0 {
+			continue
+		}
+		// One address per host: the first IPv4 one, else the first.
+		a := addrs[0]
+		for _, c := range addrs {
+			if c.Is4() {
+				a = c
+				break
+			}
+		}
+		target := a.String()
+		if a.Is6() {
+			target = "[" + target + "]"
+		}
+		rules = append(rules, "MAP "+h+" "+target)
+	}
+	rules = append(rules, "MAP * ~NOTFOUND")
+	return strings.Join(rules, ", ")
+}
+
+// allowRequest reports whether the headless browser may load rawURL: https
+// (http only in trusted mode) to one of hosts, or a data: or blob: URL.
+// Every other request, third-party scripts included, is failed.
+func allowRequest(rawURL string, hosts []string, trusted bool) bool {
+	u, err := url.Parse(rawURL)
+	if err != nil {
+		return false
+	}
+	switch strings.ToLower(u.Scheme) {
+	case "data", "blob":
+		return true
+	case "https":
+	case "http":
+		if !trusted {
+			return false
+		}
+	default:
+		return false
+	}
+	if u.User != nil {
+		return false
+	}
+	host := strings.ToLower(u.Hostname())
+	for _, h := range hosts {
+		if host == strings.ToLower(strings.Trim(h, "[]")) {
+			return true
+		}
+	}
+	return false
+}
--- a/classes/crawl/render_chrome.go
+++ b/classes/crawl/render_chrome.go
@@ -0,0 +1,224 @@
+package crawl
+
+import (
+	"context"
+	"fmt"
+	"net"
+	"net/netip"
+	"os"
+	"sort"
+	"strings"
+	"sync"
+
+	"git.golem15.com/golem15/summercms/modules/fetchguard"
+	"github.com/chromedp/cdproto/browser"
+	"github.com/chromedp/cdproto/cdp"
+	"github.com/chromedp/cdproto/fetch"
+	"github.com/chromedp/cdproto/network"
+	"github.com/chromedp/chromedp"
+)
+
+// chromeRenderer renders pages in an operator-installed headless Chrome
+// or Chromium through github.com/chromedp/chromedp (D-70, P33-55), held to
+// the crawler's fetchguard limits:
+//
+//   - Chrome resolves only the site host and its www twin, pinned through
+//     --host-resolver-rules to the addresses resolveHosts checked with
+//     fetchguard.IsPrivateAddr; every other host resolves to NOTFOUND;
+//   - every request is paused through the Fetch domain and continued only
+//     when allowRequest passes (https, http only in trusted mode, to an
+//     allowed host, or data: and blob:); the others fail with
+//     BlockedByClient, so third-party scripts are never loaded;
+//   - a fresh temporary profile per crawl, removed by Close; no cookies,
+//     credentials or extensions; downloads denied;
+//   - one tab at a time (the crawler serialises Render), 20 s per page, the
+//     DOM capped at 5 MiB; Close stops Chrome.
+//
+// chromedp's default allocator flags are used as they are: nothing is
+// added that weakens Chrome's sandbox.
+type chromeRenderer struct {
+	allowed []string
+	trusted bool
+	profile string
+
+	allocCancel   context.CancelFunc
+	browserCtx    context.Context
+	browserCancel context.CancelFunc
+
+	mu      sync.Mutex
+	blocked []string
+	closed  bool
+}
+
+// newChromeRenderer starts Chrome at path for the hosts and their checked
+// addresses.
+func newChromeRenderer(path string, hosts map[string][]netip.Addr, trusted bool, ua string) (*chromeRenderer, error) {
+	profile, err := os.MkdirTemp("", "summercms-seo-chrome-")
+	if err != nil {
+		return nil, fmt.Errorf("the Chrome profile cannot be created: %w", err)
+	}
+	r := &chromeRenderer{trusted: trusted, profile: profile}
+	for h := range hosts {
+		r.allowed = append(r.allowed, h)
+	}
+	sort.Strings(r.allowed)
+	opts := append(chromedp.DefaultExecAllocatorOptions[:],
+		chromedp.ExecPath(path),
+		chromedp.UserDataDir(profile),
+		chromedp.Flag("host-resolver-rules", resolverRules(hosts)),
+		chromedp.UserAgent(ua),
+	)
+	allocCtx, allocCancel := chromedp.NewExecAllocator(context.Background(), opts...)
+	r.allocCancel = allocCancel
+	r.browserCtx, r.browserCancel = chromedp.NewContext(allocCtx)
+	// The first Run starts the browser; it runs without a timeout, since a
+	// timeout on this context would stop the whole browser.
+	if err := chromedp.Do(r.browserCtx); err != nil {
+		_ = r.Close()
+		return nil, fmt.Errorf("Chrome did not start: %w", err)
+	}
+	if _, err := chromedp.CallBrowser(r.browserCtx, browser.SetDownloadBehavior, browser.SetDownloadBehaviorParams{Behavior: browser.SetDownloadBehaviorBehaviorDeny}); err != nil {
+		_ = r.Close()
+		return nil, fmt.Errorf("Chrome downloads cannot be denied: %w", err)
+	}
+	return r, nil
+}
+
+// Render loads pageURL in a new tab and answers the DOM after JavaScript
+// ran (the outer HTML of the document, capped at 5 MiB).
+func (r *chromeRenderer) Render(ctx context.Context, pageURL string) (string, error) {
+	r.mu.Lock()
+	closed := r.closed
+	r.mu.Unlock()
+	if closed {
+		return "", fmt.Errorf("Chrome is closed")
+	}
+	if !allowRequest(pageURL, r.allowed, r.trusted) {
+		return "", fmt.Errorf("the address %s is outside the site", pageURL)
+	}
+	tab, closeTab := chromedp.NewContext(r.browserCtx)
+	defer closeTab()
+	// The tab ends with the crawl's context (a cancelled crawl) or after
+	// the page's timeout.
+	stop := context.AfterFunc(ctx, closeTab)
+	defer stop()
+	if err := chromedp.Do(tab); err != nil {
+		return "", fmt.Errorf("Chrome cannot open a tab: %w", err)
+	}
+	page, cancel := context.WithTimeout(tab, RenderTimeout)
+	defer cancel()
+	target := chromedp.FromContext(tab).Target
+	paused := chromedp.Events(page, fetch.RequestPaused)
+	if _, err := chromedp.Call(page, fetch.Enable, fetch.EnableParams{Patterns: []*fetch.RequestPattern{{URLPattern: "*"}}}); err != nil {
+		return "", fmt.Errorf("Chrome cannot intercept requests: %w", err)
+	}
+	var wg sync.WaitGroup
+	wg.Add(1)
+	go func() {
+		defer wg.Done()
+		for ev, err := range paused {
+			if err != nil {
+				return
+			}
+			r.decide(page, target, ev)
+		}
+	}()
+	html, err := chromedp.Run(page, func(ctx context.Context, t *chromedp.Target) (string, error) {
+		if _, err := chromedp.Navigate(pageURL)(ctx, t); err != nil {
+			return "", err
+		}
+		if _, err := chromedp.WaitReady(chromedp.CSS("body"))(ctx, t); err != nil {
+			return "", err
+		}
+		return chromedp.OuterHTML(chromedp.CSS("html"))(ctx, t)
+	})
+	cancel()
+	closeTab()
+	wg.Wait()
+	if err != nil {
+		if ctx.Err() != nil {
+			return "", ctx.Err()
+		}
+		return "", fmt.Errorf("Chrome could not render %s: %w", pageURL, err)
+	}
+	if len(html) > MaxDOM {
+		html = truncateUTF8(html, MaxDOM)
+	}
+	return html, nil
+}
+
+// decide continues a paused request that allowRequest passes and fails
+// every other one with BlockedByClient, keeping its address.
+func (r *chromeRenderer) decide(ctx context.Context, target cdp.Session, ev fetch.EventRequestPaused) {
+	u := ""
+	if ev.Request != nil {
+		u = ev.Request.URL
+	}
+	if allowRequest(u, r.allowed, r.trusted) {
+		_, _ = cdp.Call(ctx, target, fetch.ContinueRequest, fetch.ContinueRequestParams{RequestID: ev.RequestID})
+		return
+	}
+	r.mu.Lock()
+	r.blocked = append(r.blocked, u)
+	r.mu.Unlock()
+	_, _ = cdp.Call(ctx, target, fetch.FailRequest, fetch.FailRequestParams{RequestID: ev.RequestID, ErrorReason: network.ErrorReasonBlockedByClient})
+}
+
+// Blocked lists the addresses the browser was kept from loading.
+func (r *chromeRenderer) Blocked() []string {
+	r.mu.Lock()
+	defer r.mu.Unlock()
+	return append([]string(nil), r.blocked...)
+}
+
+// ProfileDir is the temporary profile of this crawl's browser.
+func (r *chromeRenderer) ProfileDir() string { return r.profile }
+
+// Close stops Chrome and removes the temporary profile.
+func (r *chromeRenderer) Close() error {
+	r.mu.Lock()
+	if r.closed {
+		r.mu.Unlock()
+		return nil
+	}
+	r.closed = true
+	r.mu.Unlock()
+	if r.browserCancel != nil {
+		r.browserCancel()
+	}
+	if r.allocCancel != nil {
+		r.allocCancel()
+	}
+	return os.RemoveAll(r.profile)
+}
+
+// resolveHosts resolves the site host and its www twin. A private or
+// reserved address (fetchguard.IsPrivateAddr) refuses outside trusted
+// mode; a twin that does not resolve is left out.
+func resolveHosts(ctx context.Context, hosts []string, trusted bool) (map[string][]netip.Addr, error) {
+	out := map[string][]netip.Addr{}
+	for i, host := range hosts {
+		if addr, err := netip.ParseAddr(strings.Trim(host, "[]")); err == nil {
+			if !trusted && fetchguard.IsPrivateAddr(addr) {
+				return nil, fmt.Errorf("the site host %s is a private address", addr)
+			}
+			out[host] = []netip.Addr{addr.Unmap()}
+			continue
+		}
+		addrs, err := net.DefaultResolver.LookupNetIP(ctx, "ip", host)
+		if err != nil || len(addrs) == 0 {
+			if i == 0 {
+				return nil, fmt.Errorf("the site host %s does not resolve", host)
+			}
+			continue
+		}
+		for _, a := range addrs {
+			a = a.Unmap()
+			if !trusted && fetchguard.IsPrivateAddr(a) {
+				return nil, fmt.Errorf("the site host %s resolves to the private address %s", host, a)
+			}
+			out[host] = append(out[host], a)
+		}
+	}
+	return out, nil
+}
--- a/config/config.yaml
+++ b/config/config.yaml
@@ -51,3 +51,17 @@ route_report_token: ""
 # twin, and refuses private addresses. Only config (or the environment,
 # SUMMER_GOLEM15__SEO__CRAWL_TRUSTED) sets it, never a setting or MCP.
 crawl_trusted: false
+#
+# The site audit can compare each page's raw HTML with the page after
+# JavaScript ran, in a headless Chrome or Chromium that the operator
+# installs on the server (none is installed by default). Set the absolute
+# path of its executable here or through the environment
+# (SUMMER_GOLEM15__SEO__CRAWL_CHROME_PATH); empty skips the comparison and
+# every crawl says so. The path is config only, never a setting or MCP
+# value. Chrome loads only the site host and its www twin (pinned to
+# addresses checked like the crawler's), so third-party scripts are not
+# loaded; every crawl gets a fresh profile, removed afterwards.
+crawl_chrome_path: ""
+#
+# The number of pages rendered per crawl, the first in crawl order.
+crawl_render_max: 200
--- a/crawl_render_test.go
+++ b/crawl_render_test.go
@@ -0,0 +1,207 @@
+package seo_test
+
+import (
+	"fmt"
+	"net/http"
+	"net/http/httptest"
+	"net/url"
+	"os"
+	"os/exec"
+	"path/filepath"
+	"slices"
+	"strings"
+	"testing"
+
+	"git.golem15.com/golem15/sm-seo-plugin/classes/crawl"
+	"git.golem15.com/golem15/summercms/modules/compass"
+)
+
+// chromeBinary is the Chrome or Chromium of the test machine: SEO_TEST_CHROME,
+// else chromium, chromium-browser or google-chrome on PATH.
+func chromeBinary() string {
+	if p := strings.TrimSpace(os.Getenv("SEO_TEST_CHROME")); p != "" {
+		return p
+	}
+	for _, name := range []string{"chromium", "chromium-browser", "google-chrome"} {
+		if p, err := exec.LookPath(name); err == nil {
+			if abs, err := filepath.EvalSymlinks(p); err == nil {
+				return abs
+			}
+			return p
+		}
+	}
+	return ""
+}
+
+// renderConfig is a config with the rendered comparison's keys.
+func renderConfig(t *testing.T, env ...string) *compass.Config {
+	t.Helper()
+	dir := t.TempDir()
+	if err := os.WriteFile(filepath.Join(dir, "app.yaml"), []byte("name: seo-render-test\n"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	cfg, err := compass.Open(compass.Options{Dir: dir, Environ: env})
+	if err != nil {
+		t.Fatal(err)
+	}
+	return cfg
+}
+
+// jsSite is a site whose page /page adds a same-host link and two
+// paragraphs after JavaScript, loads a third-party script and asks a
+// link-local address; hits counts the requests that reached it.
+func jsSite(t *testing.T) (*httptest.Server, *[]string) {
+	t.Helper()
+	var hits []string
+	page := `<!doctype html><html lang="en"><head><title>JS page</title><link rel="canonical" href="/page">
+<script src="https://cdn.example.test/x.js"></script></head>
+<body><main><h1>JS page</h1><p>Raw text.</p></main>
+<script>
+try { var x = new XMLHttpRequest(); x.open("GET", "http://169.254.169.254/latest/meta-data", false); x.send(); } catch (e) {}
+var main = document.querySelector("main");
+var a = document.createElement("a"); a.href = "/js-only"; a.textContent = "Only after JavaScript"; main.appendChild(a);
+for (var i = 0; i < 2; i++) { var p = document.createElement("p"); p.textContent = "Generated paragraph text ".repeat(20); main.appendChild(p); }
+</script></body></html>`
+	srv := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
+		hits = append(hits, r.URL.Path)
+		w.Header().Set("Content-Type", "text/html; charset=utf-8")
+		switch r.URL.Path {
+		case "/":
+			fmt.Fprint(w, `<!doctype html><html lang="en"><head><title>Home</title><link rel="canonical" href="/"></head><body><main><h1>Home</h1><a href="/page">The page</a></main></body></html>`)
+		case "/page":
+			fmt.Fprint(w, page)
+		default:
+			http.NotFound(w, r)
+		}
+	}))
+	t.Cleanup(srv.Close)
+	return srv, &hits
+}
+
+// TestChromeRenderer renders a page in headless Chrome (D-70, P33-55):
+// the JavaScript-added link and text are in the DOM and the crawl reports
+// them as content_js check rendered naming /js-only; the third-party script
+// and the link-local request are blocked and never reach a server; the
+// temporary profile is gone after Close.
+func TestChromeRenderer(t *testing.T) {
+	if testing.Short() {
+		t.Skip("needs Chrome and Docker")
+	}
+	chrome := chromeBinary()
+	if chrome == "" {
+		t.Skip("no Chrome or Chromium on this machine (set SEO_TEST_CHROME)")
+	}
+	srv, hits := jsSite(t)
+	u, _ := url.Parse(srv.URL)
+	site := "http://localhost:" + u.Port()
+	siteURL, _ := url.Parse(site)
+
+	cfg := renderConfig(t, "SUMMER_GOLEM15__SEO__CRAWL_CHROME_PATH="+chrome, "SUMMER_GOLEM15__SEO__CRAWL_TRUSTED=true")
+	r, err := crawl.NewRenderer(t.Context(), cfg, siteURL, true, crawl.UserAgent(site))
+	if err != nil {
+		t.Fatalf("NewRenderer: %v", err)
+	}
+	inspect, ok := r.(interface {
+		Blocked() []string
+		ProfileDir() string
+	})
+	if !ok {
+		t.Fatalf("renderer %T has no Blocked and ProfileDir", r)
+	}
+	profile := inspect.ProfileDir()
+	if _, err := os.Stat(profile); err != nil {
+		t.Fatalf("profile %s: %v", profile, err)
+	}
+	dom, err := r.Render(t.Context(), site+"/page")
+	if err != nil {
+		t.Fatalf("Render: %v", err)
+	}
+	if !strings.Contains(dom, `href="/js-only"`) || !strings.Contains(dom, "Generated paragraph text") {
+		t.Fatalf("rendered DOM lacks the JavaScript content:\n%s", dom)
+	}
+	blocked := strings.Join(inspect.Blocked(), "\n")
+	for _, want := range []string{"https://cdn.example.test/x.js", "http://169.254.169.254/latest/meta-data"} {
+		if !strings.Contains(blocked, want) {
+			t.Fatalf("blocked = %q, want %s", blocked, want)
+		}
+	}
+	if slices.ContainsFunc(*hits, func(p string) bool { return strings.Contains(p, "latest") || strings.Contains(p, "x.js") }) {
+		t.Fatalf("a blocked request reached the site: %v", *hits)
+	}
+	if err := r.Close(); err != nil {
+		t.Fatal(err)
+	}
+	if _, err := os.Stat(profile); !os.IsNotExist(err) {
+		t.Fatalf("profile %s still exists after Close: %v", profile, err)
+	}
+
+	// The crawl with the Chrome path compares the page with its rendering.
+	env := bootCrawlEnv(t,
+		"SUMMER_GOLEM15__SEO__SITE_URL="+site,
+		"SUMMER_GOLEM15__SEO__CRAWL_TRUSTED=true",
+		"SUMMER_GOLEM15__SEO__CRAWL_CHROME_PATH="+chrome,
+	)
+	token := env.admin(t, true)
+	id := env.crawlOnce(t, token, t.Context())
+	got := env.expect(t, http.StatusOK, http.MethodGet, fmt.Sprintf("/_seo/api/v1/crawls/%d", id), token, nil).Body.String()
+	if !jsonContains(got, `"render":{"status":"done","pages":2`) {
+		t.Fatalf("crawl render = %s", got)
+	}
+	issue := findIssue(t, env.issues(t, token, ""), "content_js", "/page")
+	if issue.Details["check"] != "rendered" || !strings.Contains(fmt.Sprint(issue.Details["what"]), "/js-only") {
+		t.Fatalf("content_js = %+v", issue)
+	}
+}
+
+// TestRendererGuards checks the renderer's refusals (T-33-66, T-33-69):
+// an empty Chrome path skips the comparison, a relative or non-executable
+// path fails it, and outside trusted mode a site host that resolves to a
+// private address refuses before Chrome starts; a crawl with an invalid
+// path records render_status failed with the note and still runs its raw
+// checks.
+func TestRendererGuards(t *testing.T) {
+	site, _ := url.Parse("https://localhost")
+	exe, err := exec.LookPath("sh")
+	if err != nil {
+		t.Skip("no sh")
+	}
+	exe, _ = filepath.EvalSymlinks(exe)
+	notExec := filepath.Join(t.TempDir(), "chrome")
+	if err := os.WriteFile(notExec, []byte("x"), 0o644); err != nil {
+		t.Fatal(err)
+	}
+	for _, tc := range []struct {
+		path, want string
+	}{
+		{"", ""},
+		{"bin/chrome", "is not absolute"},
+		{notExec, "is not an executable file"},
+		{exe, "private address"},
+	} {
+		cfg := renderConfig(t, "SUMMER_GOLEM15__SEO__CRAWL_CHROME_PATH="+tc.path)
+		r, err := crawl.NewRenderer(t.Context(), cfg, site, false, "test")
+		if tc.want == "" {
+			if r != nil || err != nil {
+				t.Fatalf("empty path: %v %v", r, err)
+			}
+			continue
+		}
+		if r != nil || err == nil || !strings.Contains(err.Error(), tc.want) {
+			t.Fatalf("path %q: renderer %v, error %v, want %q", tc.path, r, err, tc.want)
+		}
+	}
+	if testing.Short() {
+		return
+	}
+	env := bootCrawlEnv(t, "SUMMER_GOLEM15__SEO__CRAWL_CHROME_PATH=bin/chrome")
+	token := env.admin(t, true)
+	fake := newFakeSite()
+	fake.sitemap("/")
+	fake.plant(doc{path: "/", title: "Home", h1: []string{"Home", "Again"}})
+	id := env.crawlOnce(t, token, siteCtx(t, fake))
+	got := env.expect(t, http.StatusOK, http.MethodGet, fmt.Sprintf("/_seo/api/v1/crawls/%d", id), token, nil).Body.String()
+	if !jsonContains(got, `"status":"finished"`) || !jsonContains(got, `"render":{"status":"failed"`) || !strings.Contains(got, "is not absolute") {
+		t.Fatalf("crawl with a relative Chrome path = %s", got)
+	}
+	findIssue(t, env.issues(t, token, ""), "h1_multiple", "/")
+}
--- a/go.mod
+++ b/go.mod
@@ -5,6 +5,8 @@ go 1.27.0
 require (
 	git.golem15.com/golem15/sm-sitemap-plugin v0.1.2
 	git.golem15.com/golem15/summercms v0.6.0
+	github.com/chromedp/cdproto v0.157.8
+	github.com/chromedp/chromedp v0.21.0
 	github.com/go-gormigrate/gormigrate/v2 v2.1.7
 	github.com/goccy/go-yaml v1.19.2
 	github.com/jackc/pgx/v5 v5.10.0
@@ -74,6 +76,7 @@ require (
 	github.com/felixge/httpsnoop v1.1.0 // indirect
 	github.com/fsnotify/fsnotify v1.10.1 // indirect
 	github.com/gabriel-vasile/mimetype v1.4.15 // indirect
+	github.com/go-json-experiment/json v0.0.0-20260213210345-44df1a37e875 // indirect
 	github.com/go-logr/logr v1.4.3 // indirect
 	github.com/go-logr/stdr v1.2.2 // indirect
 	github.com/go-ole/go-ole v1.3.0 // indirect
--- a/go.sum
+++ b/go.sum
@@ -24,6 +24,8 @@ cloud.google.com/go/storage v1.61.3 h1:VS//ZfBuPGDvakfD9xyPW1RGF1Vy3BWUoVZXgW1KM
 cloud.google.com/go/storage v1.61.3/go.mod h1:JtqK8BBB7TWv0HVGHubtUdzYYrakOQIsMLffZ2Z/HWk=
 dario.cat/mergo v1.0.2 h1:85+piFYR1tMbRrLcDwR18y4UKJ3aH1Tbzi24VRW1TK8=
 dario.cat/mergo v1.0.2/go.mod h1:E/hbnu0NxMFBjpMIE34DRGLWqDy0g5FuKDhCb31ngxA=
+git.golem15.com/golem15/sm-sitemap-plugin v0.1.2 h1:BiMOv5vEb4QNbeQjwUxE11GrZND9NmJawVBVfuHevII=
+git.golem15.com/golem15/sm-sitemap-plugin v0.1.2/go.mod h1:vuczz5X4og278Q82YSXCIHLI0x3Jp3x4Cgbhgc9WpFE=
 github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6 h1:He8afgbRMd7mFxO99hRNu+6tazq8nFF9lIwo9JFroBk=
 github.com/AdaLogics/go-fuzz-headers v0.0.0-20240806141605-e8a1dd7889d6/go.mod h1:8o94RPi1/7XTJvwPpRSzSUedZrtlirdB3r9Z20bi2f8=
 github.com/Azure/go-ansiterm v0.0.0-20250102033503-faa5f7b0171c h1:udKWzYgxTojEKWjV8V+WSxDXJ4NFATAsZjh8iIbsQIg=
@@ -122,6 +124,10 @@ github.com/charmbracelet/x/windows v0.2.2 h1:IofanmuvaxnKHuV04sC0eBy/smG6kIKrWG2
 github.com/charmbracelet/x/windows v0.2.2/go.mod h1:/8XtdKZzedat74NQFn0NGlGL4soHB0YQZrETF96h75k=
 github.com/charmbracelet/x/xpty v0.1.3 h1:eGSitii4suhzrISYH50ZfufV3v085BXQwIytcOdFSsw=
 github.com/charmbracelet/x/xpty v0.1.3/go.mod h1:poPYpWuLDBFCKmKLDnhBp51ATa0ooD8FhypRwEFtH3Y=
+github.com/chromedp/cdproto v0.157.8 h1:Tkq0AjsUlC3Z4sJfHBzV/uAAxnTF/AZyuSTjfV/1gvk=
+github.com/chromedp/cdproto v0.157.8/go.mod h1:h68JaIuFr9o3yiayxEYhBdvaxgYO3shDrPSYhNlyfu0=
+github.com/chromedp/chromedp v0.21.0 h1:nVHgS2LHyhZJ1l3BWRbG2eFIFYiJMnvkJbELbd8WBoQ=
+github.com/chromedp/chromedp v0.21.0/go.mod h1:6z1Y5FdKUxRzxx2qcC5h1xjaBJrSWrZp1mjgITfNZig=
 github.com/clipperhouse/displaywidth v0.11.0 h1:lBc6kY44VFw+TDx4I8opi/EtL9m20WSEFgwIwO+UVM8=
 github.com/clipperhouse/displaywidth v0.11.0/go.mod h1:bkrFNkf81G8HyVqmKGxsPufD3JhNl3dSqnGhOoSD/o0=
 github.com/clipperhouse/uax29/v2 v2.7.0 h1:+gs4oBZ2gPfVrKPthwbMzWZDaAFPGYK72F0NJv2v7Vk=
@@ -171,6 +177,8 @@ github.com/go-gormigrate/gormigrate/v2 v2.1.7 h1:PdT4jVPbRb4R+0Ey2R0yJOdctVf4Whi
 github.com/go-gormigrate/gormigrate/v2 v2.1.7/go.mod h1:3ouXglTuPrKF5+7cQyVGfvAXTU4vLMaYh9+EPl03uog=
 github.com/go-jose/go-jose/v4 v4.1.4 h1:moDMcTHmvE6Groj34emNPLs/qtYXRVcd6S7NHbHz3kA=
 github.com/go-jose/go-jose/v4 v4.1.4/go.mod h1:x4oUasVrzR7071A4TnHLGSPpNOm2a21K9Kf04k1rs08=
+github.com/go-json-experiment/json v0.0.0-20260213210345-44df1a37e875 h1:XGPtjxqp0FS8jj1Z7QGcSOSgyq1fUPjNyJXvi3KfI4A=
+github.com/go-json-experiment/json v0.0.0-20260213210345-44df1a37e875/go.mod h1:uNVvRXArCGbZ508SxYYTC5v1JWoz2voff5pm25jU1Ok=
 github.com/go-logr/logr v1.2.2/go.mod h1:jdQByPbusPIv2/zmleS9BjJVeZ6kBagPoEUsqbVz/1A=
 github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
 github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
--- a/plugin.yaml
+++ b/plugin.yaml
@@ -7,6 +7,6 @@ name: SEO
 description: "The SEO plugin for SummerCMS applications: an SEO tab on every content type, per-URL rules, redirects and the 404 log, robots.txt with AI bot switches, global and per-type SEO settings, change history, a site audit with a built-in crawler, and the redirect decision and effective SEO of any URL for the frontend."
 framework: v0.6.0
 requires:
-  golem15.sitemap: ""
+  golem15.sitemap: v0.1.2
 overrides:
  name: SEO