b553fc2937
- github.com/chromedp/chromedp v0.21.0 (decision note 2026-10-28-chromedp-rendered-crawl.md): with golem15.seo.crawl_chrome_path the crawl renders up to golem15.seo.crawl_render_max pages (200) and compares them with the raw HTML; without a path it records skipped - guard: site host and www twin resolved and checked with fetchguard.IsPrivateAddr, pinned through --host-resolver-rules with a final MAP * ~NOTFOUND; Fetch-domain interception fails every request outside the site (third-party scripts never load); fresh temporary profile removed on Close; downloads denied; one tab, 20 s, 5 MiB DOM - a relative or non-executable path, a private address or a Chrome that does not start records render_status failed; the crawl still finishes - config keys documented; README Rendered comparison section - TestChromeRenderer (runs with a local Chromium) and TestRendererGuards
68 lines
3.2 KiB
YAML
68 lines
3.2 KiB
YAML
# golem15.seo defaults, read as golem15.seo.<key>.
|
|
#
|
|
# site_url is the public origin of the site (for example
|
|
# https://example.com). Canonical, og:url and every other absolute URL the
|
|
# resolver answers start with it; when it is empty, app.url is used. The
|
|
# request's Host header is never used.
|
|
site_url: ""
|
|
#
|
|
# unprefixed_locale names the locale whose public URLs carry no locale
|
|
# prefix (for example pl on a site whose English pages start with /en).
|
|
# Empty leaves the choice to each content type.
|
|
unprefixed_locale: ""
|
|
#
|
|
# notfound_token is the shared secret the frontend server sends as
|
|
# "Authorization: Bearer <token>" with each 404 report to
|
|
# POST /_seo/api/v1/notfound. Empty turns the endpoint off (it answers 404).
|
|
notfound_token: ""
|
|
#
|
|
# notfound_max_rows caps the 404 log; new addresses beyond it are dropped,
|
|
# with one warning an hour.
|
|
notfound_max_rows: 10000
|
|
#
|
|
# robots_sitemaps lists the sitemap addresses robots.txt names, one
|
|
# "Sitemap:" line each, made absolute on the site URL. Unset, robots.txt
|
|
# names the sitemap plugin's index (its site address plus /sitemap.xml);
|
|
# the per-type files are never listed, the index names them. Set it only
|
|
# to name other sitemaps:
|
|
# robots_sitemaps: ["/sitemap.xml"]
|
|
#
|
|
# route_report_token is the shared secret the frontend server sends as
|
|
# "Authorization: Bearer <token>" with its route report to
|
|
# POST /_seo/api/v1/registry/routes (the page registry). Empty uses
|
|
# notfound_token; both empty turn the report off (it answers 404).
|
|
route_report_token: ""
|
|
#
|
|
# og_locales maps locale codes to their og:locale values (the resolve
|
|
# answer's seo.og.locale and locale_alternates). Codes not listed use
|
|
# pl_PL, en_US and de_DE for pl, en and de, else the language with its
|
|
# region (pt-BR: pt_BR) or the language doubled (fr: fr_FR):
|
|
# og_locales: {pl: pl_PL, en: en_GB}
|
|
#
|
|
# search_url is the site search address with the {search_term_string}
|
|
# placeholder, for example /szukaj?q={search_term_string}. When set, the
|
|
# WebSite node of the JSON-LD graph carries a SearchAction. Empty means no
|
|
# SearchAction:
|
|
# search_url: "/szukaj?q={search_term_string}"
|
|
#
|
|
# crawl_trusted lets the site audit's crawler fetch plain http addresses
|
|
# and addresses inside the network (a staging or local site). Without it
|
|
# the crawler fetches only https addresses of the site host and its www
|
|
# twin, and refuses private addresses. Only config (or the environment,
|
|
# SUMMER_GOLEM15__SEO__CRAWL_TRUSTED) sets it, never a setting or MCP.
|
|
crawl_trusted: false
|
|
#
|
|
# The site audit can compare each page's raw HTML with the page after
|
|
# JavaScript ran, in a headless Chrome or Chromium that the operator
|
|
# installs on the server (none is installed by default). Set the absolute
|
|
# path of its executable here or through the environment
|
|
# (SUMMER_GOLEM15__SEO__CRAWL_CHROME_PATH); empty skips the comparison and
|
|
# every crawl says so. The path is config only, never a setting or MCP
|
|
# value. Chrome loads only the site host and its www twin (pinned to
|
|
# addresses checked like the crawler's), so third-party scripts are not
|
|
# loaded; every crawl gets a fresh profile, removed afterwards.
|
|
crawl_chrome_path: ""
|
|
#
|
|
# The number of pages rendered per crawl, the first in crawl order.
|
|
crawl_render_max: 200
|