An SEO-less site's robots.txt, sitemap.xml and llms.txt keep reaching
the app during an outage longer than the cache TTL; a backend that never
answered still gets the 503.
quick-261104-xae, review WR-04.
- rootFileTargets gives robots.txt, sitemap.xml and llms.txt their backend
target also in state unknown without a last answer, so the failing fetch
answers 503 with Retry-After: 60 instead of the app's 404 (L4 WR-08)
- an older backend (absent) still passes through; sitemap files and
verification files still need a site answer
- expect-seo-down.txt expects the 503s and states the staging reason
With the SEO backend unreachable since start, /robots.txt and /sitemap.xml
answered 503 where v0.1.1 left them to the app (a consumer's smoke expects
the app's 404). The root proxies now need a site answer; the last one seen
is kept while the backend is down, so a failure after it is still 503.
- server/middleware/seo-root.ts proxies /robots.txt, /sitemap.xml,
/sitemap-<name>.xml, /llms.txt and the verification and IndexNow key
files the site answer lists in root_files, with the backend's status,
content type and X-Robots-Tag; a backend failure is 503 Retry-After 60
- server/lib/site.ts caches GET /_seo/api/v1/site (300 s, cleared by the
purge); a backend without it (404) keeps the v0.1.1 behaviour: no root
proxies, the verification lookup by name
- the 404 report and the verification lookup honour seo.cms and rootFiles
- smoke runs for the layer off (seo.cms and the v0.1.1 enabled switch), the
CMS down (fail open) and no 404 token; npm run purge:check
