fix: root files follow the last site answer, a 404 included, while the backend is down
An SEO-less site's robots.txt, sitemap.xml and llms.txt keep reaching the app during an outage longer than the cache TTL; a backend that never answered still gets the 503. quick-261104-xae, review WR-04.
This commit is contained in:
@@ -7,10 +7,12 @@
|
||||
// of reading a missing robots.txt as "everything allowed". Any other path
|
||||
// passes through.
|
||||
//
|
||||
// Once the backend has given its site answer (SEO v0.2.0; the last one
|
||||
// seen is kept while the backend is down) every file above is proxied.
|
||||
// While the backend has not answered since the server started, only
|
||||
// robots.txt, sitemap.xml and llms.txt are: the failing fetch answers 503
|
||||
// Once the backend has given its site answer (SEO v0.2.0) every file above
|
||||
// is proxied. The last answer, ok or absent, is kept while the backend is
|
||||
// down, so on a backend without the SEO plugin the files keep reaching the
|
||||
// app during an outage. Only while the backend has not answered since the
|
||||
// server started or the last purge are robots.txt, sitemap.xml and
|
||||
// llms.txt proxied by their fixed names: the failing fetch answers 503
|
||||
// with Retry-After, so a staging site never exposes the app's 404 as an
|
||||
// allow-all robots.txt (L4 WR-08). With a backend before v0.2.0 (the site
|
||||
// answer is 404) nothing is proxied here, as in v0.1.1: robots.txt and the
|
||||
|
||||
Reference in New Issue
Block a user