Commit Graph

6 Commits

Author SHA1 Message Date
Jakub Zych
f58d55555c fix: IndexNow hears of deleted and edited records
RecordDeleted marks the listed rows it excludes and RecordSaved the rows
whose lastmod it moves (indexnow_pending_at); the next rebuild sends
them to IndexNow as removed or changed and clears the marks. RecordSaved
checks for a nil event before reading it. TestIndexNow now deletes and
edits through the record events, as the admin does.
2026-11-04 23:02:09 +01:00
Jakub Zych
2a05516036 feat: sitemap management API and MCP tools
- GET and PUT /_sitemap/api/v1/settings and POST /_sitemap/api/v1/rebuild
  resolve the administrator from a backend token, a personal API token or
  the MCP dispatch and require the sitemap permission (401/403 otherwise);
  bucket sitemap-api
- PUT changes the fields sent with the settings screen's checks (422
  validation_failed, nothing stored), merges the HTML sitemap addresses,
  writes content types, generates the IndexNow key and queues a rebuild
- GET answers the settings, the content types, the index files with their
  counts, the left-out counts, the IndexNow state and the News source
  values
- MCP tools sitemap_get_settings, sitemap_update_settings and
  sitemap_rebuild bound to those routes, listed with the permission
- the content type settings write and the file listing move to classes,
  shared by the form, S1, the index and the API; S1 and the index now list
  the image and News files
2026-11-04 12:38:35 +01:00
Jakub Zych
8f4ff2a97d feat: HTML sitemap groups API
- GET /_sitemap/api/v1/html?locale= answers the overview: the listed
  addresses of the XML sitemaps by group, in the order pages, offers,
  services, categories (as a tree), posts (newest first), tags, authors,
  other content types, each with its title in the locale, total, the
  first html_page_size links and more
- GET /_sitemap/api/v1/html/{group}?locale=&page= answers one page of a
  group; past the last page, an unknown group or the HTML sitemap off
  answer 404; every answer is noindex; per-IP bucket sitemap-html
- the HTML sitemap tab: html_enabled, the address per locale (html_paths)
  and html_page_size 50 to 500, with errors.html_path and errors.page_size
- contract.Listing.Settings answers HTMLEnabled, HTMLPaths, HTMLPageSize
  and KeyFile
2026-11-04 12:32:47 +01:00
Jakub Zych
e02d0b173f feat: IndexNow key, key file and pings
- saving with IndexNow on generates a 32-hex key (crypto/rand);
  indexnow_new_key generates a new one, so the old key file answers 404
- GET /_sitemap/indexnow/{file} serves <key>.txt as text/plain while
  IndexNow is on; the SEO layer maps it to the site root
- after each rebuild in production the added, changed and removed
  addresses on the site host are queued as golem15.sitemap.indexnow jobs
  (at most 10 000 per request, five attempts)
- the job posts {host, key, urlList} to api.indexnow.org through a
  fetchguard client limited to that host and records the answer; 429 and
  5xx retry, 400, 403 and 422 do not
- the IndexNow tab shows the S2 status partial: key or no key, last
  notification or never, and the 403, 422 and 429 callouts
2026-11-04 12:26:47 +01:00
Jakub Zych
a10ce2d9de feat: Google News sitemap
- sitemap-news.xml lists the articles (content kind article) first
  published in the last two days, newest first, at most 1000, with the
  publication name (news_publication, else app.name), the language of the
  locale, the publication date and the title
- news_sources narrows it to type:<key> and category:<key>:<id> values;
  the Images and News tab lists the values to use in a partial
- the index lists the file while news_enabled is on; otherwise it is 404
2026-11-04 12:22:40 +01:00
Jakub Zych
5bf56fde57 feat: image sitemap files (tracer)
- migration 202610280004: image_file_no, kind, published_at and parent_refs
  on the slot table; the image, News, IndexNow and HTML sitemap settings
- the rebuild stores each page's featured and body images (absolute on the
  site, deduplicated, at most 1000) and numbers the rows with images into
  sitemap-images-<n>.xml with the per-type files' stable rules
- the index lists the image files while images_enabled is on; the files
  route serves them with image:image and image:loc only
2026-11-04 12:19:05 +01:00