feat: internal links widget with counts and suggestions

- E10 golem15-seo-links in the SEO tab after the graph preview (types
  with a body): inbound and outbound counts from the last finished crawl,
  the orphan warning, up to 10 pages to link to (Copy address) and 10
  that could link here
- classes/linkgraph.go builds the report from the crawl's pages and
  links; the crawler's orphan suggestions share NormalText
- GET /_seo/api/v1/links and seo_get_link_suggestions; widget action
  seo_links mapped in AdminOperations
- element bundle rebuilt
This commit is contained in:
Jakub Zych
2026-11-04 16:40:16 +01:00
parent ab4e83416d
commit 49cc241f58
20 changed files with 1131 additions and 12 deletions

View File

@@ -38,7 +38,7 @@ The frontend (a Nuxt layer or app) asks `GET /_seo/api/v1/resolve?url=...` for t
- **CSV import and export** of redirects, URL rules and the SEO of records (by URL), in the **Import and export** screen (element `golem15-seo-transfer`) and through MCP. Files are UTF-8 with a BOM, comma or semicolon; exported cells that start a formula (`=`, `+`, `-`, `@`, tab, CR), or apostrophes followed by one, are prefixed with an apostrophe, and the import drops exactly that one again, so a re-imported export is unchanged. An import is previewed row by row (new, changed, unchanged, refused, with notes) and applied as one batch with via `import`, or not at all while a row is refused. The screen's apply token is an HMAC-SHA256 under the app key over the file and a fingerprint of the data the preview read; it is refused with `transfer.stale` when the data changed. The screen previews files up to 60 KB and downloads exports up to 250 KB; MCP imports take up to 900 KB per call and exports come in pages of 1000 rows.
- **Record events.** A deleted record loses its entries after commit; a created record gets its address snapshot; an updated record whose address changed gets its 301.
- **Entry generation.** The `golem15.seo.generate_entries` job (conga) creates missing entries with the current address of every published record and refreshes the effective-value cache; a record whose address changed without a record event gets its address change (with its 301) like an admin save. Editor values are never changed.
- **Field types** `seotext`, `mlseotext`, `seorobots`, `mlseorobots`, `seoimage`, `robotstxt`, `seoschema` and `seoparent`, with the custom elements `golem15-seo-text`, `golem15-seo-robots`, `golem15-seo-serp` (the result preview widget), `golem15-seo-image`, `golem15-seo-robotstxt` (the robots.txt editor), `golem15-seo-robotstester` (the tester widget), `golem15-seo-transfer` (the import and export widget), `golem15-seo-schema` (structured data), `golem15-seo-graph` (the structured data preview widget) and `golem15-seo-parent` (the primary category).
- **Field types** `seotext`, `mlseotext`, `seorobots`, `mlseorobots`, `seoimage`, `robotstxt`, `seoschema` and `seoparent`, with the custom elements `golem15-seo-text`, `golem15-seo-robots`, `golem15-seo-serp` (the result preview widget), `golem15-seo-image`, `golem15-seo-robotstxt` (the robots.txt editor), `golem15-seo-robotstester` (the tester widget), `golem15-seo-transfer` (the import and export widget), `golem15-seo-schema` (structured data), `golem15-seo-graph` (the structured data preview widget), `golem15-seo-parent` (the primary category) and `golem15-seo-links` (the internal links widget).
- **Structured data.** Every `ok` resolve answer carries one JSON-LD `@graph` built from the content descriptor and the settings: the page node (WebPage or its subtype), the article with its authors, dates, images, word count and section, the person of a profile page, a Service with its provider and offer, the BreadcrumbList, one VideoObject per YouTube video, the WebSite (with a SearchAction when `search_url` is set) and the publisher. Editors pick the schema type per page, fill the Service fields and add custom JSON-LD; the SEO tab previews the graph with Google's required and recommended properties and links the Rich Results Test.
- **Publisher and local business** settings: an organisation or a person with logo, website, contact and sameAs profiles, optionally a local business with address, coordinates, opening hours and price range.
- **Breadcrumbs, hreflang and pagination.** The resolve answer carries the breadcrumb trail (through the record's primary category), the hreflang alternates with `x-default`, `html_lang` and `og:locale`, and the page of a paginated address in the site's form (`?page=N` or `/<segment>/N`).
@@ -51,6 +51,7 @@ The frontend (a Nuxt layer or app) asks `GET /_seo/api/v1/resolve?url=...` for t
- **Site answer** (`GET /_seo/api/v1/site`) with the facts the frontend layer needs, and **llms.txt** (off by default) listing the indexable pages from the sitemaps.
- **Links** settings: links to other sites open in a new tab (on by default), and domains whose links always get `nofollow` or `sponsored`; the resolve answer carries them as `outbound`.
- **Auto-links** (`golem15_seo_autolinks`, SEO › **Auto-links**): phrases that texts link to a page, per locale or for all, with an active switch, an origin and a note. Every `ok` resolve answer carries the phrases of its locale (never one linking to the page itself) as `autolinks`, with `autolink_max` from the Links tab (3 by default, 0 turns auto-linking off); the frontend layer links each phrase's first occurrence. **Add focus keyphrases** turns the records' focus keyphrases into phrases. See [Auto-links](#auto-links).
- **Internal links** in the SEO tab (E10): the page's inbound and outbound link counts from the last finished crawl, a warning when no page links to it, and up to 10 pages its text could link to and 10 pages that could link to it (titles and focus keyphrases found in the crawled text), each with **Copy address**; `GET /_seo/api/v1/links` and `seo_get_link_suggestions` answer the same.
- **Site cache purge.** Every SEO write (admin, MCP, batch, import, undo, automatic) asks the frontend to drop its cached pages after the commit, once per transaction, through the framework's site-cache purger (`site_cache.purge_url`, `site_cache.purge_token`).
## Usage
@@ -198,6 +199,8 @@ The **robots.txt** screen (a singleton) has the editor, the tester and **What cr
**SEO › Auto-links** (controller `golem15.seo.autolinks`, after Pages, icon `wand-sparkles`) lists the phrases (50 per page) with the address they link to, the language (**All languages** or a locale), the origin (**Manual**, **Focus keyphrase**, **MCP**) and the update time; an inactive phrase shows **Off**. Filters: language, origin and **Active only**. The form takes the phrase, the address, the language and the **Active** switch, and a note. A phrase needs 3 characters (`autolinks.phrase_short`); a phrase exists once per language, a phrase of every language once at all (`autolinks.phrase_taken` names the address it already links to); the address is a path starting with one `/` or a full `http(s)` address (`autolinks.url_invalid`: `javascript:`, `data:` and protocol-relative addresses are refused). The toolbar's **Add focus keyphrases** adds one phrase per record whose focus keyphrase in a locale is not yet a phrase there, linking to the record's address in that locale (origin **Focus keyphrase**), and toasts the count; a second run adds nothing new. Every change is recorded in History and purges the site's cached pages.
The SEO tab's **Internal links** (widget `golem15-seo-links`, action `seo_links`, after the structured data preview, on the update form of content types whose records have a body: not for collections, profiles or the page registry) reads the last finished crawl: **Links to this page** (pages linking here) and **Links from this page** (pages it links to on the site), a warning when nothing links here, **Pages you could link to from this text** (pages whose H1 or focus keyphrase the crawled text contains and that it does not link to yet, with the public address and **Copy address**) and **Pages that could link here** (pages whose crawled text contains this page's title, H1 or focus keyphrase and that do not link here yet, with their admin form), 10 each. Before the first crawl it says when the counts appear.
The layer links a phrase at its first occurrence in a text, as whole words and ignoring letter case, never inside headings, links, `code`, `pre`, `figcaption` or buttons, and never on the page it links to, at most `autolink_max` phrases per text (the Links tab's **Auto-links per page**, 0 to 20).
### The site answer and llms.txt
@@ -253,6 +256,7 @@ The management routes live under `/_seo/api/v1` (bucket `seo-api`, 300 requests
| `get_url_seo` | `GET /seo?url=` | Effective SEO of a URL with the source of every value, the redirect decision, the owner and the SEO tab's warnings. |
| `set_url_seo` | `PUT /seo` | Values for a URL: the record behind it in the URL's locale, else an exact URL rule; `force_rule` writes a rule. `primary_parent` (`<type>:<id>`) and `schema` (the structured data JSON string) are records only and shared by every locale. Answers `written_to` (`record` or `rule`), `type`, `id`, `locale`, `rule_id`. |
| `get_graph` | `GET /graph?url=&schema=` | The JSON-LD graph of a URL and every node's missing required and recommended properties (`rules` false for a type Google has no rules for); `schema` previews an unsaved value. Answers `{locale, url, graph, nodes}`. |
| `get_link_suggestions` | `GET /links?url=` | A page's internal links from the last finished crawl: `{crawledAt, inbound, outbound, orphan, out [{phrase, title, url}], in [{phrase, title, adminUrl}]}`, each list at most 10; `crawledAt`, `inbound` and `outbound` are `null` before the first crawl. |
| `list_rules`, `show_rule`, `create_rule`, `update_rule`, `delete_rule` | `GET/POST /rules`, `GET/PUT/DELETE /rules/{id}` | URL rules. |
| `list_redirects`, `show_redirect`, `create_redirect`, `update_redirect`, `delete_redirect` | `GET/POST /redirects`, `GET/PUT/DELETE /redirects/{id}` | Redirects; a chain is saved with a warning. |
| `list_not_found`, `redirect_not_found`, `gone_not_found`, `delete_not_found` | `GET /notfound`, `POST /notfound/{id}/redirect`, `POST /notfound/{id}/gone`, `DELETE /notfound/{id}` | The 404 log and its actions; an address a rule answers is `409`. |
@@ -283,7 +287,7 @@ Lists take `page`, `per_page` (at most 200) and `search`, and answer `{"data": [
- **Crawls** lists the runs newest first (started, status, pages, issues, started by, duration). The preview shows one status callout (running, queued, failed, cancelled, limit reached), the staging and rendered-comparison lines, the stats strip and the issues per kind, each count linking to the filtered Site audit; **Cancel crawl** while queued or running. There is no live progress: reload the page.
- The SEO settings' **Site audit** tab (before Verification): **Automatic crawl** (off, daily, weekly), **Pages per crawl** (10–50 000), **Pause between requests** (100–5000 ms), **Crawls kept** (1–50) and **Generic link texts** (seeded with the English and Polish lists).
- The SEO settings have a **Pagination** tab (after Addresses), a **Links** tab (**Auto-links per page**, 0 to 20, the new-tab switch and the nofollow and sponsored domain lists, one domain per line; `settings.domain_invalid` names a bad line), a **Publisher** tab with the generated publisher node and its findings, and a **Local business** tab. A person cannot be a local business (`settings.local_person`).
- The SEO tab adds **Primary category** (article types), **Structured data** and its **preview** after the social image; its status warns when a stored primary category is no longer assigned, and the effective values show the schema type and the breadcrumb trail.
- The SEO tab adds **Primary category** (article types), **Structured data** and its **preview** after the social image, then **Internal links** (types with a body); its status warns when a stored primary category is no longer assigned, and the effective values show the schema type and the breadcrumb trail.
- Settings tile **SEO settings** (`seo_settings`), a link to the settings singleton controller.
- Every SEO screen and the SEO tab require `golem15.seo.manage`. Without it an administrator sees no SEO tab, and a posted SEO value changes nothing.
@@ -308,6 +312,7 @@ The element bundle (`assets/dist/seo.js` and `seo.css`) is built from `assets/sr
| `schema.Build`, `schema.Input`, `schema.Validate`, `schema.Normalize`, `schema.DecodeCustom`, `schema.Merge` | Package `classes/schema`: the JSON-LD graph of a page, Google's required and recommended properties per type, the structured data value and custom JSON-LD. |
| `classes.Alternate`, `classes.Alternates`, `classes.LangTag` | hreflang alternates and language tags; `Service.OGLocale`. |
| `classes.Pagination`, `classes.PageConfig` | The resolve answer's pagination block and a site's page form (`Service.PageConfigOf`). |
| `classes.LinkReport`, `classes.LinkOut`, `classes.LinkIn`, `classes.RecordLinksAction`, `classes.LinksActionName`, `classes.MaxLinkSuggestions`, `classes.NormalText` | The internal links of a page (`Service.LinkSuggestions`, `Service.LinksOfURL`, `Service.LinksOfRecord`) and the SEO tab's `seo_links` action. |
| `classes.GraphAnswer` | `Service.GraphOfURL` and `Service.GraphOfRecord`, the graph and its findings; `classes.RecordGraphAction` is the SEO tab's preview action. |
| `classes.PrimaryParent`, `classes.ParentOption` | A record's primary category and its choices (`Service.ParentOptions`); `Service.Trail` builds the breadcrumb trail, `Service.PublisherOf` the publisher. |
| `classes.SitemapExclusion` | `Service.RecordSitemapExclusions`, why the sitemaps leave out a record's addresses; `Service.SitemapListed` reads the listed addresses; `Service.SitemapLines` the robots.txt `Sitemap:` addresses. |
@@ -392,4 +397,4 @@ npm ci && npm run typecheck && npm test && bash scripts/check-dist.sh
The element tests (`tests/elements`, vitest on happy-dom) drive the widget elements through a fake of the admin's widget host (`tests/elements/host.ts`), including a host that drops an action.
The smoke tests boot the plugin with fake content plugins (`acme.pages`, and `acme.blog` with a category segment for strict URLs) and exercise the SEO tab, the settings tabs, the resolver precedence, history, generation, the URL rule form, redirects (engine, loops and chains, address changes from the admin and from events), the 404 log, normalisation and strict URLs, robots.txt and verification files through the admin and public APIs, and the management API as an MCP client (`TestMCPSetURLSeoTracer`, `TestManagementAPI`, `TestIssues`), batches, undo and CSV (`TestBatch`, `TestUndo`, `TestCSVTransfer`, and the `mcp` and `import` cases of `TestLoopRefused`). The sitemap, the registry and the site-level endpoints are covered by `TestSitemapFilterTracer`, `TestFilterReasons`, `TestNoindexPublishedConflict`, `TestSitemapBlockedIssue`, `TestRegistry`, `TestRouteReport`, `TestSiteEndpoint`, `TestLLMSTxt`, `TestRobotsSitemapLine`, `TestSettingsLinks` and `TestPurgeAfterWrite`; the fixture boots the sitemap plugin next to the SEO plugin. The site audit is covered by `TestCrawlerTracer`, `TestCrawlerChecks` (a planted site with one page per kind and a clean control page, served to the crawler through `fetchguard.WithTransport`), `TestIssueMerge`, `TestStagingCrawl`, `TestRenderSkipped`, `TestCrawlAdmin`, `TestCrawlSchedule` and `TestCrawlTools`, with the fake content plugin `acme.crawl` naming fixing tools. Auto-links are covered by `TestAutolinksTracer` (the phrases of a page's locale in the resolve answer, never the page's own, `autolink_max` 0) and `TestAutolinks` (the screen, its refusals, Add focus keyphrases, the API and MCP). `TestChromeRenderer` renders a page in a real headless Chrome (it runs when `SEO_TEST_CHROME` names a binary or `chromium`, `chromium-browser` or `google-chrome` is on `PATH`, and is skipped otherwise) and `TestRendererGuards` checks the refused paths and private addresses. `TestEveryAdminOperationHasTool`, `TestMCPReadToolsAreGET`, `TestIssueKindLabels` and the static parts of `TestCrawlTools` and `TestCrawlSchedule` run without Docker.
The smoke tests boot the plugin with fake content plugins (`acme.pages`, and `acme.blog` with a category segment for strict URLs) and exercise the SEO tab, the settings tabs, the resolver precedence, history, generation, the URL rule form, redirects (engine, loops and chains, address changes from the admin and from events), the 404 log, normalisation and strict URLs, robots.txt and verification files through the admin and public APIs, and the management API as an MCP client (`TestMCPSetURLSeoTracer`, `TestManagementAPI`, `TestIssues`), batches, undo and CSV (`TestBatch`, `TestUndo`, `TestCSVTransfer`, and the `mcp` and `import` cases of `TestLoopRefused`). The sitemap, the registry and the site-level endpoints are covered by `TestSitemapFilterTracer`, `TestFilterReasons`, `TestNoindexPublishedConflict`, `TestSitemapBlockedIssue`, `TestRegistry`, `TestRouteReport`, `TestSiteEndpoint`, `TestLLMSTxt`, `TestRobotsSitemapLine`, `TestSettingsLinks` and `TestPurgeAfterWrite`; the fixture boots the sitemap plugin next to the SEO plugin. The site audit is covered by `TestCrawlerTracer`, `TestCrawlerChecks` (a planted site with one page per kind and a clean control page, served to the crawler through `fetchguard.WithTransport`), `TestIssueMerge`, `TestStagingCrawl`, `TestRenderSkipped`, `TestCrawlAdmin`, `TestCrawlSchedule` and `TestCrawlTools`, with the fake content plugin `acme.crawl` naming fixing tools. Auto-links are covered by `TestAutolinksTracer` (the phrases of a page's locale in the resolve answer, never the page's own, `autolink_max` 0) and `TestAutolinks` (the screen, its refusals, Add focus keyphrases, the API and MCP); `TestLinkSuggestions` crawls a planted site and checks E10's counts and suggestions through the API and the widget action. `TestChromeRenderer` renders a page in a real headless Chrome (it runs when `SEO_TEST_CHROME` names a binary or `chromium`, `chromium-browser` or `google-chrome` is on `PATH`, and is skipped otherwise) and `TestRendererGuards` checks the refused paths and private addresses. `TestEveryAdminOperationHasTool`, `TestMCPReadToolsAreGET`, `TestIssueKindLabels` and the static parts of `TestCrawlTools` and `TestCrawlSchedule` run without Docker.