feat: batches with dry run, undo, and CSV import and export
- POST /_seo/api/v1/batch runs changes in one transaction: dry_run answers the diff and the conflicts (loops, chains, noindex on a published record, canonicals to redirected or noindex addresses) and rolls back; a refused change refuses the batch (D-12) - undo of a change or a whole batch from History and through MCP, with the undo_stale, undo_loop and undo_gone refusals (D-13) - CSV export and import of redirects, rules and records by URL: BOM, comma or semicolon, formula cells escaped, a dry-run preview, all or nothing with via import; the screen's apply token is HMAC-SHA256 over the file and a fingerprint of the data (transfer.stale) - Import and export singleton with the seo_transfer widget action, History record actions undo and undo_batch, navigation entry - MCP tools undo_change, undo_batch, apply_batch, export_csv, import_csv - TestBatch, TestUndo, TestCSVTransfer, TestLoopRefused mcp and import
This commit is contained in:
19
README.md
19
README.md
@@ -32,10 +32,13 @@ The frontend (a Nuxt layer or app) asks `GET /_seo/api/v1/resolve?url=...` for t
|
||||
- **AI bots.** Switches for OAI-SearchBot, Claude-SearchBot, PerplexityBot (search), ChatGPT-User, Claude-User (user fetch), GPTBot, ClaudeBot, Google-Extended, Applebot-Extended, CCBot and Meta-ExternalAgent (training), all allowed by default. A bot switched off gets `User-agent: <token>` / `Disallow: /`.
|
||||
- **Verification files** stored in the settings are served by `GET /_seo/api/v1/verification/{name}` for the frontend to map at the site root.
|
||||
- **Non-production lock.** Outside the `production` environment every resolve answer is `noindex, nofollow` with `staging: true`, and `/robots.txt` is `User-agent: *` / `Disallow: /` with `X-Robots-Tag: noindex, nofollow`, whatever is stored. It depends on the environment only; no setting turns it off.
|
||||
- **History** (`golem15_seo_revisions`): every SEO write (the SEO tab, rules, redirects, robots.txt, settings) is recorded with author, time, via (admin, MCP, or auto for address changes) and the changed fields before and after. Deleted records keep their values in history.
|
||||
- **History** (`golem15_seo_revisions`): every SEO write (the SEO tab, rules, redirects, robots.txt, settings, the 404 log) is recorded with author, time, via (admin, MCP, import, auto for address changes, undo) and the changed fields before and after. Deleted records keep their values in history.
|
||||
- **Undo** of one change or of a whole batch, from the History preview (record actions **Undo this change** and **Undo the whole batch**) and through MCP. An undo is refused when a later change touched the same values (`history.undo_stale`, naming who and when), when it would create a redirect loop (`history.undo_loop`) or when the record is gone (`history.undo_gone`); an undo is itself recorded.
|
||||
- **Batches** (D-12): a list of changes (SEO by URL, redirects, URL rules) applied all or nothing under one batch id. A dry run answers the diff and the conflicts (loops, chains, noindex on a published record's address, a canonical pointing to a redirected or noindex address) and writes nothing.
|
||||
- **CSV import and export** of redirects, URL rules and the SEO of records (by URL), in the **Import and export** screen (element `golem15-seo-transfer`) and through MCP. Files are UTF-8 with a BOM, comma or semicolon; exported cells that start a formula (`=`, `+`, `-`, `@`, tab, CR) are prefixed with an apostrophe, which the import drops again. An import is previewed row by row (new, changed, unchanged, refused, with notes) and applied as one batch with via `import`, or not at all while a row is refused. The screen's apply token is an HMAC-SHA256 under the app key over the file and a fingerprint of the data the preview read; it is refused with `transfer.stale` when the data changed. The screen previews files up to 60 KB and downloads exports up to 250 KB; MCP imports take up to 900 KB per call and exports come in pages of 1000 rows.
|
||||
- **Record events.** A deleted record loses its entries after commit; a created record gets its address snapshot; an updated record whose address changed gets its 301.
|
||||
- **Entry generation.** The `golem15.seo.generate_entries` job (conga) creates missing entries with the current address of every published record and refreshes the effective-value cache; editor values are never changed.
|
||||
- **Field types** `seotext`, `mlseotext`, `seorobots`, `mlseorobots`, `seoimage` and `robotstxt`, with the custom elements `golem15-seo-text`, `golem15-seo-robots`, `golem15-seo-serp` (the result preview widget), `golem15-seo-image`, `golem15-seo-robotstxt` (the robots.txt editor) and `golem15-seo-robotstester` (the tester widget).
|
||||
- **Field types** `seotext`, `mlseotext`, `seorobots`, `mlseorobots`, `seoimage` and `robotstxt`, with the custom elements `golem15-seo-text`, `golem15-seo-robots`, `golem15-seo-serp` (the result preview widget), `golem15-seo-image`, `golem15-seo-robotstxt` (the robots.txt editor), `golem15-seo-robotstester` (the tester widget) and `golem15-seo-transfer` (the import and export widget).
|
||||
- **One permission**, `golem15.seo.manage`, for the SEO tab, rules, redirects, the 404 log, robots.txt, settings and history, in the admin and through MCP.
|
||||
- **Management API and MCP.** Every SEO admin operation has a JSON route under `/_seo/api/v1` and an MCP tool bound to it (pier, prefix `seo_`). Tools address pages by URL: a change goes to the record behind the URL, else to an exact URL rule, and the answer says where. A read grant sees only the read tools. `AdminOperations` maps each admin operation to its tool, and a test fails when one has none.
|
||||
- **Issue report.** Missing and duplicate titles and descriptions, URL rules overriding a record, redirect chains, canonicals pointing to a noindex or redirected address, and noindex addresses robots.txt blocks, each with the tool that fixes it.
|
||||
@@ -148,13 +151,17 @@ The management routes live under `/_seo/api/v1` (bucket `seo-api`, 300 requests
|
||||
| `show_robots`, `update_robots`, `test_robots` | `GET/PUT /robots`, `GET /robots/test?url=&agent=` | robots.txt, the AI bot switches and the tester on the saved file. |
|
||||
| `show_settings`, `update_settings` | `GET/PUT /settings` | The SEO settings and, under `types`, the per-type settings. |
|
||||
| `list_history`, `show_history` | `GET /history`, `GET /history/{id}` | History, filtered by `via`, `kind` and `batch`. |
|
||||
| `undo_change`, `undo_batch` | `POST /history/{id}/undo`, `POST /history/batches/{batch}/undo` | Undo; refusals are `409` with `undo_stale`, `undo_loop`, `undo_gone` or `already_undone`. |
|
||||
| `apply_batch` | `POST /batch` | Changes (`set_seo`, `create_redirect`, `update_redirect`, `delete_redirect`, `create_rule`, `update_rule`, `delete_rule`) all or nothing; `dry_run` answers the diff and the conflicts. A refused change is `422` (`redirect_loop` with the change and the path, or `batch_refused`). |
|
||||
| `export_csv` | `GET /export?dataset=&delimiter=&page=` | One page of 1000 rows as CSV text. |
|
||||
| `import_csv` | `POST /import` | A CSV file of a dataset; `dry_run` previews it. A file over 900 KB is `413`; `missing_column`, `unreadable` and `import_refused` are `422`. |
|
||||
| `list_issues` | `GET /issues` | The CMS-data issue report with a count per kind. |
|
||||
|
||||
Lists take `page`, `per_page` (at most 200) and `search`, and answer `{"data": [...], "page", "per_page", "total"}`. GET tools are read-only; the deletes and `gone_not_found` are destructive.
|
||||
|
||||
### Admin
|
||||
|
||||
- Navigation item **SEO** (icon `search`), landing on **Redirects**, with the side menu **Redirects**, **404 log**, **URL rules**, **robots.txt** and **History**.
|
||||
- Navigation item **SEO** (icon `search`), landing on **Redirects**, with the side menu **Redirects**, **404 log**, **URL rules**, **robots.txt**, **History** and **Import and export**.
|
||||
- Settings tile **SEO settings** (`seo_settings`), a link to the settings singleton controller.
|
||||
- Every SEO screen and the SEO tab require `golem15.seo.manage`. Without it an administrator sees no SEO tab, and a posted SEO value changes nothing.
|
||||
|
||||
@@ -174,6 +181,10 @@ The element bundle (`assets/dist/seo.js` and `seo.css`) is built from `assets/sr
|
||||
| `classes.Owner`, `classes.URLSEOWrite`, `classes.URLSEOResult` | The owner of a URL (`classes.WrittenToRecord` or `classes.WrittenToRule`) and a write addressed by URL. |
|
||||
| `classes.Issue` | One finding of the issue report (`classes.IssueMissingTitle` and the other kinds). |
|
||||
| `classes.WithTx` | Puts a transaction on the context so every read of the service sees it. |
|
||||
| `classes.BatchChange`, `classes.BatchOptions`, `classes.BatchResult`, `classes.BatchRefusedError` | A batch: `Service.RunBatch` runs it as a dry run or all or nothing (`classes.WithBatch` marks the history rows). |
|
||||
| `classes.UndoRefusal`, `classes.Undoable` | `Service.Undo` and `Service.UndoBatch` and their refusals. |
|
||||
| `classes.ReadCSV`, `classes.WriteCSV`, `classes.Export`, `classes.ImportRequest`, `classes.ImportResult` | CSV files and `Service.Export` and `Service.Import`; `Service.TransferToken` and `Service.ReadTransferToken` sign and verify the screen's apply token. |
|
||||
| `controllers.TransferControllerID`, `controllers.TransferAction`, `controllers.HistoryActionUndo`, `controllers.HistoryActionUndoBatch` | The import and export screen, its widget action and the History record actions. |
|
||||
| `classes.Resolver` | `Resolve(ctx, url)` answers a `classes.Result` (actions `ok`, `not_found`, `redirect`, `gone`) with a source per value in `Result.Sources`; `ResolveRecord` resolves a record at its own address; `Strict` checks a path against the record's address. |
|
||||
| `classes.RedirectSet`, `classes.NewRedirectSet`, `classes.LoadRedirectSet` | A compiled redirect rule set: `Match` and `Follow` (`classes.Chain`, at most `classes.MaxRedirectHops`). |
|
||||
| `classes.RedirectCheck`, `classes.LoopError`, `classes.ProblemsError` | The save-time simulation's answer and the refusals of `SaveRedirect`. |
|
||||
@@ -222,4 +233,4 @@ go test ./... # with Docker: the Postgres smoke tests
|
||||
npm ci && npm run typecheck && bash scripts/check-dist.sh
|
||||
```
|
||||
|
||||
The smoke tests boot the plugin with fake content plugins (`acme.pages`, and `acme.blog` with a category segment for strict URLs) and exercise the SEO tab, the settings tabs, the resolver precedence, history, generation, the URL rule form, redirects (engine, loops and chains, address changes from the admin and from events), the 404 log, normalisation and strict URLs, robots.txt and verification files through the admin and public APIs, and the management API as an MCP client (`TestMCPSetURLSeoTracer`, `TestManagementAPI`, `TestIssues`). `TestEveryAdminOperationHasTool` and `TestMCPReadToolsAreGET` run without Docker.
|
||||
The smoke tests boot the plugin with fake content plugins (`acme.pages`, and `acme.blog` with a category segment for strict URLs) and exercise the SEO tab, the settings tabs, the resolver precedence, history, generation, the URL rule form, redirects (engine, loops and chains, address changes from the admin and from events), the 404 log, normalisation and strict URLs, robots.txt and verification files through the admin and public APIs, and the management API as an MCP client (`TestMCPSetURLSeoTracer`, `TestManagementAPI`, `TestIssues`), batches, undo and CSV (`TestBatch`, `TestUndo`, `TestCSVTransfer`, and the `mcp` and `import` cases of `TestLoopRefused`). `TestEveryAdminOperationHasTool` and `TestMCPReadToolsAreGET` run without Docker.
|
||||
|
||||
Reference in New Issue
Block a user