fix: a crawl cancel waits for its job, and a stale crawl is ended

Cancelling a running crawl records cancel_requested_at and leaves the
row running until its job stops and writes cancelled, so no second
crawl starts and the row cannot be deleted or pruned meanwhile. The job
writes heartbeat_at every minute; a running crawl without a heartbeat
for 10 minutes is ended by the next start or delete.

README.md

@@ -247,7 +247,7 @@ A crawl is a conga job (`golem15.seo.crawl`, one attempt) started from **Start c
 
 **What it fetches.** `robots.txt`, the sitemap index at `/sitemap.xml` and its files (or a single urlset), then breadth-first from the home page and the sitemap addresses over same-host links, redirect targets, canonicals and hreflang alternates. Requests go through a `fetchguard` client limited to the site host and its `www` twin: HTTPS only, private and reserved addresses refused at dial time, redirects never followed (each `Location` is a new address checked again), the User-Agent `SummerCMS-SEO-Crawler/1 (+<site>)`. Links to other hosts, credentials in a URL and plain http are never fetched. A staging or local site served over http or on a private address needs `golem15.seo.crawl_trusted: true` in config (or `SUMMER_GOLEM15__SEO__CRAWL_TRUSTED`), which no setting or MCP call can set; trusted mode still fetches only the site host and its twin. robots.txt rules are read for the report, not obeyed.
 
-**Limits.** Two requests at a time with the settings' pause between requests (default 250 ms), a 10 s timeout and a 5 MiB body cap per request, at most 1000 links read per page, and the settings' page limit (default 5000; a crawl that stops there is marked "limit reached"). Cancelling (**Cancel crawl**, `seo_cancel_crawl`) stops between requests and keeps what was fetched; a failed crawl keeps the pages and issues found before the failure and records the error. A crawl that reaches the job's time limit (`crawl.JobTimeout`, 6 h) ends failed with "the crawl reached its time limit of 6h" and keeps what it fetched: the final writes run on a context the job's cancellation cannot reach, bounded by `crawl.FinishTimeout` (10 min). The crawler writes only its own tables (`golem15_seo_crawls`, `golem15_seo_crawl_pages`, `golem15_seo_crawl_links`, `golem15_seo_issues`); crawls beyond the settings' "Crawls kept" are deleted with their pages, links and issues.
+**Limits.** Two requests at a time with the settings' pause between requests (default 250 ms), a 10 s timeout and a 5 MiB body cap per request, at most 1000 links read per page, and the settings' page limit (default 5000; a crawl that stops there is marked "limit reached"). Cancelling (**Cancel crawl**, `seo_cancel_crawl`) ends a queued crawl at once; a running crawl records the request (`cancel_requested` in the API and MCP answers, **Cancelling** in the list) and keeps running until its job stops between requests, then ends cancelled with what was fetched. Until then no other crawl starts (409 `crawls.already_running`) and the crawl cannot be deleted; a failed crawl keeps the pages and issues found before the failure and records the error. A crawl that reaches the job's time limit (`crawl.JobTimeout`, 6 h) ends failed with "the crawl reached its time limit of 6h" and keeps what it fetched: the final writes run on a context the job's cancellation cannot reach, bounded by `crawl.FinishTimeout` (10 min). A running crawl's job writes `heartbeat_at` every minute (`crawl.HeartbeatEvery`); a running crawl without a heartbeat for 10 minutes (`crawl.StaleAfter`, its job is gone, for example after a worker restart) is ended by the next start or delete (`crawl.RecoverStale`): failed with "the crawl job stopped without finishing (no heartbeat since …)", or cancelled when its cancel was requested. The crawler writes only its own tables (`golem15_seo_crawls`, `golem15_seo_crawl_pages`, `golem15_seo_crawl_links`, `golem15_seo_issues`); crawls beyond the settings' "Crawls kept" are deleted with their pages, links and issues.
 
 **Checks** (kind, severity): `page_status` (error), `sitemap_error` (error), `sitemap_noindex` (error), `sitemap_blocked` (error), `sitemap_missing` (notice), `broken_link` (error, with the linking pages), `redirected_link` (notice), `canonical_missing` (warning), `canonical_error` (error), `missing_title` and `duplicate_title` (warning), `missing_description` and `duplicate_description` (notice), `h1_missing` and `h1_multiple` (error), `heading_skipped` and `heading_empty` (warning), `h1_title_mismatch` (notice), `jsonld_invalid` (error), `jsonld_multiple` (warning), `jsonld_required` (error) and `jsonld_recommended` (notice) from the same rules as the structured data preview (`schema.Validate`; FAQPage and other types without Google rules find nothing), `hreflang_reciprocal` (warning), `hreflang_error` (error), `generic_anchor` (warning, the settings' generic link texts, whole text, any case), `image_alt` (warning: an `<img>` without `alt`; `alt=""` is decorative), `image_size` and `featured_lazy` (notice), `content_js` (warning) and `orphan_page` (warning: an indexable sitemap page no crawled page links to, with up to 10 pages whose text mentions its title or focus keyphrase). The CMS-data checks run at the end of every crawl (and when the list is opened before any crawl) and are stored with the crawl's issues as source `cms`; a CMS issue the crawl found too is stored once.
 

classes/crawl/actions.go

@@ -71,11 +71,16 @@ func Unignore(tx *gorm.DB, issues []models.Issue) (int, error) {
 }
 
 // DeleteCrawls deletes crawls with their pages, links and issues; a
-// queued or running one refuses the whole call with ErrDeleteRunning.
+// queued or running one (also one whose cancel is requested) refuses the
+// whole call with ErrDeleteRunning. A running crawl without a heartbeat is
+// ended first (RecoverStale), so it can be deleted.
 func DeleteCrawls(tx *gorm.DB, ids []uint) (int, error) {
 	if len(ids) == 0 {
 		return 0, nil
 	}
+	if _, err := RecoverStale(tx, time.Now()); err != nil {
+		return 0, err
+	}
 	var active int64
 	if err := tx.Model(&models.Crawl{}).Where("id IN ? AND status IN ?", ids, []string{models.CrawlQueued, models.CrawlRunning}).Count(&active).Error; err != nil {
 		return 0, err

classes/crawl/job.go

@@ -45,12 +45,19 @@ func (CrawlArgs) Kind() string { return CrawlKind }
 
 // The crawl job's time bounds (P33-60). JobTimeout is the job's deadline;
 // FinishTimeout bounds storing what a crawl found, on a context the job's
-// cancellation cannot reach.
+// cancellation cannot reach; the job writes heartbeat_at every
+// HeartbeatEvery, and a running crawl without a heartbeat for StaleAfter
+// is ended by the next Start or delete (RecoverStale).
 const (
-	JobTimeout    = 6 * time.Hour
-	FinishTimeout = 10 * time.Minute
+	JobTimeout     = 6 * time.Hour
+	FinishTimeout  = 10 * time.Minute
+	HeartbeatEvery = time.Minute
+	StaleAfter     = 10 * time.Minute
 )
 
+// heartbeatWrite bounds one heartbeat update.
+const heartbeatWrite = 10 * time.Second
+
 // The crawl limits used until the Site audit settings say otherwise.
 const (
 	DefaultMaxPages = 5000
@@ -78,9 +85,11 @@ func (e *AlreadyRunningError) Since() time.Time {
 
 // Start queues a crawl (D-14, P33-39) inside tx: the settings row is
 // locked so two starts cannot both pass the check that no crawl is queued
-// or running, then the crawl row is written and the conga job dispatched
-// in the same transaction. A missing site address refuses with
-// ErrNoSiteURL, a running crawl with *AlreadyRunningError.
+// or running, a running crawl whose job stopped writing its heartbeat is
+// ended (RecoverStale), then the crawl row is written and the conga job
+// dispatched in the same transaction. A missing site address refuses with
+// ErrNoSiteURL, a queued or running crawl (also one whose cancel is
+// requested but whose job has not stopped) with *AlreadyRunningError.
 func Start(ctx context.Context, svc *classes.Service, tx *gorm.DB, trigger, name string) (models.Crawl, error) {
 	site := svc.SiteURL()
 	if site == "" {
@@ -92,6 +101,11 @@ func Start(ctx context.Context, svc *classes.Service, tx *gorm.DB, trigger, name
 	if err := tx.Exec(`SELECT id FROM golem15_seo_settings WHERE id = 1 FOR UPDATE`).Error; err != nil {
 		return models.Crawl{}, err
 	}
+	stale, err := RecoverStale(tx, time.Now().UTC())
+	if err != nil {
+		return models.Crawl{}, err
+	}
+	endStaleJobs(ctx, svc, tx, stale)
 	active, err := ActiveCrawl(tx)
 	if err != nil {
 		return models.Crawl{}, err
@@ -182,6 +196,7 @@ func LoadCrawl(db *gorm.DB, id uint) (models.Crawl, error) {
 // context) or at the job timeout; what was fetched is then stored on a
 // context detached from it with its own FinishTimeout bound, so a
 // cancelled or timed-out crawl keeps its results and never stays running.
+// While it runs, the job writes heartbeat_at every HeartbeatEvery.
 func Run(ctx context.Context, svc *classes.Service, id uint) error {
 	gdb, err := svc.GormDB()
 	if err != nil {
@@ -197,19 +212,49 @@ func Run(ctx context.Context, svc *classes.Service, id uint) error {
 	}
 	now := time.Now().UTC()
 	moved := db.Model(&models.Crawl{}).Where("id = ? AND status = ?", id, models.CrawlQueued).
-		Updates(map[string]any{"status": models.CrawlRunning, "started_at": now})
+		Updates(map[string]any{"status": models.CrawlRunning, "started_at": now, "heartbeat_at": now})
 	if moved.Error != nil {
 		return moved.Error
 	}
 	if moved.RowsAffected == 0 {
 		return nil
 	}
+	stop := heartbeat(ctx, gdb, id)
+	defer stop()
 	res, runErr := run(ctx, svc, db, row)
 	fctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), FinishTimeout)
 	defer cancel()
 	return finish(fctx, svc, gdb.WithContext(fctx), row, res, runErr, ctx.Err())
 }
 
+// heartbeat writes the crawl's heartbeat_at every HeartbeatEvery until the
+// returned stop is called; each write runs on a context detached from the
+// job's, bounded by heartbeatWrite.
+func heartbeat(ctx context.Context, gdb *gorm.DB, id uint) (stop func()) {
+	done := make(chan struct{})
+	finished := make(chan struct{})
+	go func() {
+		defer close(finished)
+		t := time.NewTicker(HeartbeatEvery)
+		defer t.Stop()
+		for {
+			select {
+			case <-done:
+				return
+			case <-t.C:
+				hctx, cancel := context.WithTimeout(context.WithoutCancel(ctx), heartbeatWrite)
+				_ = gdb.WithContext(hctx).Model(&models.Crawl{}).Where("id = ? AND status = ?", id, models.CrawlRunning).
+					Update("heartbeat_at", time.Now().UTC()).Error
+				cancel()
+			}
+		}
+	}()
+	return func() {
+		close(done)
+		<-finished
+	}
+}
+
 // run fetches and checks the site of one crawl.
 func run(ctx context.Context, svc *classes.Service, db *gorm.DB, row models.Crawl) (*Result, error) {
 	f, err := NewFetcher(row.Site, svc.Config())
@@ -267,14 +312,17 @@ func run(ctx context.Context, svc *classes.Service, db *gorm.DB, row models.Craw
 	return Crawl(ctx, f, opt)
 }
 
-// cancelRequested reports whether the crawl was cancelled: its status is
-// cancelled.
+// cancelRequested reports whether a cancel of the crawl was requested: its
+// cancel_requested_at is set, or its status is cancelled.
 func cancelRequested(db *gorm.DB, id uint) bool {
-	var status string
-	if err := db.Raw(`SELECT status FROM golem15_seo_crawls WHERE id = ?`, id).Scan(&status).Error; err != nil {
+	var rows []struct {
+		Status    string
+		Requested bool
+	}
+	if err := db.Raw(`SELECT status, cancel_requested_at IS NOT NULL AS requested FROM golem15_seo_crawls WHERE id = ?`, id).Scan(&rows).Error; err != nil || len(rows) == 0 {
 		return false
 	}
-	return status == models.CrawlCancelled
+	return rows[0].Requested || rows[0].Status == models.CrawlCancelled
 }
 
 // timeLimitError is the error of a crawl that reached JobTimeout.
@@ -370,6 +418,31 @@ func finish(ctx context.Context, svc *classes.Service, db *gorm.DB, row models.C
 	return nil
 }
 
+// endStaleJobs fails or stops the conga jobs of crawls RecoverStale ended,
+// once tx commits; best effort.
+func endStaleJobs(ctx context.Context, svc *classes.Service, tx *gorm.DB, rows []models.Crawl) {
+	if len(rows) == 0 {
+		return
+	}
+	lagoon.AfterCommit(ctx, tx, func(ctx context.Context, _ *gorm.DB) {
+		m, err := conga.From(svc.App())
+		if err != nil {
+			return
+		}
+		for _, r := range rows {
+			if r.JobID == nil {
+				continue
+			}
+			meta := map[string]any{"crawl_id": r.ID, "status": r.Status}
+			if r.Status == models.CrawlCancelled {
+				_ = m.StopJob(ctx, *r.JobID, meta)
+			} else {
+				_ = m.FailJob(ctx, *r.JobID, meta)
+			}
+		}
+	})
+}
+
 // Lookup returns the CMS record behind a crawled address (cached per
 // address): the content type whose Match answers the path exactly, its
 // descriptor facts in the matched locale, its primary parent's address and
@@ -455,20 +528,31 @@ func Keyphrases(db *gorm.DB) (map[string]string, error) {
 // running.
 var errCancelledCrawl = errors.New("golem15.seo: the crawl is not running")
 
-// Cancel stops a queued or running crawl inside tx: the row is marked
-// cancelled by name (the crawl keeps what it fetched) and its conga job
-// cancelled.
+// Cancel stops a queued or running crawl inside tx. A queued crawl ends
+// cancelled at once (by name, finished_at set). A running crawl gets
+// cancel_requested_at and cancelled_by and stays running: its job stops
+// between requests and writes cancelled with the pages and issues fetched
+// so far, and until then no other crawl starts. The conga job is cancelled
+// in both cases.
 func Cancel(ctx context.Context, svc *classes.Service, tx *gorm.DB, row models.Crawl, name string) error {
 	if !row.Active() {
 		return errCancelledCrawl
 	}
 	now := time.Now().UTC()
-	updates := map[string]any{"status": models.CrawlCancelled, "cancelled_by": name}
-	if row.Status == models.CrawlQueued {
-		updates["finished_at"] = now
+	queued := tx.Model(&models.Crawl{}).Where("id = ? AND status = ?", row.ID, models.CrawlQueued).
+		Updates(map[string]any{"status": models.CrawlCancelled, "cancelled_by": name, "finished_at": now})
+	if queued.Error != nil {
+		return queued.Error
 	}
-	if err := tx.Model(&models.Crawl{}).Where("id = ?", row.ID).Updates(updates).Error; err != nil {
-		return err
+	if queued.RowsAffected == 0 {
+		running := tx.Model(&models.Crawl{}).Where("id = ? AND status = ?", row.ID, models.CrawlRunning).
+			Updates(map[string]any{"cancel_requested_at": now, "cancelled_by": name})
+		if running.Error != nil {
+			return running.Error
+		}
+		if running.RowsAffected == 0 {
+			return errCancelledCrawl
+		}
 	}
 	if row.JobID != nil {
 		if m, err := conga.From(svc.App()); err == nil {

classes/crawl/store.go

@@ -189,6 +189,55 @@ func ActiveCrawl(db *gorm.DB) (*models.Crawl, error) {
 	return &rows[0], nil
 }
 
+// RecoverStale ends the running crawls whose job stopped writing its
+// heartbeat (P33-60): a running row whose COALESCE(heartbeat_at,
+// started_at, created_at) is older than StaleAfter before now becomes
+// cancelled when its cancel was requested, else failed with the error
+// "the crawl job stopped without finishing (no heartbeat since <time>)",
+// with finished_at now. It answers the rows it ended, with their new
+// status, so the caller can end their conga jobs.
+func RecoverStale(tx *gorm.DB, now time.Time) ([]models.Crawl, error) {
+	now = now.UTC()
+	var rows []models.Crawl
+	if err := tx.Where("status = ? AND COALESCE(heartbeat_at, started_at, created_at) < ?", models.CrawlRunning, now.Add(-StaleAfter)).
+		Order("id ASC").Find(&rows).Error; err != nil {
+		return nil, err
+	}
+	out := rows[:0]
+	for _, r := range rows {
+		last := r.HeartbeatAt
+		if last == nil {
+			last = r.StartedAt
+		}
+		if last == nil {
+			last = r.CreatedAt
+		}
+		updates := map[string]any{"finished_at": now}
+		if r.CancelRequestedAt != nil {
+			updates["status"], updates["error"] = models.CrawlCancelled, ""
+		} else {
+			since := "an unknown time"
+			if last != nil {
+				since = last.UTC().Format(time.RFC3339)
+			}
+			updates["status"], updates["error"] = models.CrawlFailed, "the crawl job stopped without finishing (no heartbeat since "+since+")"
+		}
+		res := tx.Model(&models.Crawl{}).Where("id = ? AND status = ?", r.ID, models.CrawlRunning).Updates(updates)
+		if res.Error != nil {
+			return nil, res.Error
+		}
+		if res.RowsAffected == 0 {
+			continue
+		}
+		r.Status = updates["status"].(string)
+		r.Error = updates["error"].(string)
+		r.FinishedAt = &now
+		pluginLog.Warn("golem15.seo: ended a stale crawl", "crawl", r.ID, "status", r.Status)
+		out = append(out, r)
+	}
+	return out, nil
+}
+
 // CurrentScope narrows an issues query to the audit's current issues: the
 // newest finished, failed or cancelled crawl's, or before any crawl the
 // stored CMS-data issues.

controllers/api/crawls.go

@@ -186,7 +186,7 @@ func crawlMap(ctx context.Context, svc *classes.Service, c models.Crawl) map[str
 		"id": v.ID, "status": v.Status, "trigger": v.Trigger, "trigger_name": v.TriggerName, "site": v.Site,
 		"max_pages": v.MaxPages, "pages": v.Pages, "limit_hit": v.LimitHit, "error": v.Error,
 		"started_at": v.StartedAt, "finished_at": v.FinishedAt, "created_at": v.CreatedAt,
-		"cancelled_by": c.CancelledBy, "render": renderView(ctx, svc, c),
+		"cancelled_by": c.CancelledBy, "cancel_requested": c.CancelRequestedAt != nil, "render": renderView(ctx, svc, c),
 	}
 }
 
@@ -230,9 +230,10 @@ func ShowCrawl(service func() *classes.Service) http.HandlerFunc {
 	})
 }
 
-// CancelCrawl serves POST /_seo/api/v1/crawls/{id}/cancel: a queued or
-// running crawl stops and keeps what it fetched; another answers 409
-// crawls.not_running.
+// CancelCrawl serves POST /_seo/api/v1/crawls/{id}/cancel: a queued crawl
+// ends cancelled at once; a running one answers with cancel_requested true
+// and stays running until its job stops, then ends cancelled and keeps
+// what it fetched; another answers 409 crawls.not_running.
 func CancelCrawl(service func() *classes.Service) http.HandlerFunc {
 	return manage(service, func(ctx context.Context, svc *classes.Service, r *http.Request) (int, any, error) {
 		id, err := pathID(r, "id")

controllers/crawls.go

@@ -81,7 +81,7 @@ func (c crawlsAdminController) AdminRecordActions() []pact.AdminRecordAction {
 		Permissions: []string{PermissionManage},
 		Applies: func(_ context.Context, record any) (bool, error) {
 			row, ok := record.(*models.Crawl)
-			return ok && row != nil && row.Active(), nil
+			return ok && row != nil && row.Active() && !row.Cancelling(), nil
 		},
 		Run: func(ctx context.Context, in pact.AdminRecordActionInput) (pact.AdminRecordActionResult, error) {
 			row, ok := in.Record.(*models.Crawl)
@@ -194,7 +194,9 @@ func (c crawlsAdminController) ListExtendRecords(ctx context.Context, db *gorm.D
 		row.IssuesCell = strconv.Itoa(counts[row.ID])
 		row.DurationCell = Duration(*row)
 		row.TriggerCell = TriggerLabel(ctx, svc, *row)
-		if key, ok := statusKeys[row.Status]; ok {
+		if row.Cancelling() {
+			row.Status = svc.PhraseCtx(ctx, "golem15.seo::lang.crawls.st_cancelling", nil)
+		} else if key, ok := statusKeys[row.Status]; ok {
 			row.Status = svc.PhraseCtx(ctx, key, nil)
 		}
 	}

crawl_lifecycle_test.go

@@ -2,7 +2,9 @@ package seo_test
 
 import (
 	"context"
+	"encoding/json"
 	"errors"
+	"fmt"
 	"net/http"
 	"strings"
 	"sync"
@@ -133,6 +135,90 @@ func (e *testEnv) crawlIssueKinds(t *testing.T, id uint) map[string][]string {
 	return out
 }
 
+// TestCrawlCancelRunning pins L3 CR-01 and WR-01 (D-14): a running crawl
+// that is cancelled stays running with cancel_requested until its job
+// stops; meanwhile no other crawl starts, it cannot be deleted and Prune
+// keeps it. When the job's context is cancelled (as River does), the job
+// stores the pages fetched so far with their issues and ends cancelled
+// with finished_at; the fetch cut short is not a network error.
+func TestCrawlCancelRunning(t *testing.T) {
+	env := bootCrawlEnv(t)
+	token := env.admin(t, true)
+	if err := env.db.Exec(`UPDATE golem15_seo_settings SET crawl_delay = 100 WHERE id = 1`).Error; err != nil {
+		t.Fatal(err)
+	}
+	site := newBlockingSite("/b")
+	defer site.unblock()
+	lifecycleSite(site)
+	id := env.startCrawl(t, token)
+	ctx, cancelJob := context.WithCancel(fetchguard.WithTransport(t.Context(), site))
+	defer cancelJob()
+	done := env.runCrawlAsync(t, ctx, id)
+	waitReached(t, site)
+
+	rec := env.expect(t, http.StatusOK, http.MethodPost, fmt.Sprintf("/_seo/api/v1/crawls/%d/cancel", id), token, map[string]any{})
+	var out struct {
+		Data map[string]any `json:"data"`
+	}
+	if err := json.Unmarshal(rec.Body.Bytes(), &out); err != nil {
+		t.Fatal(err)
+	}
+	if out.Data["status"] != "running" || out.Data["cancel_requested"] != true {
+		t.Fatalf("cancel answer = %v, want running with cancel_requested", out.Data)
+	}
+	row := env.crawlRow(t, id)
+	if row.Status != models.CrawlRunning || row.CancelRequestedAt == nil || row.CancelledBy == "" || row.FinishedAt != nil {
+		t.Fatalf("row after the cancel = %+v", row)
+	}
+	if list := env.adminList(t, crawlsAPI, token); len(list.Data) == 0 || list.Data[0]["status"] != "Cancelling" {
+		t.Fatalf("the Crawls list shows %v, want Cancelling", list.Data)
+	}
+	if rec := env.call(t, http.MethodPost, "/_seo/api/v1/crawls", token, map[string]any{}); rec.Code != http.StatusConflict || !strings.Contains(rec.Body.String(), "crawls.already_running") {
+		t.Fatalf("a start while the cancelled crawl stops = %d %s", rec.Code, rec.Body.String())
+	}
+	if rec := env.call(t, http.MethodDelete, fmt.Sprintf("/_seo/api/v1/crawls?ids=%d", id), token, nil); rec.Code != http.StatusForbidden {
+		t.Fatalf("delete of the stopping crawl = %d %s", rec.Code, rec.Body.String())
+	}
+	if err := crawl.Prune(env.db, 1); err != nil {
+		t.Fatal(err)
+	}
+	if env.crawlRow(t, id).ID != id {
+		t.Fatal("Prune deleted the stopping crawl")
+	}
+
+	// River cancels the running job's context; the transport lets go.
+	cancelJob()
+	site.unblock()
+	if err := waitJob(t, done); err != nil {
+		t.Fatalf("the cancelled job answered %v", err)
+	}
+	row = env.crawlRow(t, id)
+	if row.Status != models.CrawlCancelled || row.FinishedAt == nil {
+		t.Fatalf("row after the job = %+v, want cancelled with finished_at", row)
+	}
+	pages := env.storedPages(t, id)
+	if pages["/"] != 200 || pages["/a"] != 200 {
+		t.Fatalf("stored pages = %v, want / and /a with 200", pages)
+	}
+	kinds := env.crawlIssueKinds(t, id)
+	if !strings.Contains(strings.Join(kinds["/a"], ","), "missing_title") {
+		t.Fatalf("issues of /a = %v, want missing_title", kinds["/a"])
+	}
+	for url, ks := range kinds {
+		for _, k := range ks {
+			if k == "page_status" || k == "sitemap_error" {
+				if url == "/b" {
+					t.Fatalf("the fetch cut short by the cancel is an issue: %s on %s", k, url)
+				}
+			}
+		}
+	}
+	latest, err := crawl.LatestCrawl(env.db)
+	if err != nil || latest == nil || latest.ID != id {
+		t.Fatalf("LatestCrawl = %+v %v, want the cancelled crawl", latest, err)
+	}
+}
+
 // TestCrawlTimeout pins L3 CR-01: a crawl whose job context reaches its
 // deadline stores what it fetched and ends failed with the time-limit
 // error and finished_at; a later Start succeeds.
@@ -163,3 +249,55 @@ func TestCrawlTimeout(t *testing.T) {
 	}
 	env.startCrawl(t, token)
 }
+
+// TestCrawlStaleRecovery pins P33-60: a running crawl whose heartbeat is
+// older than StaleAfter is ended by the next Start (failed with the
+// no-heartbeat error, or cancelled when its cancel was requested), which
+// then queues a new crawl; a fresh heartbeat still blocks Start.
+func TestCrawlStaleRecovery(t *testing.T) {
+	env := bootCrawlEnv(t)
+	token := env.admin(t, true)
+	old := time.Now().UTC().Add(-11 * time.Minute)
+	insert := func(heartbeat time.Time, cancelRequested bool) uint {
+		t.Helper()
+		row := models.Crawl{Status: models.CrawlRunning, Trigger: models.TriggerAdmin, Site: siteURL, RenderStatus: models.RenderSkipped, StartedAt: &old, CreatedAt: &old, HeartbeatAt: &heartbeat}
+		if cancelRequested {
+			row.CancelRequestedAt = &heartbeat
+			row.CancelledBy = "Admin"
+		}
+		if err := env.db.Create(&row).Error; err != nil {
+			t.Fatal(err)
+		}
+		return row.ID
+	}
+
+	stale := insert(old, false)
+	next := env.startCrawl(t, token)
+	row := env.crawlRow(t, stale)
+	if row.Status != models.CrawlFailed || !strings.Contains(row.Error, "no heartbeat since") || row.FinishedAt == nil {
+		t.Fatalf("stale crawl after Start = %+v", row)
+	}
+	if env.crawlRow(t, next).Status != models.CrawlQueued {
+		t.Fatal("Start queued no new crawl")
+	}
+	if err := env.db.Exec(`UPDATE golem15_seo_crawls SET status = 'finished', finished_at = NOW() WHERE id = ?`, next).Error; err != nil {
+		t.Fatal(err)
+	}
+
+	cancelled := insert(old, true)
+	env.startCrawl(t, token)
+	if row := env.crawlRow(t, cancelled); row.Status != models.CrawlCancelled || row.FinishedAt == nil || row.Error != "" {
+		t.Fatalf("stale crawl with a requested cancel = %+v", row)
+	}
+	if err := env.db.Exec(`UPDATE golem15_seo_crawls SET status = 'finished', finished_at = NOW() WHERE status = 'queued'`).Error; err != nil {
+		t.Fatal(err)
+	}
+
+	fresh := insert(time.Now().UTC(), false)
+	if rec := env.call(t, http.MethodPost, "/_seo/api/v1/crawls", token, map[string]any{}); rec.Code != http.StatusConflict {
+		t.Fatalf("Start beside a crawl with a fresh heartbeat = %d %s", rec.Code, rec.Body.String())
+	}
+	if env.crawlRow(t, fresh).Status != models.CrawlRunning {
+		t.Fatal("a crawl with a fresh heartbeat was ended")
+	}
+}

lang/en/lang.yaml

@@ -591,6 +591,7 @@ crawls:
     st_finished: "Finished"
     st_failed: "Failed"
     st_cancelled: "Cancelled"
+    st_cancelling: "Cancelling"
     pages_limit: ":count (limit reached)"
     by_admin: "Manual: :name"
     by_schedule: "Schedule"

lang/pl/lang.yaml

@@ -609,6 +609,7 @@ crawls:
     st_finished: "Zakończone"
     st_failed: "Nieudane"
     st_cancelled: "Anulowane"
+    st_cancelling: "Anulowanie"
     pages_limit: ":count (osiągnięto limit)"
     by_admin: "Ręcznie: :name"
     by_schedule: "Harmonogram"

mcp.go

@@ -500,7 +500,7 @@ func (p *Plugin) MCP() pact.MCP {
 		{
 			Name:        "cancel_crawl",
 			Title:       "Cancel a crawl",
-			Description: "Cancels a queued or running crawl by id; the pages fetched so far and their issues are kept.",
+			Description: "Cancels a queued or running crawl by id. A queued crawl ends cancelled at once. A running crawl answers cancel_requested true and keeps running until its job stops (after the requests in flight), then ends cancelled with the pages fetched so far and their issues; no other crawl starts before that.",
 			Guard:       mcpGuard,
 			Method:      http.MethodPost,
 			Path:        "/_seo/api/v1/crawls/{id}/cancel",

models/crawl.go

@@ -36,7 +36,9 @@ const (
 // (D-14, P33-37): its state, what started it and who, the site address,
 // the page limit, the pages fetched and whether the limit stopped it, the
 // error of a failed run, the state of the rendered comparison, the conga
-// job and its times.
+// job and its times. HeartbeatAt is written every minute by the running
+// crawl's job; CancelRequestedAt is set when a running crawl is cancelled,
+// which then stays running until its job stops and writes cancelled.
 type Crawl struct {
 	ID           uint       `gorm:"column:id;primaryKey"`
 	Status       string     `gorm:"column:status"`
@@ -55,6 +57,9 @@ type Crawl struct {
 	StartedAt    *time.Time `gorm:"column:started_at"`
 	FinishedAt   *time.Time `gorm:"column:finished_at"`
 	CreatedAt    *time.Time `gorm:"column:created_at"`
+	// HeartbeatAt and CancelRequestedAt (P33-60).
+	HeartbeatAt       *time.Time `gorm:"column:heartbeat_at"`
+	CancelRequestedAt *time.Time `gorm:"column:cancel_requested_at"`
 	// The list's values, filled by the Crawls controller's
 	// ListExtendRecords and never saved.
 	IssuesCell   string `gorm:"-:all;column:issues"`
@@ -77,6 +82,10 @@ func (Crawl) Rules() map[string]string { return map[string]string{} }
 // Active reports whether the crawl is queued or running.
 func (c Crawl) Active() bool { return c.Status == CrawlQueued || c.Status == CrawlRunning }
 
+// Cancelling reports whether the crawl is running with a requested cancel:
+// its job has not stopped yet.
+func (c Crawl) Cancelling() bool { return c.Status == CrawlRunning && c.CancelRequestedAt != nil }
+
 // The Crawls list's filter scope.
 const FilterCrawlStatus = "FilterCrawlStatus"
 

updates/202611040002_add_golem15_seo_crawls_heartbeat.go

@@ -0,0 +1,31 @@
+package updates
+
+import (
+	"github.com/go-gormigrate/gormigrate/v2"
+	"gorm.io/gorm"
+)
+
+// addCrawlsHeartbeat adds heartbeat_at and cancel_requested_at to
+// golem15_seo_crawls (P33-60): a running crawl's job writes heartbeat_at
+// every minute, so a crawl whose job is gone is recognised and ended; a
+// cancel of a running crawl records cancel_requested_at and leaves the
+// terminal status to the job.
+var addCrawlsHeartbeat = []*gormigrate.Migration{
+	{
+		ID: "202611040002_add_golem15_seo_crawls_heartbeat",
+		Migrate: func(tx *gorm.DB) error {
+			return tx.Exec(`ALTER TABLE golem15_seo_crawls
+	ADD COLUMN heartbeat_at TIMESTAMP(0) NULL,
+	ADD COLUMN cancel_requested_at TIMESTAMP(0) NULL`).Error
+		},
+		Rollback: func(tx *gorm.DB) error {
+			return tx.Exec(`ALTER TABLE golem15_seo_crawls
+	DROP COLUMN IF EXISTS cancel_requested_at,
+	DROP COLUMN IF EXISTS heartbeat_at`).Error
+		},
+	},
+}
+
+func init() {
+	Register(addCrawlsHeartbeat...)
+}