fix: a crawl cancel waits for its job, and a stale crawl is ended

Cancelling a running crawl records cancel_requested_at and leaves the
row running until its job stops and writes cancelled, so no second
crawl starts and the row cannot be deleted or pruned meanwhile. The job
writes heartbeat_at every minute; a running crawl without a heartbeat
for 10 minutes is ended by the next start or delete.
This commit is contained in:
Jakub Zych
2026-11-04 22:51:34 +01:00
parent d7597bb1a5
commit cd7182395f
12 changed files with 351 additions and 30 deletions

View File

@@ -36,7 +36,9 @@ const (
// (D-14, P33-37): its state, what started it and who, the site address,
// the page limit, the pages fetched and whether the limit stopped it, the
// error of a failed run, the state of the rendered comparison, the conga
// job and its times.
// job and its times. HeartbeatAt is written every minute by the running
// crawl's job; CancelRequestedAt is set when a running crawl is cancelled,
// which then stays running until its job stops and writes cancelled.
type Crawl struct {
ID uint `gorm:"column:id;primaryKey"`
Status string `gorm:"column:status"`
@@ -55,6 +57,9 @@ type Crawl struct {
StartedAt *time.Time `gorm:"column:started_at"`
FinishedAt *time.Time `gorm:"column:finished_at"`
CreatedAt *time.Time `gorm:"column:created_at"`
// HeartbeatAt and CancelRequestedAt (P33-60).
HeartbeatAt *time.Time `gorm:"column:heartbeat_at"`
CancelRequestedAt *time.Time `gorm:"column:cancel_requested_at"`
// The list's values, filled by the Crawls controller's
// ListExtendRecords and never saved.
IssuesCell string `gorm:"-:all;column:issues"`
@@ -77,6 +82,10 @@ func (Crawl) Rules() map[string]string { return map[string]string{} }
// Active reports whether the crawl is queued or running.
func (c Crawl) Active() bool { return c.Status == CrawlQueued || c.Status == CrawlRunning }
// Cancelling reports whether the crawl is running with a requested cancel:
// its job has not stopped yet.
func (c Crawl) Cancelling() bool { return c.Status == CrawlRunning && c.CancelRequestedAt != nil }
// The Crawls list's filter scope.
const FilterCrawlStatus = "FilterCrawlStatus"