enhance(#4036): persist CI shard/job timeout-vs-cap trending, warn at 90% (#4043)

* feat(#4036): persist CI shard/job timeout-vs-cap trending, warn at 90% Adds two new mechanisms plus an audit-coverage extension: - scripts/lib/ci-job-timing.cjs: shared elapsed-vs-cap arithmetic - scripts/ci-check-job-near-cap.cjs: in-job advisory near-cap check, wired into test/test-full/mutate/smoke as each job's last step - scripts/ci-timeout-report.cjs + .github/workflows/ci-timeout-report.yml: scheduled REST-API poll that appends new records to tests/ci-timeout-budget-history.jsonl and opens a small data-only PR - tests/ci-test-job-timeout-budget.test.cjs: extended to cover mutate (mutation.yml) and smoke (install-smoke.yml), which previously had no headroom-factor gate coverage at all Does not change any timeout-minutes value, shard composition, or shard-1 contents — those stay maintainer policy calls per the issue's own scope. * fix(#4036): address two-orthogonal-review findings - Parity tests guarding the two hand-duplicated literals this design cannot single-source through GH Actions YAML: CI_JOB_TIMEOUT_MINUTES vs each job's own timeout-minutes, and ci-timeout-report.cjs's JOB_RULES name-prefixes vs each job's actual name: template. - Thread run.event through as runEvent on every persisted record, so PR-context and push-context install-smoke timings (genuinely different matrix shape) are distinguishable in the history rather than silently conflated under one job name. - Replace the Windows near-cap start-time step's ambiguous PowerShell +/>> precedence with GitHub's documented string-interpolation form. - Move github.run_id out of direct ${{ }} shell interpolation into an env: var in the new scheduled workflow, per this repo's own expression-injection-safe convention. * test(#4036): regenerate golden install-tree fixtures for scripts/lib/ci-job-timing.cjs npm run gen:install-tree — scripts/ ships wholesale into the installed package (per ADR/known-defect precedent from #4012's own PR history: a new scripts/lib/*.cjs file needs its golden entry regenerated or every runtime's install-tree test fails). Confirmed via gsd-test: this was the sole cause of the first real verification run's 25 failures (all in tests/golden-install-tree.test.cjs, one per runtime). Top-level scripts/*.cjs files (ci-check-job-near-cap.cjs, ci-timeout-report.cjs) are not individually tracked in these fixtures — consistent with every other existing top-level scripts/*.cjs file, so no entry was expected or added for those two. * fix(#4036): register new lib file with installer, fix H1 shell policy - bin/install.js: add ci-job-timing.cjs to GSD_SCRIPTS_LIB_FILES (a hand-maintained registry, not generated — tests/install.test.cjs asserts every scripts/lib/ file is enumerated here) - test.yml: replace the two OS-conditional "Record job start time" step pairs (test + test-full jobs) with a single unconditional `node -e` step. The prior pair's Windows variant declared an explicit shell: pwsh, which scripts/workflow-policy.cjs's H1 checker statically flags against every OS a job's matrix can realize, independent of the step's own if: gate. A single Node one-liner needs no shell override at all — it's syntactically valid and behaves identically under bash, zsh, and pwsh — which is both H1 compliant and removes the last OS-specific shell syntax from this change entirely. Both defects were found by a real gsd-test run, not local gates — lint:ci and build:lib were clean throughout because neither the scripts/lib/ install-manifest parity check nor the H1 shell-policy baseline runs as part of lint:ci; both are gsd-test-only suites. * docs(#4036): how-to for reading CI timeout budget signals The phase-gate docs check correctly flagged the enablement sequence as 3 real steps (read the near-cap warning, find the accumulated trend file, pick the right maintainer lever) — a reference table can't carry a sequence. Adds docs/how-to/read-ci-timeout-signals.md, indexed from docs/README.md. * chore(#4036): backfill changeset PR number (4043) --------- Co-authored-by: sim <sim@local>
2026-08-29 16:13:15 -04:00
parent b431ae9f0d
commit 370cfc6680
34 changed files with 1158 additions and 5 deletions
--- a/tests/ci-job-timing.test.cjs
+++ b/tests/ci-job-timing.test.cjs
@@ -0,0 +1,198 @@
+'use strict';
+
+/**
+ * scripts/lib/ci-job-timing.cjs — pure budget-percentage arithmetic (#4036).
+ *
+ * This module is shared by the in-job near-cap check
+ * (scripts/ci-check-job-near-cap.cjs) and the scheduled trending report
+ * (scripts/ci-timeout-report.cjs). Both surfaces need to agree on exactly
+ * what "elapsed" and "near-cap" mean, so the arithmetic is factored out here
+ * and tested once, at every boundary the two callers depend on: zero
+ * elapsed, exactly-at-cap, just-under-threshold, exactly-at-threshold,
+ * just-over-threshold, and past-cap. It also covers the invalid-input
+ * failure modes (bad timeoutMinutes, malformed timestamps, completedAt
+ * before startedAt) that both callers need to fail loudly on rather than
+ * silently misreport.
+ */
+
+const test = require('node:test');
+const assert = require('node:assert/strict');
+const fc = require('fast-check');
+
+const {
+  THRESHOLD_PCT,
+  computeElapsedPct,
+  isNearCap,
+  formatNearCapNotice,
+} = require('../scripts/lib/ci-job-timing.cjs');
+
+test('computeElapsedPct', async (t) => {
+  await t.test('zero elapsed yields pct exactly 0', () => {
+    const result = computeElapsedPct({
+      startedAt: '2026-01-01T00:00:00.000Z',
+      completedAt: '2026-01-01T00:00:00.000Z',
+      timeoutMinutes: 15,
+    });
+    assert.equal(result.pct, 0);
+    // Deterministic arithmetic over a fixed literal fixture (not a measured
+    // wall-clock duration); elapsedMs is part of the function's public
+    // return-shape contract that formatNearCapNotice reads directly.
+    // eslint-disable-next-line local/no-elapsed-assertion -- deterministic arithmetic, not measured timing
+    assert.equal(result.elapsedMs, 0);
+    assert.equal(result.capMs, 900000);
+  });
+
+  await t.test('elapsed exactly equal to cap yields pct exactly 1 and is near-cap', () => {
+    const result = computeElapsedPct({
+      startedAt: '2026-01-01T00:00:00.000Z',
+      completedAt: '2026-01-01T00:15:00.000Z',
+      timeoutMinutes: 15,
+    });
+    assert.equal(result.pct, 1);
+    assert.equal(isNearCap(result.pct), true);
+  });
+
+  await t.test('elapsed at 89.99% of cap is NOT near-cap', () => {
+    const startedAt = new Date('2026-01-01T00:00:00.000Z');
+    const completedAt = new Date(startedAt.getTime() + 809910);
+    const result = computeElapsedPct({
+      startedAt: startedAt.toISOString(),
+      completedAt: completedAt.toISOString(),
+      timeoutMinutes: 15,
+    });
+    assert.equal(isNearCap(result.pct), false);
+  });
+
+  await t.test('elapsed at exactly 90% of cap IS near-cap (threshold is inclusive)', () => {
+    const startedAt = new Date('2026-01-01T00:00:00.000Z');
+    const completedAt = new Date(startedAt.getTime() + 810000);
+    const result = computeElapsedPct({
+      startedAt: startedAt.toISOString(),
+      completedAt: completedAt.toISOString(),
+      timeoutMinutes: 15,
+    });
+    assert.equal(result.pct, 0.9);
+    assert.equal(isNearCap(result.pct), true);
+  });
+
+  await t.test('elapsed at 90.01% of cap is near-cap', () => {
+    const startedAt = new Date('2026-01-01T00:00:00.000Z');
+    const completedAt = new Date(startedAt.getTime() + 810090);
+    const result = computeElapsedPct({
+      startedAt: startedAt.toISOString(),
+      completedAt: completedAt.toISOString(),
+      timeoutMinutes: 15,
+    });
+    assert.equal(isNearCap(result.pct), true);
+  });
+
+  await t.test('elapsed beyond the cap does not throw, pct > 1, near-cap', () => {
+    const result = computeElapsedPct({
+      startedAt: '2026-01-01T00:00:00.000Z',
+      completedAt: '2026-01-01T00:20:00.000Z',
+      timeoutMinutes: 15,
+    });
+    assert.ok(result.pct > 1);
+    assert.equal(isNearCap(result.pct), true);
+  });
+
+  await t.test('timeoutMinutes of 0 throws', () => {
+    assert.throws(
+      () =>
+        computeElapsedPct({
+          startedAt: '2026-01-01T00:00:00.000Z',
+          completedAt: '2026-01-01T00:15:00.000Z',
+          timeoutMinutes: 0,
+        }),
+      /timeoutMinutes must be a positive finite number/,
+    );
+  });
+
+  await t.test('negative timeoutMinutes throws', () => {
+    assert.throws(
+      () =>
+        computeElapsedPct({
+          startedAt: '2026-01-01T00:00:00.000Z',
+          completedAt: '2026-01-01T00:15:00.000Z',
+          timeoutMinutes: -5,
+        }),
+      /timeoutMinutes must be a positive finite number/,
+    );
+  });
+
+  await t.test('malformed startedAt throws', () => {
+    assert.throws(
+      () =>
+        computeElapsedPct({
+          startedAt: 'not-a-date',
+          completedAt: '2026-01-01T00:15:00.000Z',
+          timeoutMinutes: 15,
+        }),
+      /startedAt is not a valid timestamp/,
+    );
+  });
+
+  await t.test('completedAt before startedAt throws', () => {
+    assert.throws(
+      () =>
+        computeElapsedPct({
+          startedAt: '2026-01-01T01:00:00.000Z',
+          completedAt: '2026-01-01T00:00:00.000Z',
+          timeoutMinutes: 15,
+        }),
+      /completedAt .* is before startedAt/,
+    );
+  });
+});
+
+test('formatNearCapNotice', async (t) => {
+  await t.test('produces a warningLine and summaryMarkdown containing label and pct', () => {
+    const { warningLine, summaryMarkdown } = formatNearCapNotice({
+      label: 'test (ubuntu-latest, 24, shard 1/3)',
+      pct: 0.92,
+      elapsedMs: 828000,
+      capMs: 900000,
+    });
+
+    assert.ok(warningLine.startsWith('::warning'));
+    assert.ok(warningLine.includes('test (ubuntu-latest, 24, shard 1/3)'));
+    assert.ok(warningLine.includes('92%'));
+
+    assert.ok(summaryMarkdown.includes('test (ubuntu-latest, 24, shard 1/3)'));
+    assert.ok(summaryMarkdown.includes('92%'));
+  });
+});
+
+test('computeElapsedPct / isNearCap — property: pct matches direct division and threshold predicate agrees', () => {
+  fc.assert(
+    fc.property(
+      fc.record({
+        timeoutMinutes: fc.integer({ min: 1, max: 500 }),
+        // Expressed as thousandths of the cap so elapsedMs stays a whole
+        // number of ms in [0, 10 * capMs] without generating capMs first.
+        fractionOfCapMilli: fc.integer({ min: 0, max: 10000 }),
+      }),
+      ({ timeoutMinutes, fractionOfCapMilli }) => {
+        const capMs = timeoutMinutes * 60000;
+        const elapsedMs = Math.floor((capMs * fractionOfCapMilli) / 1000);
+
+        const startedAt = new Date('2026-01-01T00:00:00.000Z');
+        const completedAt = new Date(startedAt.getTime() + elapsedMs);
+
+        const result = computeElapsedPct({
+          startedAt: startedAt.toISOString(),
+          completedAt: completedAt.toISOString(),
+          timeoutMinutes,
+        });
+
+        // elapsedMs/capMs here are deterministically constructed by this test
+        // from fast-check inputs (see above), not a measured wall-clock
+        // duration; this is the core invariant under test.
+        // eslint-disable-next-line local/no-elapsed-assertion -- deterministic arithmetic, not measured timing
+        assert.equal(result.pct, elapsedMs / capMs);
+        assert.equal(isNearCap(result.pct), result.pct >= THRESHOLD_PCT);
+      },
+    ),
+    { numRuns: 200 },
+  );
+});
--- a/tests/ci-test-job-timeout-budget.test.cjs
+++ b/tests/ci-test-job-timeout-budget.test.cjs
@@ -31,6 +31,7 @@ const assert = require('node:assert/strict');
 const fs = require('node:fs');
 const path = require('node:path');
 const yaml = require('js-yaml');
+const { COVERED } = require('../scripts/mutation-matrix.cjs');

 const WORKFLOWS_DIR = path.join(__dirname, '..', '.github', 'workflows');

@@ -98,6 +99,19 @@ const LANE_COSTS = [
    // around a minute. Listed so its budget cannot be dropped to nothing.
    evidence: 'targeted-only lane, ~1m observed',
  },
+  {
+    job: 'smoke',
+    workflowFile: 'install-smoke.yml',
+    measuredMinutes: 2,
+    // Worst observed wall-clock across the two most recent PUSH-triggered
+    // (full-matrix, macos-latest included) runs: 65s on macos-latest, run
+    // 32260569855 (2026-08-19). A second push run (31240989202, 2026-08-08)
+    // measured 43-56s across its three jobs, all under this figure. PR-context
+    // runs are faster (~50-62s, ubuntu only, macOS full_only row skipped) and
+    // are not the binding case. Rounded up to whole minutes per this file's
+    // convention.
+    evidence: 'run 32260569855 — 65s, macos-latest push (full matrix)',
+  },
 ];

 function requiredBudgetMinutes(measuredMinutes, headroomFactor = HEADROOM_FACTOR) {
@@ -110,13 +124,14 @@ function hasSufficientBudget(budgetMinutes, measuredMinutes, headroomFactor = HE
 }

 test('CI job timeout budgets carry headroom over measured cost (#2952)', async (t) => {
-  const workflow = loadWorkflow('test.yml');
-
  for (const lane of LANE_COSTS) {
    await t.test(`${lane.job} is budgeted above its measured cost`, () => {
+      const workflowFile = lane.workflowFile || 'test.yml';
+      const workflow = loadWorkflow(workflowFile);
+
      assert.ok(
        workflow.jobs && Object.prototype.hasOwnProperty.call(workflow.jobs, lane.job),
-        `.github/workflows/test.yml declares no job \`${lane.job}\`. If it was `
+        `.github/workflows/${workflowFile} declares no job \`${lane.job}\`. If it was `
        + 'renamed or removed, update LANE_COSTS in this file to match — do not '
        + 'delete the entry to make this pass.',
      );
@@ -126,7 +141,7 @@ test('CI job timeout budgets carry headroom over measured cost (#2952)', async (

      assert.equal(
        typeof budget, 'number',
-        `.github/workflows/test.yml jobs.${lane.job} must declare timeout-minutes`,
+        `.github/workflows/${workflowFile} jobs.${lane.job} must declare timeout-minutes`,
      );
      assert.ok(
        hasSufficientBudget(budget, lane.measuredMinutes),
@@ -170,3 +185,92 @@ test('CI job timeout budgets carry headroom over measured cost (#2952)', async (
    assert.equal(requiredBudgetMinutes(10, 1.5), 15);
  });
 });
+
+test('mutation.yml mutate job timeout budgets (#4036)', async (t) => {
+  await t.test('mutate job timeout-minutes is matrix-driven, not a fixed literal', () => {
+    const workflow = loadWorkflow('mutation.yml');
+    assert.equal(
+      workflow.jobs.mutate['timeout-minutes'],
+      '${{ matrix.timeoutMinutes }}',
+      'mutation.yml jobs.mutate.timeout-minutes must stay matrix-driven so each covered '
+      + 'module can declare its own per-shard budget via scripts/mutation-matrix.cjs — a '
+      + 'fixed literal here would either under-budget a slow module or over-budget every '
+      + 'fast one.',
+    );
+  });
+
+  await t.test('every covered module declares a sane per-shard timeout', () => {
+    for (const [name, mod] of Object.entries(COVERED)) {
+      const timeoutMinutes = mod.timeoutMinutes || 15;
+      assert.ok(
+        Number.isInteger(timeoutMinutes) && timeoutMinutes >= 15,
+        `COVERED.${name}.timeoutMinutes resolves to ${timeoutMinutes}, but must be an `
+        + 'integer >= 15 (the shared default) — a module\'s override must never budget '
+        + 'BELOW the shared floor every other module gets for free.',
+      );
+    }
+  });
+
+  await t.test('frontmatter mutate shard is budgeted above its measured cost', () => {
+    // Measured: CI run 33026833181 — 713s (11m53s) under the tap runner
+    // (coverageAnalysis: 'perTest'). See scripts/mutation-matrix.cjs's own
+    // comment on the `frontmatter` COVERED entry for the full citation.
+    const measuredMinutes = 12; // 713s rounded up
+    const declared = COVERED.frontmatter.timeoutMinutes || 15;
+    const required = requiredBudgetMinutes(measuredMinutes);
+    assert.ok(
+      hasSufficientBudget(declared, measuredMinutes),
+      `COVERED.frontmatter.timeoutMinutes is ${declared}, but the shard measured `
+      + `${measuredMinutes}m (run 33026833181 — 713s) and needs at least ${required} — `
+      + `${HEADROOM_FACTOR}x — so this module cannot quietly regress toward its cap.`,
+    );
+  });
+});
+
+test('near-cap check CI_JOB_TIMEOUT_MINUTES literals match each job\'s own timeout-minutes (#4036)', async (t) => {
+  const staticLanes = [
+    { workflowFile: 'test.yml', jobKey: 'test', envLiteral: '15' },
+    { workflowFile: 'test.yml', jobKey: 'test-full', envLiteral: '45' },
+    { workflowFile: 'install-smoke.yml', jobKey: 'smoke', envLiteral: '12' },
+  ];
+
+  for (const lane of staticLanes) {
+    await t.test(`${lane.workflowFile} jobs.${lane.jobKey}: CI_JOB_TIMEOUT_MINUTES matches timeout-minutes`, () => {
+      const workflow = loadWorkflow(lane.workflowFile);
+      const declared = workflow.jobs[lane.jobKey]['timeout-minutes'];
+      assert.equal(
+        String(declared), lane.envLiteral,
+        `.github/workflows/${lane.workflowFile} jobs.${lane.jobKey}.timeout-minutes is ${declared}, `
+        + `but the near-cap check step's CI_JOB_TIMEOUT_MINUTES literal is hardcoded to '${lane.envLiteral}' `
+        + '— these two must be updated together (GH Actions has no expression to read a sibling job-level '
+        + 'key from within a step\'s env, so this parity test is the drift guard instead). Update BOTH the '
+        + 'literal in this test AND the CI_JOB_TIMEOUT_MINUTES env value in the workflow step when the cap changes.',
+      );
+    });
+  }
+
+  await t.test('ci-timeout-report.cjs JOB_RULES prefixes still match each job\'s declared name: template', () => {
+    const { JOB_RULES } = require('../scripts/ci-timeout-report.cjs');
+    const testWorkflow = loadWorkflow('test.yml');
+
+    const testRule = JOB_RULES.find((r) => r.workflowFile === 'test.yml' && r.jobKey === 'test');
+    assert.ok(testWorkflow.jobs.test.name.startsWith('test ('),
+      'test.yml jobs.test.name no longer starts with "test (" — update JOB_RULES in scripts/ci-timeout-report.cjs to match');
+    assert.equal(testRule.test('test (ubuntu-latest, 24, shard 1/3)'), true);
+
+    const testFullRule = JOB_RULES.find((r) => r.workflowFile === 'test.yml' && r.jobKey === 'test-full');
+    assert.ok(testWorkflow.jobs['test-full'].name.startsWith('full test ('),
+      'test.yml jobs.test-full.name no longer starts with "full test (" — update JOB_RULES to match');
+    assert.equal(testFullRule.test('full test (windows-latest, 24, shard 1/3)'), true);
+
+    const testInertRule = JOB_RULES.find((r) => r.workflowFile === 'test.yml' && r.jobKey === 'test-inert');
+    assert.equal(testWorkflow.jobs['test-inert'].name, 'test (inert CI)',
+      'test.yml jobs.test-inert.name changed — update JOB_RULES to match');
+    assert.equal(testInertRule.test('test (inert CI)'), true);
+
+    const coverageGateRule = JOB_RULES.find((r) => r.workflowFile === 'test.yml' && r.jobKey === 'coverage-gate');
+    assert.equal(testWorkflow.jobs['coverage-gate'].name, 'Coverage gate (merged shards)',
+      'test.yml jobs.coverage-gate.name changed — update JOB_RULES to match');
+    assert.equal(coverageGateRule.test('Coverage gate (merged shards)'), true);
+  });
+});
--- a/tests/ci-timeout-report.test.cjs
+++ b/tests/ci-timeout-report.test.cjs
@@ -0,0 +1,264 @@
+'use strict';
+
+/**
+ * tests/ci-timeout-report.test.cjs
+ *
+ * Unit tests for scripts/ci-timeout-report.cjs's pure exports (#4036).
+ * main() is impure orchestration requiring a live Octokit/GitHub Actions
+ * context and is intentionally NOT covered here.
+ */
+
+const test = require('node:test');
+const assert = require('node:assert/strict');
+
+const {
+  resolveJobTimeoutMinutes,
+  parseJobRecord,
+  buildReportLines,
+  dedupeAgainstHistory,
+} = require('../scripts/ci-timeout-report.cjs');
+
+test('resolveJobTimeoutMinutes', async (t) => {
+  await t.test('static job: resolves timeout-minutes from workflow YAML', () => {
+    const yamlText = [
+      'jobs:',
+      '  test:',
+      '    timeout-minutes: 15',
+      '',
+    ].join('\n');
+
+    const result = resolveJobTimeoutMinutes({
+      jobName: 'test (ubuntu-latest, 24, shard 1/3)',
+      workflowFile: 'test.yml',
+      workflowYamlText: yamlText,
+      covered: null,
+    });
+
+    assert.equal(result, 15);
+  });
+
+  await t.test('mutation job: resolves override timeoutMinutes from COVERED', () => {
+    const result = resolveJobTimeoutMinutes({
+      jobName: 'Stryker (frontmatter)',
+      workflowFile: 'mutation.yml',
+      workflowYamlText: null,
+      covered: { frontmatter: { timeoutMinutes: 20 }, 'adr-parser': {} },
+    });
+
+    assert.equal(result, 20);
+  });
+
+  await t.test('mutation job: falls back to default 15 when no override', () => {
+    const result = resolveJobTimeoutMinutes({
+      jobName: 'Stryker (adr-parser)',
+      workflowFile: 'mutation.yml',
+      workflowYamlText: null,
+      covered: { frontmatter: { timeoutMinutes: 20 }, 'adr-parser': {} },
+    });
+
+    assert.equal(result, 15);
+  });
+
+  await t.test('mutation job: unknown module returns null', () => {
+    const result = resolveJobTimeoutMinutes({
+      jobName: 'Stryker (totally-unknown-module)',
+      workflowFile: 'mutation.yml',
+      workflowYamlText: null,
+      covered: { frontmatter: { timeoutMinutes: 20 }, 'adr-parser': {} },
+    });
+
+    assert.equal(result, null);
+  });
+
+  await t.test('test-inert resolves against the test-inert job key, not test', () => {
+    const yamlText = [
+      'jobs:',
+      '  test:',
+      '    timeout-minutes: 15',
+      '  test-inert:',
+      '    timeout-minutes: 2',
+      '',
+    ].join('\n');
+
+    const result = resolveJobTimeoutMinutes({
+      jobName: 'test (inert CI)',
+      workflowFile: 'test.yml',
+      workflowYamlText: yamlText,
+      covered: null,
+    });
+
+    assert.equal(result, 2);
+  });
+});
+
+test('parseJobRecord', async (t) => {
+  await t.test('still-running job (completed_at null) returns null', () => {
+    const result = parseJobRecord({
+      job: {
+        name: 'test (ubuntu-latest, 24, shard 1/3)',
+        completed_at: null,
+        started_at: '2026-08-29T00:00:00Z',
+        run_id: 1,
+        head_sha: 'abc123',
+        runEvent: 'pull_request',
+      },
+      workflowFile: 'test.yml',
+      workflowYamlText: 'jobs:\n  test:\n    timeout-minutes: 15\n',
+      covered: null,
+    });
+
+    assert.equal(result, null);
+  });
+
+  await t.test('untracked job name returns null', () => {
+    for (const jobName of ['preflight', 'changes', 'lint-tests']) {
+      const result = parseJobRecord({
+        job: {
+          name: jobName,
+          completed_at: '2026-08-29T00:10:00Z',
+          started_at: '2026-08-29T00:00:00Z',
+          run_id: 1,
+          head_sha: 'abc123',
+          runEvent: 'pull_request',
+        },
+        workflowFile: 'test.yml',
+        workflowYamlText: 'jobs:\n  test:\n    timeout-minutes: 15\n',
+        covered: null,
+      });
+
+      assert.equal(result, null, `expected null for job name ${jobName}`);
+    }
+  });
+
+  await t.test('valid smoke job returns a full record', () => {
+    const yamlText = [
+      'jobs:',
+      '  smoke:',
+      '    timeout-minutes: 12',
+      '',
+    ].join('\n');
+
+    const result = parseJobRecord({
+      job: {
+        name: 'smoke (ubuntu-latest)',
+        started_at: '2026-08-29T00:00:00Z',
+        completed_at: '2026-08-29T00:06:00Z',
+        run_id: 42,
+        head_sha: 'deadbeef',
+        runEvent: 'push',
+      },
+      workflowFile: 'install-smoke.yml',
+      workflowYamlText: yamlText,
+      covered: null,
+    });
+
+    assert.ok(result);
+    assert.equal(result.jobName, 'smoke (ubuntu-latest)');
+    assert.equal(result.workflowFile, 'install-smoke.yml');
+    assert.equal(result.runId, 42);
+    assert.equal(result.sha, 'deadbeef');
+    assert.equal(result.runEvent, 'push');
+    assert.equal(result.timeoutMinutes, 12);
+    assert.equal(typeof result.pct, 'number');
+  });
+});
+
+test('dedupeAgainstHistory', async (t) => {
+  await t.test('excludes only the record already present in history', () => {
+    const records = [
+      { runId: 1, jobName: 'test (ubuntu-latest, 24, shard 1/3)', pct: 0.5 },
+      { runId: 2, jobName: 'test (ubuntu-latest, 24, shard 2/3)', pct: 0.6 },
+    ];
+    const historyText = `${JSON.stringify({ runId: 1, jobName: 'test (ubuntu-latest, 24, shard 1/3)' })}\n`;
+
+    const result = dedupeAgainstHistory(records, historyText);
+
+    assert.equal(result.length, 1);
+    assert.equal(result[0].runId, 2);
+  });
+
+  await t.test('same runId, different jobName: both kept when history is empty', () => {
+    const records = [
+      { runId: 1, jobName: 'test (ubuntu-latest, 24, shard 1/3)', pct: 0.5 },
+      { runId: 1, jobName: 'test (ubuntu-latest, 24, shard 2/3)', pct: 0.6 },
+    ];
+
+    const result = dedupeAgainstHistory(records, '');
+
+    assert.equal(result.length, 2);
+  });
+
+  await t.test('malformed history lines are skipped, not thrown', () => {
+    const records = [
+      { runId: 1, jobName: 'test (ubuntu-latest, 24, shard 1/3)', pct: 0.5 },
+      { runId: 2, jobName: 'test (ubuntu-latest, 24, shard 2/3)', pct: 0.6 },
+    ];
+    const historyText = [
+      JSON.stringify({ runId: 1, jobName: 'test (ubuntu-latest, 24, shard 1/3)' }),
+      '',
+      'not json{',
+      '',
+    ].join('\n');
+
+    const result = dedupeAgainstHistory(records, historyText);
+
+    assert.equal(result.length, 1);
+    assert.equal(result[0].runId, 2);
+  });
+});
+
+test('buildReportLines', async (t) => {
+  await t.test('end-to-end: only tracked+completed jobs produce records', () => {
+    const workflowYamlText = [
+      'jobs:',
+      '  test:',
+      '    timeout-minutes: 15',
+      '',
+    ].join('\n');
+
+    const runs = [
+      {
+        run: { id: 1, head_sha: 'sha1', event: 'pull_request' },
+        jobs: [
+          {
+            name: 'test (ubuntu-latest, 24, shard 1/3)',
+            started_at: '2026-08-29T00:00:00Z',
+            completed_at: '2026-08-29T00:05:00Z',
+          },
+          {
+            name: 'test (ubuntu-latest, 24, shard 2/3)',
+            started_at: '2026-08-29T00:00:00Z',
+            completed_at: null,
+          },
+          {
+            name: 'lint-tests',
+            started_at: '2026-08-29T00:00:00Z',
+            completed_at: '2026-08-29T00:01:00Z',
+          },
+        ],
+      },
+      {
+        run: { id: 2, head_sha: 'sha2', event: 'push' },
+        jobs: [
+          {
+            name: 'test (ubuntu-latest, 24, shard 3/3)',
+            started_at: '2026-08-29T00:00:00Z',
+            completed_at: '2026-08-29T00:07:00Z',
+          },
+        ],
+      },
+    ];
+
+    const result = buildReportLines(runs, { workflowFile: 'test.yml', workflowYamlText, covered: null });
+
+    assert.equal(result.length, 2);
+    const names = result.map((r) => r.jobName).sort();
+    assert.deepEqual(names, [
+      'test (ubuntu-latest, 24, shard 1/3)',
+      'test (ubuntu-latest, 24, shard 3/3)',
+    ]);
+    for (const name of names) {
+      assert.equal(names.filter((n) => n === name).length, 1);
+    }
+  });
+});
--- a/tests/fixtures/install-tree/antigravity.json
+++ b/tests/fixtures/install-tree/antigravity.json
@@ -430,6 +430,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/augment.json
+++ b/tests/fixtures/install-tree/augment.json
@@ -500,6 +500,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/claude-local.json
+++ b/tests/fixtures/install-tree/claude-local.json
@@ -500,6 +500,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/claude.json
+++ b/tests/fixtures/install-tree/claude.json
@@ -429,6 +429,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/cline.json
+++ b/tests/fixtures/install-tree/cline.json
@@ -392,6 +392,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/codebuddy.json
+++ b/tests/fixtures/install-tree/codebuddy.json
@@ -500,6 +500,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/codex.json
+++ b/tests/fixtures/install-tree/codex.json
@@ -431,6 +431,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/copilot.json
+++ b/tests/fixtures/install-tree/copilot.json
@@ -392,6 +392,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/cursor.json
+++ b/tests/fixtures/install-tree/cursor.json
@@ -403,6 +403,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/hermes.json
+++ b/tests/fixtures/install-tree/hermes.json
@@ -429,6 +429,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/kilo.json
+++ b/tests/fixtures/install-tree/kilo.json
@@ -503,6 +503,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/kimi-code.json
+++ b/tests/fixtures/install-tree/kimi-code.json
@@ -430,6 +430,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/kimi.json
+++ b/tests/fixtures/install-tree/kimi.json
@@ -427,6 +427,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/opencode.json
+++ b/tests/fixtures/install-tree/opencode.json
@@ -503,6 +503,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/pi.json
+++ b/tests/fixtures/install-tree/pi.json
@@ -396,6 +396,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/qwen.json
+++ b/tests/fixtures/install-tree/qwen.json
@@ -429,6 +429,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/trae.json
+++ b/tests/fixtures/install-tree/trae.json
@@ -390,6 +390,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/windsurf.json
+++ b/tests/fixtures/install-tree/windsurf.json
@@ -393,6 +393,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",
--- a/tests/fixtures/install-tree/zcode.json
+++ b/tests/fixtures/install-tree/zcode.json
@@ -461,6 +461,7 @@
  "scripts/gen-loop-host-contract.cjs",
  "scripts/lib/alias-drift-families.cjs",
  "scripts/lib/allowlist-ratchet.cjs",
+  "scripts/lib/ci-job-timing.cjs",
  "scripts/lib/cli-exit.cjs",
  "scripts/lib/drift-scan.cjs",
  "scripts/lib/exit-code-registry.cjs",