fix(#3319): mutation-gate has_work=false renders skipped, not success (#3399)

The has_work=false branch previously exit-0'd from a step that ran,
so the job's conclusion was `success` -- identical to a PR that
actually ran mutation testing and passed. Moved the trivial-pass
condition to the job's own `if:`, so the job is SKIPPED (not run)
when has_work=false, matching the `coverage-gate` precedent in
test.yml and confirmed (via job-level `if:` research plus this
repo's own live PR #3392) that a skipped required check does not
block merge while still rendering visibly distinct from a pass.

First design attempt (splitting into two step-level `if:`-gated
steps) was verified WRONG before landing: a job whose every step is
individually skipped via step-level `if:` still reports `success`,
not `skipped` -- confirmed against documented GitHub Actions
behavior, not assumed.

Empirically verified via `workflow_dispatch`:
  pre-fix  (run 31652963610, on next): mutation-gate conclusion = success
  post-fix (see PR): mutation-gate conclusion = skipped

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-08-12 21:00:54 -04:00
committed by GitHub
parent 0f2a14e3a3
commit 67a860ca98

View File

@@ -5,7 +5,7 @@ name: Mutation Testing
# are "covered" (have meaningful test coverage for mutation). It computes
# changed modules from git diff and emits a GitHub Actions matrix so each
# changed module gets its own Stryker shard running in parallel.
# If no covered modules changed the gate passes trivially (has_work: false).
# If no covered modules changed the gate is skipped, not trivially passed (has_work: false).
on:
pull_request:
@@ -147,39 +147,38 @@ jobs:
retention-days: 14
# ── Job 3: mutation-gate ───────────────────────────────────────────────────
# Stable required-check name for branch protection. Passes when:
# • has_work is false (nothing to mutate — trivial pass), OR
# • all mutate shards succeeded.
# Fails when any shard failed.
# Stable required-check name for branch protection.
# • has_work is false → job is SKIPPED (not run) — nothing to mutate is not
# the same claim as "mutated and passed" (#3319).
# • has_work is true and all mutate shards succeeded → PASSES.
# • has_work is true and any shard failed, or detect itself failed → FAILS.
# Summary job keeps the LEGACY check name so existing branch protection
# (which requires "Stryker mutation score (changed files only)") needs no
# change. The per-module shards report as "Stryker (<module>)".
mutation-gate:
name: Stryker mutation score (changed files only)
needs: [detect, mutate]
if: always()
# has_work=false is rendered by SKIPPING this job entirely (job-level `if:`), not by an
# early `exit 0` inside the step — a step that runs and exits 0 always reports `success`,
# indistinguishable in the UI from "evaluated and passed". A job-level-skipped job reports
# `skipped` instead, matching the `coverage-gate` precedent in test.yml, and (verified)
# skipped required checks do not block merge. See #3319.
if: always() && (needs.detect.result != 'success' || needs.detect.outputs.has_work == 'true')
runs-on: ubuntu-latest
steps:
- name: Evaluate gate
run: |
DETECT="${{ needs.detect.result }}"
MUTATE="${{ needs.mutate.result }}"
HAS_WORK="${{ needs.detect.outputs.has_work }}"
echo "detect result : ${DETECT}"
echo "mutate result : ${MUTATE}"
echo "has_work : ${HAS_WORK}"
if [ "${DETECT}" != "success" ]; then
echo "FAIL: detect job did not succeed (${DETECT})"
exit 1
fi
if [ "${HAS_WORK}" = "false" ]; then
echo "PASS: no covered modules changed — gate trivially green"
exit 0
fi
if [ "${MUTATE}" = "success" ]; then
echo "PASS: all mutation shards passed"
exit 0