test-conformance's macos-latest leg (Phase 2, #4591) has been running the
same 546-file, Windows-oriented conformance-tier list as windows-latest --
built from signals like windows-shell-token/windows-env-var that have
nothing to do with macOS. Issue #4593 asked for macOS coverage sized to
its own evidence-backed surface (zsh dispatch, case-sensitivity, darwin-
specific behavior) instead.
Issue #4593 was filed before Phase 5 (#4603) existed and referenced
updating test-full's macOS legs -- that job is gone. Corrected the issue's
body before any code was touched: the "shrink from full replay" half of
the original ask was already done by Phase 5; what remained was narrowing
the still-Windows-oriented tier macOS was inheriting.
Two design assumptions were measured and rejected before accepting a
design (documented in docs/adr/4593-macos-conformance-tier-architecture.md):
- Reusing the general tier's signals minus its 3 Windows-specific
categories barely narrows anything (546 -> 424, 78% retained) -- most
files match multiple signals and only need one to survive exclusion.
- A standalone CRLF/autocrlf signal, despite the issue naming
"CRLF-checkout behavior": even narrowed to /\bCRLF\b|autocrlf/i it hit
143/930 files. Root cause: CRLF is primarily a Windows checkout concern
in this codebase (ADR-1703 files it under DEFECT.WINDOWS-TEST-
PORTABILITY), so the signal was really re-selecting Windows-relevant
files already covered by the general tier, not narrowing macOS
specifically.
Built 5 new, genuinely macOS-specific signals instead: darwin-literal
(darwin alone, not the general tier's win32-OR-darwin), zsh-dispatch,
case-sensitivity, plus chmod-mode-bit and symlink-keyword reused verbatim
from the general tier (genuinely Unix-relevant, not Windows-motivated).
Measured against the real tree: 196 of 930 eligible unit-suite files
(21%), versus the general tier's 546 (59%) -- a real, evidence-backed
narrowing.
scripts/gen-platform-conformance-tier.cjs gains classifyMacosContent/
classifyMacosTree/renderMacosGeneratedFile and a --target windows
(default, unchanged)/--target macos CLI flag, so the same generator
produces two independent, gated outputs rather than needing a second
script. New committed output: scripts/lib/macos-conformance-tier.
generated.cjs. .github/workflows/test.yml's test-conformance job: only
the macos-latest leg's file-list source changes; windows-latest is
byte-for-byte untouched. New shipped-file ripples handled proactively
(19 install-tree fixtures regenerated, bin/install.js registered).
An isolated code-review pass found one real defect: the ADR's per-
category count table had drifted by 1 (zsh-dispatch, case-sensitivity)
because the new test file's own fixture strings joined the tree it
classifies after the table was authored -- fixed, with the union total
(196, what CI actually gates on) confirmed unaffected. An isolated
security-review pass found no qualifying findings.
The ADR also records an explicit requirement for any future widening
proposal: check whether the motivating regression is already covered by
Phase 1's no-rendered-text-length-assert lint rule (#4590) before
re-proposing full macOS/Linux parity, since that is exactly what #4421's
root cause was (a rendered-text-length assertion, not a real behavioral
divergence).
Co-authored-by: sim <sim@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
scripts/ci-test-scope.cjs's classify() previously set full_matrix=true for
ANY changed tests/**/*.test.cjs file, unconditionally (restored by #4421
after #962's narrowing let a real macOS-only regression, PR #4384, land
undetected). This replaces that blanket rule with a reachability check
against real data instead of a path prefix:
- A changed test file forces full_matrix only when it is present in Phase
2's committed CONFORMANCE_TIER_FILES list (scripts/lib/platform-
conformance-tier.generated.cjs) -- direct membership, not a graph walk.
- A changed src/ file forces full_matrix when its own content carries a
genuine platform-conditional signal, reusing gen-platform-conformance-
tier.cjs's classifyContent with a narrowed, source-code-safe signal
subset (excludes two categories -- hardcoded-path-vs-path-call and
symlink-keyword -- empirically found to flag 100/235 src/ files when
applied verbatim, versus 28/235 with the narrow subset, all verified to
carry genuine platform branches). New export: NOISY_FOR_SOURCE_REACHABILITY.
- A change to the classification mechanism's own definition files
(gen-platform-conformance-tier.cjs, the generated tier list, or
suite-detection.cjs) always forces full_matrix -- the mechanism being
changed cannot presume its own new output is safe.
- Any computation error (a require/read failure, a malformed module) fails
safe to full_matrix=true, per the issue's explicit requirement.
The existing RULES array entries with their own fullMatrix:true (workflow
automation, installer/package layout, hooks, environment/dependency gates,
test harness) are deliberately left untouched -- they are curated,
narrowly-scoped triggers for "this diff changes the CI/installer/hooks
mechanism itself," a different and still-valid reason than "product code
might reach a platform branch." Disclosed in .gsd/phase/.../40-design.md
as a scope decision, since the issue's "Done when" wording read broader
than its "Proposed work" bullets.
Two design assumptions were caught and corrected before any code was
written (rubber-duck pass, documented in 40-design.md): (1) reusing Phase
2's classifyContent verbatim against src/ was far too noisy; (2) a single
hardcoded seam file (src/shell-command-projection.cts only, per CLAUDE.md's
"single platform seam" framing) would have silently missed genuine,
independent platform branches in src/runtime-hooks-surface.cts,
src/capability-lock.cts, src/capability-ledger.cts, and src/surface.cts --
reintroducing the #4421 failure shape inside src/ instead of tests/.
An isolated code-review pass found and fixed one real defect (a dead,
untested branch that would have survived Stryker mutation testing) and one
design-doc completeness gap (2 of 8 "narrow" signal categories were left
implicitly rather than explicitly audited). An isolated security-review
pass found no qualifying findings.
tests/ci-test-scope.test.cjs gains the full #4592 boundary-case matrix
(.gsd/phase/.../50-test-matrix.md), including a named #4421 regression case
proving tests/state-todos-render.test.cjs still forces full_matrix, now for
the documented reason instead of the removed blanket rule. Two pre-existing
tests were corrected: one used a nonexistent fixture path (src/semver.cts
-> src/semver-compare.cts, a real file); one (A3) asserted the exact old
blanket-rule behavior this issue removes, updated to the new, verified-
correct expectation.
Co-authored-by: sim <sim@local>
Co-authored-by: Claude Sonnet 5 <noreply@anthropic.com>
