test(#1074): swap workflow size enforcement to baseline + loose hard caps (PR 2/3) (#1096)

Completes the #1074 migration for workflows. The per-file baseline (PR 1) is now the primary anti-creep guard, so the tier-max tighten-only ceilings are retired here. - Delete the 'SIZE: tier anti-creep' describe (3 assertTightCeiling tests) and the GRACE constant — per-file baseline already guards every file by name, strictly stronger than max(tier). - Convert the per-file tier test into 'SIZE: workflow tier hard caps': absolute red lines (XL 96 KiB / LARGE 60 KiB / DEFAULT 40 KiB) that mean 'extract, do not raise', each with real headroom above its high-water file. - Add a 32 KiB (Codex project_doc_max_bytes) cap for net-new workflow files not yet in the baseline and not explicitly tiered. - Rewrite the header doc comment for the new two-guard model; drop the assertTightCeiling import (now unused here; still exported + used by the agent test until PR 3). Addresses the three PR-2 items from the #1089 review: - Finish the enumeration consolidation (Minor #1): the tier hard-cap and new-file guards now read both their file list and byte sizes from measureWorkflows()/listWorkflowStems(), removing the inline readdirSync + per-file byteCount split-brain. Enumeration and measurement share one source. - Rewrite CONTEXT.md RULESET.WORKFLOW_SIZE_BUDGET (Minor #2): stale caps (XL<=90000/LARGE<=54000/DEFAULT<=38000) replaced with the baseline-first model, current hard caps, the new-file anchor, and the size:baseline remediation. - Ship docs (contract-change requirement): a Diataxis how-to + reference for the size guard in docs/TESTING-SUITES.md, beside the sibling regression-name ratchet (the 'file grew, CI red -> npm run size:baseline, commit the one-line diff, justify or extract lazily' workflow). CONTRIBUTING.md's workflow-tree note is updated to the baseline model and points at the new guide. Negative proof: each guard fails independently when violated (new-file cap at 33 KB; hard cap with baseline current; baseline on any per-file growth). Refs #1074. Part 2 of 3.
2026-06-12 09:30:44 -04:00
parent 9e2ef2c94d
commit b055ca14e2
4 changed files with 154 additions and 91 deletions
--- a/CONTEXT.md
+++ b/CONTEXT.md
@@ -267,7 +267,7 @@ The canonical lint infrastructure adopted in ADR 452 (`docs/adr/452-eslint-lint-
 `RULESET.TESTS.eslint-harness=ADR 452 (2026-05-28): ESLint flat config + typescript-eslint + eslint-plugin-n + eslint-plugin-no-only-tests + local plugin at scripts/eslint-rules/; replaces scripts/lint-*.cjs regex scanners; three test-rigor rules (local/no-source-grep, local/no-magic-sleep-in-tests, local/no-elapsed-assertion) ship at warn, promoted to error after #453 cleanup sweep merges`

 `RULESET.WORKFLOW_MARKDOWN.FENCES=preserve opening language fence when editing shell snippets in workflow markdown; malformed fence creates fresh CR threads (MD040)`
-`RULESET.WORKFLOW_SIZE_BUDGET=workflow-size-budget (#717) measures BYTES not lines; tiers XL<=90000 / LARGE<=54000 / DEFAULT<=38000 bytes, discuss-phase<30000; can fail otherwise-valid review fixes — trim prose or extract LAZILY-loaded content (eager @-imports don't reduce loaded context) before final checks`
+`RULESET.WORKFLOW_SIZE_BUDGET=workflow size enforcement (#1074; BYTES not lines per #717; LF-normalized per #683) = per-file baseline (PRIMARY anti-creep: tests/workflow-size-baseline.json pins each file's exact size) + loose tier hard caps (outer red lines, NEVER raised on approach: XL<=98304 / LARGE<=61440 / DEFAULT<=40960) + new-file cap (un-baselined files <32768, the Codex anchor) + discuss-phase<32000; a file that grew fails the baseline guard — fix with `npm run size:baseline`, commit the one-line diff, and justify the growth in the PR (or extract LAZILY-loaded content; eager @-imports don't reduce loaded context); crossing a hard cap means EXTRACT, not bump`
 `RULESET.WORKFLOW_FILE_NAMES=workflow files use hyphens; <step name="..."> XML attributes must match (extract-learnings not extract_learnings); tests should pin exact hyphenated name`
 `RULESET.WORKFLOW_EXECUTION_CONTEXT=@-ref in commands/gsd/*.md must resolve to an existing file on disk; regression test in tests/bug-3135-capture-backlog-workflow.test.cjs; INVENTORY.md row + INVENTORY-MANIFEST.json families.workflows must stay in sync; "Invoked by" attribution must move when a flag absorbs a micro-skill`
 `RULESET.WORKFLOW_EXECUTE_END_TO_END=ADR-0002 standard for single-workflow commands is "Execute end-to-end." (no bolded **Follow the X workflow** fragments); flag-dispatch routing uses "execute the X workflow end-to-end." in routing bullets`
--- a/CONTRIBUTING.md
+++ b/CONTRIBUTING.md
@@ -850,8 +850,15 @@ gsd-core/
                          the canonical example (#2551). New modes for
                          discuss-phase land in
                          workflows/discuss-phase/modes/<mode>.md.
-                          Per-file budgets enforced by
-                          tests/workflow-size-budget.test.cjs.
+                          Per-file sizes are pinned by a committed baseline
+                          (tests/workflow-size-baseline.json) plus loose tier
+                          hard caps, both in tests/workflow-size-budget.test.cjs.
+                          If you legitimately grow or shrink a workflow file,
+                          run `npm run size:baseline` to update the snapshot and
+                          justify any growth in your PR (or extract content
+                          lazily). Full how-to + reference in
+                          docs/TESTING-SUITES.md (Workflow size budget); see
+                          issue #1074.
  references/           — Reference documentation (.md)
  templates/            — File templates
 agents/                 — Agent definitions (.md) — CANONICAL SOURCE
--- a/docs/TESTING-SUITES.md
+++ b/docs/TESTING-SUITES.md
@@ -59,6 +59,59 @@ the sanctioned layout (see the #443 strategy below), not a one-off regression
 pattern. If `issue-*`/`perf-*` one-offs start accumulating the same way
 `bug-*` did, extend the ratchet's regex and regenerate the allowlist.

+## Workflow size budget
+
+> Tracked by issue [#1074](https://github.com/open-gsd/gsd-core/issues/1074).
+> Bytes (not lines) per [#717](https://github.com/open-gsd/gsd-core/issues/717);
+> LF-normalized per [#683](https://github.com/open-gsd/gsd-core/issues/683).
+
+Workflow files (`gsd-core/workflows/*.md`) ship in the installed runtime and are
+loaded into the agent's context, so their byte size is a real cost. The size
+guard in `tests/workflow-size-budget.test.cjs` keeps that cost from creeping up
+invisibly. It is an **anti-creep ratchet**, sibling to the regression-name
+ratchet above — three layers, ordered from day-to-day to last-resort:
+
+| Layer | What it does | Where |
+|---|---|---|
+| **Per-file baseline** (primary) | Pins every workflow's *exact* current size in a committed snapshot. Any growth, shrink, add, or removal fails until the snapshot is regenerated — so sub-ceiling creep is caught by name and delta, not just at the tier's single largest file. | `tests/workflow-size-baseline.json` |
+| **Loose tier hard caps** (backstop) | Absolute outer red lines per tier — `XL ≤ 98304`, `LARGE ≤ 61440`, `DEFAULT ≤ 40960` bytes. Unlike the old tighten-only ceiling, a cap is **never raised** when a file approaches it: crossing it means *extract*, not bump. | `XL/LARGE/DEFAULT_CAP` |
+| **New-file cap** | A workflow not yet in the baseline must stay under `32768` bytes (the Codex `project_doc_max_bytes` anchor) unless explicitly tiered into `XL_WORKFLOWS`/`LARGE_WORKFLOWS` in the same PR. Keeps net-new orchestrators from being born oversized. | `NEW_FILE_CAP` |
+
+`discuss-phase.md` additionally has a thin-dispatcher target of `< 32000` bytes
+(issue [#2551](https://github.com/open-gsd/gsd-core/issues/2551)).
+
+### How-to: a workflow grew and CI is red
+
+The baseline guard reports the file and the byte delta. To resolve:
+
+1. **Regenerate the snapshot** and inspect the one-line diff:
+   ```bash
+   npm run size:baseline
+   git diff tests/workflow-size-baseline.json
+   ```
+2. **Justify the growth in your PR** (a sentence in the description is enough) —
+   the committed baseline diff is the review record that the larger size was a
+   deliberate, seen decision, not silent drift.
+3. **Or shrink it instead of baselining.** Prefer extraction when the growth is
+   incidental: move per-mode bodies to `workflows/<name>/modes/`, templates to
+   `workflows/<name>/templates/`, and shared prose to `gsd-core/references/` —
+   then load them **LAZILY**. Do *not* convert them to eager `@-required_reading`
+   includes: that shrinks the file's bytes without shrinking loaded context, so
+   it games the guard while making the real cost worse. See
+   `workflows/discuss-phase/` for the progressive-disclosure pattern.
+
+If a hard cap (not the baseline) is what failed, regeneration will **not** help —
+that is the signal to extract, per step 3.
+
+### Reference
+
+| Artifact | Role |
+|---|---|
+| `scripts/workflow-size.cjs` | Single source of truth — LF-normalized byte counter (`lfByteCount`) plus workflow enumeration (`listWorkflowStems`, `measureWorkflows`). Imported by **both** the guard and the generator so they can never measure or enumerate differently. |
+| `scripts/update-size-baseline.cjs` (`npm run size:baseline`) | Regenerates `tests/workflow-size-baseline.json` — sorted keys, trailing newline, idempotent. |
+| `tests/workflow-size-baseline.json` | The committed per-file snapshot (one entry per workflow). |
+| `tests/workflow-size-budget.test.cjs` | The three guards above, plus the `discuss-phase` progressive-disclosure checks. |
+
 ## Running suites locally

 ```bash
--- a/tests/workflow-size-budget.test.cjs
+++ b/tests/workflow-size-budget.test.cjs
@@ -39,25 +39,34 @@
 * content Read only at the step that needs it (see the discuss-phase mode/
 * template tests below, which forbid templates in <required_reading>).
 *
- * Tiered the same way as agent budgets (#2361):
+ * ## Enforcement model (issue #1074)
+ *
+ * Two complementary guards, neither of which is a tier-max ceiling:
+ *
+ *   1. Per-file baseline (the anti-creep): every workflow is pinned to its
+ *      exact size in `tests/workflow-size-baseline.json`. Any growth fails with
+ *      the file and delta; `npm run size:baseline` records a deliberate change
+ *      as a one-line reviewable diff. This replaced the tier-max tighten-only
+ *      ratchet (#597), which only bound the single largest file per tier and
+ *      left the other ~85 files able to grow silently.
+ *
+ *   2. Tier hard caps (the outer bound): XL/LARGE/DEFAULT are absolute red
+ *      lines with real headroom, never raised in normal work. Crossing one
+ *      means lazy extraction (the `workflows/discuss-phase/modes/`
+ *      progressive-disclosure pattern), not a +N bump. New workflow files get
+ *      the Codex `project_doc_max_bytes` anchor (32 KiB) unless explicitly
+ *      tiered in the same PR.
+ *
+ * Tiers:
 *   - XL       : top-level orchestrators (e.g., execute-phase, plan-phase)
 *   - LARGE    : multi-step planners
 *   - DEFAULT  : focused single-purpose workflows (target tier)
 *
- * Raising a budget is a deliberate choice — adjust the constant, write a
- * rationale in the PR, and confirm the bloat is not duplicated content
- * that belongs in `gsd-core/references/` (lazily loaded) or a per-mode
- * subdirectory (see `workflows/discuss-phase/modes/`, #2551).
- *
- * Tighten-only invariant (issue #597): ceilings track the tier high-water mark
- * within GRACE bytes. Budgets may only decrease, never silently creep upward.
- * The assertTightCeiling() call below enforces this automatically.
- *
 * See:
+ *   - https://github.com/open-gsd/gsd-core/issues/1074 (per-file baseline + hard caps)
 *   - https://github.com/open-gsd/gsd-core/issues/717  (bytes re-base + rationale)
- *   - https://github.com/open-gsd/gsd-core/issues/2551 (this test)
- *   - https://github.com/open-gsd/gsd-core/issues/2361 (agent budget)
- *   - https://developers.openai.com/codex/guides/agents-md (Codex 32 KB cap)
+ *   - https://github.com/open-gsd/gsd-core/issues/683  (LF-normalized byte count)
+ *   - https://developers.openai.com/codex/guides/agents-md (Codex 32 KiB cap)
 *   - https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents
 */

@@ -66,33 +75,31 @@ const assert = require('node:assert/strict');
 const fs = require('fs');
 const os = require('node:os');
 const path = require('path');
-const { assertTightCeiling, assertFileBaseline } = require('../scripts/lib/allowlist-ratchet.cjs');
-const { lfByteCount: byteCount, measureWorkflows } = require('../scripts/workflow-size.cjs');
+const { assertFileBaseline } = require('../scripts/lib/allowlist-ratchet.cjs');
+const { lfByteCount: byteCount, listWorkflowStems, measureWorkflows } = require('../scripts/workflow-size.cjs');
 const { cleanup } = require('./helpers.cjs');

 const WORKFLOWS_DIR = path.join(__dirname, '..', 'gsd-core', 'workflows');
 const BASELINE_PATH = path.join(__dirname, 'workflow-size-baseline.json');

-// Grace band: maximum allowed slack (ceiling − actualMax) in BYTES before a
-// ceiling is considered too loose. 3000 bytes ≈ the prior 60-line grace
-// re-expressed for the #717 unit swap (these files run ~36–50 bytes/line, so
-// ~60–80 lines of breathing room) without permitting gross inflation.
-const GRACE = 3000;
+// Tier HARD CAPS (#1074) — absolute red lines, not high-water-hugging ceilings.
+// Day-to-day creep is caught per-file by the baseline guard below; these exist
+// only as the outer bound where the correct response is lazy extraction, never
+// a raise. Each sits above its tier's current high-water mark with real
+// headroom (vs the old GRACE=3000 hug):
+//   XL      96 KiB — high-water plan-phase.md 92,965 → ~5.2 KB headroom
+//   LARGE   60 KiB — high-water docs-update.md 54,600 → ~6.6 KB headroom
+//   DEFAULT 40 KiB — high-water settings-advanced.md 39,160 → ~1.8 KB headroom
+// (DEFAULT is deliberately the tightest: a single-purpose workflow approaching
+// 40 KiB is the strongest extraction signal of the three.)
+const XL_CAP = 98304;       // 96 KiB
+const LARGE_CAP = 61440;    // 60 KiB
+const DEFAULT_CAP = 40960;  // 40 KiB

-// Byte ceilings (#717 re-base from lines). Each tier's ceiling tracks the
-// current high-water mark within GRACE (#597 tighten-only ratchet).
-// XL high-water mark is execute-phase.md — note that under LINES it was
-// plan-phase; bytes genuinely re-rank the tier, which is the point of #717.
-// actualMax=93130 (plan-phase, #381 CLAUDE_ENV_FILE persist clause in per-file launcher preamble);
-// slack=70 ≤ GRACE. execute-phase.md=92880, new-project.md=61685.
-// 93200: +200B headroom for the #381 CLAUDE_ENV_FILE persist clause added to every per-file launcher preamble (legit content growth, ratchet-up per #717).
-const XL_BUDGET = 93200;
-// LARGE high-water mark is docs-update.md. actualMax=54410 (#891 launcher shim expansion);
-// slack=1590 ≤ GRACE. quick.md=45710, autonomous.md=38030.
-const LARGE_BUDGET = 56000;
-// DEFAULT high-water mark is settings-advanced.md. actualMax=38409 (#891 launcher shim expansion);
-// slack=1591 ≤ GRACE.
-const DEFAULT_BUDGET = 40000;
+// New workflow files (not yet in the committed baseline) must stay under the
+// Codex project_doc_max_bytes anchor unless explicitly tiered into XL/LARGE in
+// the same PR. Keeps net-new orchestrators from being born oversized.
+const NEW_FILE_CAP = 32768; // 32 KiB

 // Top-level orchestrators that own end-to-end multi-phase rubrics.
 // Grandfathered at current sizes — see PR #2551 for the progressive-disclosure
@@ -119,14 +126,19 @@ const LARGE_WORKFLOWS = new Set([
  'code-review',           // 28726
 ]);

-const ALL_WORKFLOWS = fs.readdirSync(WORKFLOWS_DIR)
-  .filter(f => f.endsWith('.md'))
-  .map(f => f.replace('.md', ''));
+// Single source of truth for BOTH enumeration and measurement (#1074; finishes
+// the consolidation flagged in trek-e's #1089 review). The tier guards below
+// iterate exactly the files measureWorkflows() measured and read their bytes
+// from the same map, so enumeration and byte-counting can never split-brain.
+// `byteCount` (lfByteCount) is retained only for the single-file discuss-phase
+// checks below, which target files outside the workflow root.
+const SIZES = measureWorkflows();           // { 'execute-phase.md': 92880, ... }
+const ALL_WORKFLOWS = listWorkflowStems();  // ['execute-phase', ...] — same source

-function budgetFor(workflow) {
-  if (XL_WORKFLOWS.has(workflow)) return { tier: 'XL', limit: XL_BUDGET };
-  if (LARGE_WORKFLOWS.has(workflow)) return { tier: 'LARGE', limit: LARGE_BUDGET };
-  return { tier: 'DEFAULT', limit: DEFAULT_BUDGET };
+function capFor(workflow) {
+  if (XL_WORKFLOWS.has(workflow)) return { tier: 'XL', cap: XL_CAP };
+  if (LARGE_WORKFLOWS.has(workflow)) return { tier: 'LARGE', cap: LARGE_CAP };
+  return { tier: 'DEFAULT', cap: DEFAULT_CAP };
 }

 // byteCount (LF-normalized, #683) is imported as `lfByteCount` from
@@ -134,35 +146,55 @@ function budgetFor(workflow) {
 // baseline generator so the guard and the snapshot can never measure
 // differently. See the #683 regression test at the bottom of this file.

-describe('SIZE: workflow byte-size budget', () => {
+describe('SIZE: workflow tier hard caps (issue #1074)', () => {
+  // Absolute outer bound per tier. Unlike the old tighten-only ceiling, a cap
+  // is NOT raised when a file approaches it — crossing it means extract, not
+  // bump. Per-file creep is handled by the baseline guard below; this only
+  // catches a file that has grown to the point where lazy extraction is the
+  // only correct answer.
  for (const workflow of ALL_WORKFLOWS) {
-    const { tier, limit } = budgetFor(workflow);
-    test(`${workflow} (${tier}) stays under ${limit} bytes`, () => {
-      const filePath = path.join(WORKFLOWS_DIR, workflow + '.md');
-      const bytes = byteCount(filePath);
+    const { tier, cap } = capFor(workflow);
+    test(`${workflow} (${tier}) stays under the ${tier} hard cap (${cap} bytes)`, () => {
+      const bytes = SIZES[`${workflow}.md`];
      assert.ok(
-        bytes <= limit,
-        `${workflow}.md is ${bytes} bytes — exceeds ${tier} budget of ${limit}. ` +
-        `Extract per-mode bodies to a workflows/${workflow}/modes/ subdirectory, ` +
-        `templates to workflows/${workflow}/templates/, or shared references ` +
-        `to gsd-core/references/ — and load them LAZILY (not via @-required_reading, ` +
-        `which would shrink this file's bytes without shrinking loaded context). ` +
-        `See workflows/discuss-phase/ for the pattern.`
+        bytes <= cap,
+        `${workflow}.md is ${bytes} bytes — exceeds the ${tier} hard cap of ${cap}. ` +
+        `This cap is a red line, NOT a budget to raise: extract per-mode bodies to a ` +
+        `workflows/${workflow}/modes/ subdirectory, templates to ` +
+        `workflows/${workflow}/templates/, or shared references to gsd-core/references/ — ` +
+        `and load them LAZILY (not via @-required_reading, which would shrink this ` +
+        `file's bytes without shrinking loaded context). See workflows/discuss-phase/.`
      );
    });
  }
+
+  test('new workflow files (not yet baselined) stay under the 32 KiB Codex anchor', () => {
+    const baseline = JSON.parse(fs.readFileSync(BASELINE_PATH, 'utf-8'));
+    for (const workflow of ALL_WORKFLOWS) {
+      const key = `${workflow}.md`;
+      const isNew = !(key in baseline);
+      const isTiered = XL_WORKFLOWS.has(workflow) || LARGE_WORKFLOWS.has(workflow);
+      if (!isNew || isTiered) continue;
+      const bytes = SIZES[key];
+      assert.ok(
+        bytes < NEW_FILE_CAP,
+        `${key} is a new workflow at ${bytes} bytes — new files must stay under ` +
+        `${NEW_FILE_CAP} (the Codex project_doc_max_bytes anchor) unless explicitly ` +
+        `tiered into XL_WORKFLOWS/LARGE_WORKFLOWS in this same PR with a rationale.`
+      );
+    }
+  });
 });

 describe('SIZE: per-file workflow baseline (issue #1074)', () => {
-  // Per-file exact-size ratchet. Unlike the tier anti-creep block below — which
-  // only binds the single largest file in each tier — this guards EVERY
-  // workflow file by name against a committed snapshot
+  // Per-file exact-size ratchet — the primary anti-creep guard (it replaced the
+  // tier-max tighten-only ceilings, which only bound the single largest file in
+  // each tier). Guards EVERY workflow file by name against a committed snapshot
  // (tests/workflow-size-baseline.json). Growth fails with the file and delta;
  // shrinkage fails as a stale snapshot (regenerate to ratchet down). The fix
  // for any failure is `npm run size:baseline` plus a PR justification for
-  // genuine growth (or lazy extraction). Runs side-by-side with the tier tests
-  // during the #1074 migration; the tier anti-creep block is removed in a
-  // follow-up once this is established.
+  // genuine growth (or lazy extraction). The tier hard caps above are the outer
+  // bound; this is the day-to-day creep control.
  test('every workflow file matches its committed baseline', () => {
    const baseline = JSON.parse(fs.readFileSync(BASELINE_PATH, 'utf-8'));
    const current = measureWorkflows();
@@ -178,35 +210,6 @@ describe('SIZE: per-file workflow baseline (issue #1074)', () => {
  });
 });

-describe('SIZE: tier anti-creep (tighten-only ceilings, issue #597)', () => {
-  // For each tier, compute the high-water mark (in bytes) across all files in
-  // that tier and assert the ceiling stays tight. Prevents budgets from
-  // silently drifting upward: ceiling − actualMax must not exceed GRACE.
-  test('XL tier: ceiling tracks high-water mark within GRACE', () => {
-    const values = ALL_WORKFLOWS
-      .filter(w => XL_WORKFLOWS.has(w))
-      .map(w => byteCount(path.join(WORKFLOWS_DIR, w + '.md')));
-    const actualMax = Math.max(...values);
-    assertTightCeiling({ label: 'XL', actualMax, ceiling: XL_BUDGET, grace: GRACE, fail: assert.fail });
-  });
-
-  test('LARGE tier: ceiling tracks high-water mark within GRACE', () => {
-    const values = ALL_WORKFLOWS
-      .filter(w => LARGE_WORKFLOWS.has(w))
-      .map(w => byteCount(path.join(WORKFLOWS_DIR, w + '.md')));
-    const actualMax = Math.max(...values);
-    assertTightCeiling({ label: 'LARGE', actualMax, ceiling: LARGE_BUDGET, grace: GRACE, fail: assert.fail });
-  });
-
-  test('DEFAULT tier: ceiling tracks high-water mark within GRACE', () => {
-    const values = ALL_WORKFLOWS
-      .filter(w => !XL_WORKFLOWS.has(w) && !LARGE_WORKFLOWS.has(w))
-      .map(w => byteCount(path.join(WORKFLOWS_DIR, w + '.md')));
-    const actualMax = Math.max(...values);
-    assertTightCeiling({ label: 'DEFAULT', actualMax, ceiling: DEFAULT_BUDGET, grace: GRACE, fail: assert.fail });
-  });
-});
-
 describe('SIZE: discuss-phase progressive disclosure (issue #2551)', () => {
  // Issue #2551 targets discuss-phase.md as a thin dispatcher, separate from
  // the per-tier grandfathered budgets above. Originally expressed as <500