fix(#2977): tolerate a leading UTF-8 BOM before the frontmatter fence (#3076)

* test(#2977): prove extractFrontmatter returns {} on a leading BOM

Failing-first regression for #2977. extractFrontmatter's startsWith('---') byte-0
check fails on any leading byte, so a UTF-8 BOM (Windows PowerShell/Out-File, several
editors) makes every frontmatter field silently disappear. Rows 1-2 assert BOM-prefixed
frontmatter parses identically to no-BOM (incl. BOM+CRLF); Row 3 guards the no-frontmatter
negative space; Row 4 covers STATE/PLAN/SUMMARY/UAT artifact shapes; Row 5 is the control.

* fix(#2977): tolerate a leading UTF-8 BOM before the frontmatter fence

extractFrontmatter's byte-0 startsWith('---') fence check failed on any leading
byte, so a UTF-8 BOM (\uFEFF) — written by default by Windows PowerShell
`>`/Out-File (PS 5.1) and several editors — made every frontmatter field
silently disappear. STATE.md/ROADMAP.md/PLAN.md/UAT.md/SUMMARY.md all lost
phase/status/name with no error, no warning.

Strip a single leading BOM before the fence check. The rest of the function is
unchanged — the BOM is one codepoint, and removing it restores byte-0 alignment.
Negative space preserved: BOM + no-frontmatter and BOM + thematic-break Markdown
both still return {} (no false diagnostic). Scope: BOM only; the generalized
'arbitrary content before the fence' fork (tolerate vs diagnose) is a separate
product-intent decision, surfaced in the PR.

* chore(#2977): add changeset fragment

* chore(#2977): backfill changeset PR number 3076

---------

Co-authored-by: sim <sim@local>

.changeset/noble-elks-roar.md

@@ -0,0 +1,5 @@
+---
+type: Fixed
+pr: 3076
+---
+**Planning artifacts whose frontmatter is preceded by a UTF-8 byte-order mark no longer lose all their frontmatter fields** — the frontmatter parser's fence check required the opening dashes at byte zero, so a BOM written by Windows PowerShell or several editors made every field silently disappear. A leading BOM is now stripped before the check, so the fields parse identically to the no-BOM case. The no-frontmatter and thematic-break cases stay silent and empty as before.

src/frontmatter.cts

@@ -204,6 +204,18 @@ function parseYamlRegion(yaml: string): Frontmatter {
  *   hold only an in-memory string; those dedup on a content digest instead.
  */
 function extractFrontmatter(content: string, sourcePath?: string): Frontmatter {
+  // #2977: tolerate a single leading UTF-8 BOM (\uFEFF), which Windows tooling
+  // (PowerShell `>`/`Out-File` on PS 5.1, several editors) writes by default. Without this
+  // strip, the byte-0 `startsWith('---')` fence check below fails on the BOM and the whole
+  // parse collapses to {} — every frontmatter field silently disappears, and the engine
+  // proceeds as though the file had no frontmatter at all. The BOM is a single codepoint;
+  // stripping it here restores byte-0 alignment so the rest of the function is unchanged.
+  // Scope: BOM only. Arbitrary non-BOM content before the fence (leading whitespace/blank
+  // line/comment) is a separate product-intent decision (tolerate vs diagnose) left to a
+  // future change — this fix does not broaden the byte-0 fence rule beyond the BOM.
+  if (content.charCodeAt(0) === 0xFEFF) {
+    content = content.slice(1);
+  }
   // Match frontmatter only at byte 0 — a `---` block later in the document
   // body (YAML examples, horizontal rules) must never be treated as frontmatter.
   const headerEnd = content.startsWith('---\r\n') ? 5 : content.startsWith('---\n') ? 4 : -1;

tests/issue-2977-frontmatter-bom.test.cjs

@@ -0,0 +1,78 @@
+'use strict';
+process.env.GSD_TEST_MODE = '1';
+
+/**
+ * Regression test for #2977 — `extractFrontmatter` returns {} for any file whose
+ * frontmatter fence is preceded by a UTF-8 BOM (Windows PowerShell `>`/`Out-File`,
+ * several editors). The `startsWith('---')` byte-0 check fails on any leading byte,
+ * so every frontmatter field silently disappears with no error.
+ *
+ * The fix strips a leading UTF-8 BOM (\uFEFF) before the fence check. Scope: BOM only
+ * (acceptance criteria 1-3). The generalized "arbitrary content before the fence" fork
+ * (tolerate vs diagnose) is a product-intent decision, surfaced in the PR — out of scope.
+ *
+ * Matrix: .gsd/bug/fix/2977-frontmatter-bom-tolerance/50-test-matrix.md
+ */
+
+const { describe, test } = require('node:test');
+const assert = require('node:assert/strict');
+const { extractFrontmatter } = require('../gsd-core/bin/lib/frontmatter.cjs');
+
+const BOM = '\uFEFF';
+
+describe('extractFrontmatter BOM tolerance (#2977)', () => {
+  test('bomPrefixedFrontmatterParses', () => {
+    // Row 1 (failing-first regression): a BOM-prefixed frontmatter document parses
+    // identically to the same document without the BOM.
+    const clean = '---\ntitle: T\nphase: "01"\nstatus: passed\n---\n\n# Body\n';
+    const bommed = BOM + clean;
+    const expected = extractFrontmatter(clean, 'a.md');
+    const actual = extractFrontmatter(bommed, 'a.md');
+    assert.deepEqual(actual, expected, 'BOM-prefixed frontmatter must parse identically to no-BOM');
+    assert.strictEqual(actual.title, 'T', 'title field recovered');
+    assert.strictEqual(actual.phase, '01', 'phase field recovered');
+    assert.strictEqual(actual.status, 'passed', 'status field recovered');
+  });
+
+  test('bomWithCrlfParses', () => {
+    // Row 2 (acceptance #2): BOM + CRLF line endings together still parse correctly.
+    const clean = '---\r\ntitle: T\r\nphase: "01"\r\n---\r\n\r\n# Body\r\n';
+    const bommed = BOM + clean;
+    const actual = extractFrontmatter(bommed, 'a.md');
+    assert.strictEqual(actual.title, 'T', 'title recovered (BOM + CRLF)');
+    assert.strictEqual(actual.phase, '01', 'phase recovered (BOM + CRLF)');
+  });
+
+  test('bomWithNoFrontmatterStaysEmpty', () => {
+    // Row 3 (acceptance #3): a BOM prefixing a document with no frontmatter (or genuinely
+    // empty frontmatter) returns {} with no false diagnostic — same as no-BOM.
+    assert.deepEqual(extractFrontmatter(BOM + 'just plain text', 'a.md'), {}, 'BOM + no frontmatter -> {}');
+    assert.deepEqual(extractFrontmatter(BOM + '', 'a.md'), {}, 'BOM + empty -> {}');
+    // A thematic-break-first-line Markdown doc (--- then prose) must stay {} — protected by
+    // the existing false-positive threshold; the BOM strip must not lower that bar.
+    assert.deepEqual(extractFrontmatter(BOM + '---\n\nA horizontal rule, not frontmatter.\n', 'a.md'), {},
+      'BOM + thematic-break Markdown -> {} (no false diagnostic)');
+  });
+
+  test('bomAcrossArtifactTypes', () => {
+    // Row 4 (acceptance #1 across artifact types): each frontmatter-bearing artifact shape
+    // recovers its fields when BOM-prefixed.
+    const cases = [
+      { name: 'STATE.md', body: '---\ncurrent_phase: "01"\nstatus: "In progress"\n---\n\n# State\n', expect: { current_phase: '01', status: 'In progress' } },
+      { name: 'PLAN.md', body: '---\nphase: "01"\nplan: "01-01"\nstatus: "done"\n---\n\n# Plan\n', expect: { phase: '01', plan: '01-01', status: 'done' } },
+      { name: 'SUMMARY.md', body: '---\none-liner: "shipped the thing"\n---\n\n# Summary\n', expect: { 'one-liner': 'shipped the thing' } },
+      { name: 'UAT.md', body: '---\nphase: "02"\nverdict: "pass"\n---\n\n# UAT\n', expect: { phase: '02', verdict: 'pass' } },
+    ];
+    for (const c of cases) {
+      const actual = extractFrontmatter(BOM + c.body, c.name);
+      assert.deepEqual(actual, c.expect, `${c.name}: BOM-prefixed frontmatter must recover fields`);
+    }
+  });
+
+  test('controlNoBom', () => {
+    // Row 5 (no regression): no BOM, valid frontmatter still parses correctly (unchanged).
+    const actual = extractFrontmatter('---\ntitle: T\nphase: "01"\n---\n\n# Body\n', 'a.md');
+    assert.strictEqual(actual.title, 'T');
+    assert.strictEqual(actual.phase, '01');
+  });
+});