enhance(#2721): regenerating merge driver, regen:derived, and a name for the emitted-artifact family (#2730)
* test(#2721): failing-first suite for the gsd-regen driver and CONTEXT.md parity Tests precede the implementation per the TDD gate. The driver module does not exist yet, so tests/git-merge-regen-driver.test.cjs fails at require time; the contributor-standards parity assertions fail against next as it stands today, where the standards doc names two CONTEXT.md headings that have never existed. Refs #2721 * feat(#2721): add the gsd-regen merge driver and regen:derived The golden parity manifests and the two size baselines are pure functions of the source tree, so their only correct merge is "recompute" -- something git's ours/theirs interface cannot express. 140 of 143 conflicted-file instances across the open PR queue are these files. The driver deliberately does NOT regenerate. Four probes established that at merge-driver time neither the working tree nor the index reflects the merge: both hold the ours side, a file added by theirs does not exist yet, and MERGE_HEAD is unwritten. Git also invokes the driver once per conflicted path (20 here). A regenerating driver would therefore read the ours-side tree and emit a plausible-but-wrong hash manifest -- worse than a conflict, because a conflict is visible. So it accepts %A, runs zero subprocesses, records the resolved paths, and prints one notice pointing at npm run regen:derived. Staleness stays caught where it already was, by golden-install-parity in CI. Every failure path degrades toward today's behaviour (a normal conflict). install-tree is deliberately excluded per ADR-2719 section 7. Also folded in, per the no-defer rule: workflow-size.cjs claimed .md files have no eol=lf in .gitattributes; git check-attr shows eol: lf, set by .gitattributes line 2 since #1088. Refs #2721 * docs(#2721): document regen:derived and the gsd-regen merge driver Adds the how-to a contributor actually reaches for when the generated parity manifests or size baselines conflict, in both places they would look: the merge-conflict path in CONTRIBUTING.md and the full guide in TESTING-SUITES.md, including what the driver deliberately does not do (it does not clear GitHub's CONFLICTING label, and it does not regenerate mid-merge). Also scopes the new contributor-standards parity assertion to the doc's own CONTEXT.md section. Its first run flagged `## Decision`, `## Consequences` and `## Standards followed`, which the doc attributes to an ADR body and a PR body rather than to CONTEXT.md -- a doc-wide extractor would have demanded CONTEXT.md grow headings that do not belong to it. Refs #2721 * fix(#2721): stop passing %P to the merge driver — shell injection The isolated adversarial review found, and I independently reproduced, local arbitrary command execution. Git does not invoke a merge driver with an argv array. It substitutes %O %A %B %L %P textually into the configured string and runs the whole thing through a shell, and $(...) executes inside POSIX double quotes -- so quoting the placeholder does not neutralise it. %O/%A/%B are git-generated temp names and %L is an integer, but %P is the file's own path, chosen freely by any contributor. A branch renaming a covered fixture to evil$(touch PWNED_SENTINEL).json executed that command on the machine of every maintainer who merged it, and the merge still reported success. Fix removes the input rather than filtering it: %P is no longer registered, so the driver receives no attacker-controlled argument at all. The marker records a count instead of path names. A metacharacter filter would have been a guess about shell grammar; passing nothing is a property. Re-ran the identical exploit against the fixed command: nothing executed, conflict still resolved. Two regressions guard it -- a platform-independent assertion that the registered command carries no %P, and a real merge driven by the actual planInstall output with a $(...) filename. Also from review: CLI dispatch had no coverage at all (CONTRIBUTING's "CLI and command routing" matrix), which is why runInstall/runStatus now take {repoRoot} -- hardcoding REPO_ROOT was what made them untestable. Renamed planResolution to resolveAndRecord since the plan* prefix promised purity it did not have. Reconciled the eleven-vs-twelve generator count across CONTEXT.md, CONTRIBUTING.md and the changeset. Refs #2721 * test(#2721): scope safe.directory for the check-attr helper The 66f4d85a run failed 11 assertions, all in the .gitattributes scoping block, with "fatal: detected dubious ownership in repository at '/work'". The test container checks the repo out at a path its user does not own, so git refuses check-attr outright. Everything else passed (27,185). `check-attr` is a pure read of .gitattributes -- no hooks, no filters -- so the exemption is scoped to that one invocation. It is deliberately NOT applied to the driver's own production `git config` calls, which run in the user's own clone and should keep the protection. Refs #2721 * test(#2721): delete the stale assertion that the driver command carries %P The plex2 run on bdfd0856 left exactly two failures, both this test: it still asserted the pre-fix command string, i.e. the vulnerable behaviour. Deleted rather than relaxed, per RULESET.TESTS.delete-bad-tests -- its useful half is already covered, in both directions, by registeredDriverCommandNeverPassesThePlaceholderForTheFilePath. Refs #2721 * test(#2721): drive the end-to-end merges from the real planInstall output The e2e helper hand-rolled its own driver registration, and still carried %P. That meant the five real-git tests were not exercising the production command string at all -- planInstall could drift and they would keep passing. They now register exactly what a contributor gets from npm run setup:merge-driver. Refs #2721 * chore(#2721): backfill changeset pr number to 2730
This commit is contained in:
Tom Boucher
367
scripts/git-merge-regen-driver.cjs
Normal file
367
scripts/git-merge-regen-driver.cjs
Normal file
@@ -0,0 +1,367 @@
|
||||
#!/usr/bin/env node
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* git-merge-regen-driver.cjs — the `gsd-regen` git merge driver (#2721, ADR-2719 Phase 1).
|
||||
*
|
||||
* ## Why
|
||||
*
|
||||
* `tests/fixtures/golden-install-parity/*.json` and the two size baselines are pure
|
||||
* functions of the source tree. Git offers ours or theirs; both are wrong, because the
|
||||
* only correct value is recomputed from the merged tree. 140 of 143 conflicted-file
|
||||
* instances across the open PR queue are these artifacts (ADR-2719).
|
||||
*
|
||||
* ## What this driver does — and deliberately does NOT do
|
||||
*
|
||||
* It does **not** regenerate. That is not implementable, and the constraint is git's, not
|
||||
* a design preference: at the moment git invokes a merge driver, **neither the working
|
||||
* tree nor the index reflects the merge**. Both hold the ours side; a file added by
|
||||
* theirs does not exist yet; `.git/MERGE_HEAD` has not been written. A driver that shelled
|
||||
* out to the generators there would read the ours-side tree and write a
|
||||
* plausible-but-wrong hash manifest — strictly worse than a conflict, because a conflict
|
||||
* is visible and a wrong manifest is not. Git also invokes the driver once **per
|
||||
* conflicted path** (20 in this repo), so a regenerating driver would run the full build
|
||||
* plus 19 installer spawns up to twenty times per merge.
|
||||
*
|
||||
* So it resolves deterministically and without content knowledge: it accepts `%A` (which
|
||||
* already holds ours verbatim), runs **zero subprocesses**, records the resolved paths
|
||||
* under the git dir, and prints **one** notice per operation pointing at
|
||||
* `npm run regen:derived`. Staleness is caught where it already was — by
|
||||
* `tests/golden-install-parity.test.cjs` in CI.
|
||||
*
|
||||
* Every failure path degrades toward *today's* behavior (a normal conflict), never toward
|
||||
* a silent wrong resolution.
|
||||
*
|
||||
* ## Bridge, not a destination
|
||||
*
|
||||
* This driver is explicitly temporary. #2724 deletes the artifacts it guards and retires
|
||||
* it. Keeping it past that point would preserve the problem it exists to relieve.
|
||||
*
|
||||
* node scripts/git-merge-regen-driver.cjs --install # register in .git/config
|
||||
* node scripts/git-merge-regen-driver.cjs --uninstall
|
||||
* node scripts/git-merge-regen-driver.cjs --status
|
||||
*/
|
||||
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
const cp = require('node:child_process');
|
||||
|
||||
const { runMain, ExitError } = require('./lib/cli-exit.cjs');
|
||||
|
||||
const REPO_ROOT = path.join(__dirname, '..');
|
||||
const MARKER_NAME = 'gsd-regen-pending.json';
|
||||
|
||||
/** Bounded per the repo's unbounded-subprocess rule (5-30s for git). */
|
||||
const GIT_TIMEOUT_MS = 15_000;
|
||||
|
||||
/**
|
||||
* A single git operation's driver invocations land milliseconds apart, so this window is
|
||||
* ~4 orders of magnitude wider than it needs to be. It exists only so a *later* operation
|
||||
* does not inherit the previous one's silence.
|
||||
*/
|
||||
const NOTICE_WINDOW_MS = 60_000;
|
||||
|
||||
const ACTION = Object.freeze({
|
||||
ACCEPT_OURS: 'accept_ours',
|
||||
DECLINE: 'decline',
|
||||
});
|
||||
|
||||
const REASON = Object.freeze({
|
||||
OK_RESOLVED: 'ok_resolved',
|
||||
FAIL_BAD_ARGV: 'fail_bad_argv',
|
||||
FAIL_OURS_UNREADABLE: 'fail_ours_unreadable',
|
||||
});
|
||||
|
||||
const GITDIR_SOURCE = Object.freeze({
|
||||
DIRECTORY: 'directory',
|
||||
GITFILE: 'gitfile',
|
||||
UNRESOLVED: 'unresolved',
|
||||
});
|
||||
|
||||
/**
|
||||
* Locate the git dir from `cwd` without spawning git — the driver runs up to twenty times
|
||||
* per merge, so a subprocess per invocation is not affordable.
|
||||
*
|
||||
* Handles both shapes: `.git` as a directory, and `.git` as a pointer file
|
||||
* (`gitdir: <path>`) in a linked worktree or submodule. Never throws; an unresolvable git
|
||||
* dir is a degraded-but-correct state, not an error.
|
||||
*
|
||||
* @param {string} cwd
|
||||
* @returns {{gitDir: string|null, source: string}}
|
||||
*/
|
||||
function resolveGitDir(cwd) {
|
||||
const unresolved = { gitDir: null, source: GITDIR_SOURCE.UNRESOLVED };
|
||||
const dotGit = path.join(cwd, '.git');
|
||||
|
||||
let stat;
|
||||
try {
|
||||
stat = fs.statSync(dotGit);
|
||||
} catch {
|
||||
return unresolved;
|
||||
}
|
||||
if (stat.isDirectory()) return { gitDir: dotGit, source: GITDIR_SOURCE.DIRECTORY };
|
||||
|
||||
let raw;
|
||||
try {
|
||||
raw = fs.readFileSync(dotGit, 'utf8');
|
||||
} catch {
|
||||
return unresolved;
|
||||
}
|
||||
|
||||
// Anchored /m with an explicit trailing-whitespace eat: `$` under /m sits before the
|
||||
// \n, so a CRLF checkout would otherwise carry the \r into the path.
|
||||
const match = /^gitdir:\s*(.*?)\s*$/m.exec(raw);
|
||||
if (!match || match[1] === '') return unresolved;
|
||||
return { gitDir: path.resolve(cwd, match[1]), source: GITDIR_SOURCE.GITFILE };
|
||||
}
|
||||
|
||||
/**
|
||||
* Read the pending-resolution marker, treating anything unusable as absent.
|
||||
*
|
||||
* Valid JSON is not the same as a usable marker: `0`, `"str"`, `[]`, `null` and `true` all
|
||||
* parse. So does an object whose `startedAt` or `count` is a string. Every one of those
|
||||
* means "no previous invocation I can trust" — reset, do not throw.
|
||||
*
|
||||
* @returns {{startedAt: number, count: number}|null}
|
||||
*/
|
||||
function readMarker(markerPath) {
|
||||
let parsed;
|
||||
try {
|
||||
parsed = JSON.parse(fs.readFileSync(markerPath, 'utf8'));
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
if (typeof parsed !== 'object' || parsed === null || Array.isArray(parsed)) return null;
|
||||
if (!Number.isFinite(parsed.startedAt)) return null;
|
||||
if (!Number.isFinite(parsed.count) || parsed.count < 0) return null;
|
||||
return { startedAt: parsed.startedAt, count: parsed.count };
|
||||
}
|
||||
|
||||
function decline(reason) {
|
||||
return {
|
||||
action: ACTION.DECLINE,
|
||||
reason,
|
||||
exitCode: 1,
|
||||
notice: false,
|
||||
pendingCount: 0,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide how to resolve one conflicted path, and record it.
|
||||
*
|
||||
* Deliberately NOT named `plan*` like `planInstall`: that prefix promises purity, and this
|
||||
* function reads and writes the marker file. The name says both halves out loud.
|
||||
*
|
||||
* @param {object} opts
|
||||
* @param {string[]} opts.argv exactly what git supplies: [%O, %A, %B, %L]. Any further
|
||||
* entry is ignored — see planInstall on why `%P` is deliberately not registered.
|
||||
* @param {string|null} opts.gitDir from resolveGitDir; null means marker-less (degraded)
|
||||
* @param {number} opts.now injected clock — never Date.now() inline, so tests are
|
||||
* deterministic and never assert on elapsed wall-clock
|
||||
* @returns {{action: string, reason: string, exitCode: number, notice: boolean,
|
||||
* pendingCount: number}}
|
||||
*/
|
||||
function resolveAndRecord({ argv, gitDir, now }) {
|
||||
if (!Array.isArray(argv) || argv.length < 3) return decline(REASON.FAIL_BAD_ARGV);
|
||||
|
||||
const oursPath = argv[1];
|
||||
if (typeof oursPath !== 'string' || oursPath.trim() === '') {
|
||||
return decline(REASON.FAIL_BAD_ARGV);
|
||||
}
|
||||
// %A is the one input the driver's contract depends on. If it is not there, we do not
|
||||
// know what "ours" is, so we hand the conflict back to git rather than inventing one.
|
||||
try {
|
||||
fs.statSync(oursPath);
|
||||
} catch {
|
||||
return decline(REASON.FAIL_OURS_UNREADABLE);
|
||||
}
|
||||
|
||||
const markerPath = gitDir ? path.join(gitDir, MARKER_NAME) : null;
|
||||
const previous = markerPath ? readMarker(markerPath) : null;
|
||||
const sameOperation = previous !== null && now - previous.startedAt <= NOTICE_WINDOW_MS;
|
||||
|
||||
const startedAt = sameOperation ? previous.startedAt : now;
|
||||
const pendingCount = sameOperation ? previous.count + 1 : 1;
|
||||
|
||||
if (markerPath) {
|
||||
// A diagnostic must never fail a merge: a read-only .git degrades to a repeated
|
||||
// notice, which is noisy but correct.
|
||||
try {
|
||||
fs.writeFileSync(markerPath, `${JSON.stringify({ startedAt, count: pendingCount })}\n`);
|
||||
} catch {
|
||||
/* degraded, not failed */
|
||||
}
|
||||
}
|
||||
|
||||
return {
|
||||
action: ACTION.ACCEPT_OURS,
|
||||
reason: REASON.OK_RESOLVED,
|
||||
exitCode: 0,
|
||||
notice: !sameOperation,
|
||||
pendingCount,
|
||||
};
|
||||
}
|
||||
|
||||
/**
|
||||
* The `.git/config` entries that register this driver.
|
||||
*
|
||||
* ## Why `%P` is NOT passed — do not "helpfully" add it back
|
||||
*
|
||||
* Git does **not** invoke a merge driver with an argv array. It substitutes `%O %A %B %L
|
||||
* %P` textually into this string and runs the whole thing through a shell. Quoting a
|
||||
* placeholder does not make it safe: inside POSIX double quotes `$(…)` and backticks still
|
||||
* execute, and a `"` in the value ends the quoting outright.
|
||||
*
|
||||
* `%O`, `%A` and `%B` are git-generated temp names (`.merge_file_XXXXXX`) and `%L` is an
|
||||
* integer, so none of them are attacker-controlled. **`%P` is the file's own path**, which
|
||||
* any contributor chooses freely. A branch that renames a covered fixture to
|
||||
* `evil$(touch PWNED).json` would execute that command on the machine of every maintainer
|
||||
* who merges it — silently, since the merge still reports success. Verified by reproduction
|
||||
* against a real `git merge`, not by inspection.
|
||||
*
|
||||
* So the driver takes no attacker-controlled argument at all, and records a count rather
|
||||
* than path names. A filter would have been a guess about shell grammar; passing nothing is
|
||||
* a property.
|
||||
*
|
||||
* Paths are normalized to forward slashes **unconditionally** — a backslash path can reach a
|
||||
* config value on any platform, and git shells this command everywhere including Windows.
|
||||
*
|
||||
* @param {{repoRoot: string}} opts
|
||||
* @returns {{entries: Array<{key: string, value: string}>}}
|
||||
*/
|
||||
function planInstall({ repoRoot }) {
|
||||
const script = path
|
||||
.join(repoRoot, 'scripts', 'git-merge-regen-driver.cjs')
|
||||
.replace(/\\/g, '/');
|
||||
return {
|
||||
entries: [
|
||||
{
|
||||
key: 'merge.gsd-regen.name',
|
||||
value: 'gsd-regen — keep ours for generated artifacts; regenerate with npm run regen:derived',
|
||||
},
|
||||
{
|
||||
key: 'merge.gsd-regen.driver',
|
||||
value: `node "${script}" "%O" "%A" "%B" "%L"`,
|
||||
},
|
||||
],
|
||||
};
|
||||
}
|
||||
|
||||
function gitConfig(args, cwd = REPO_ROOT) {
|
||||
const r = cp.spawnSync('git', ['config', ...args], {
|
||||
cwd,
|
||||
encoding: 'utf8',
|
||||
timeout: GIT_TIMEOUT_MS,
|
||||
});
|
||||
if (r.error && r.error.code === 'ETIMEDOUT') {
|
||||
throw new ExitError(1, `git config timed out after ${GIT_TIMEOUT_MS}ms`);
|
||||
}
|
||||
return r;
|
||||
}
|
||||
|
||||
function runInstall({ repoRoot = REPO_ROOT } = {}) {
|
||||
const { entries } = planInstall({ repoRoot });
|
||||
for (const { key, value } of entries) {
|
||||
const r = gitConfig([key, value], repoRoot);
|
||||
if (r.status !== 0) throw new ExitError(1, `git config ${key} failed: ${r.stderr}`);
|
||||
}
|
||||
process.stdout.write(
|
||||
'Registered the gsd-regen merge driver in this clone.\n' +
|
||||
'Conflicts on the generated parity manifests and size baselines now resolve to your\n' +
|
||||
'branch’s copy; run `npm run regen:derived` afterwards to recompute them.\n' +
|
||||
'This is a bridge for #2721 and is retired by #2724.\n',
|
||||
);
|
||||
return 0;
|
||||
}
|
||||
|
||||
function runUninstall({ repoRoot = REPO_ROOT } = {}) {
|
||||
for (const key of ['merge.gsd-regen.driver', 'merge.gsd-regen.name']) {
|
||||
// exit 5 == "was not set"; uninstalling something absent is success here
|
||||
gitConfig(['--unset-all', key], repoRoot);
|
||||
}
|
||||
process.stdout.write('Removed the gsd-regen merge driver from this clone.\n');
|
||||
return 0;
|
||||
}
|
||||
|
||||
/**
|
||||
* @returns {{registered: boolean, pendingCount: number}} the same object it prints, so
|
||||
* callers and tests consume the structure rather than re-parsing the rendered JSON.
|
||||
* A count, not path names: the driver is never handed the conflicted path, by design
|
||||
* (see planInstall).
|
||||
*/
|
||||
function statusOf({ repoRoot = REPO_ROOT } = {}) {
|
||||
const registered = gitConfig(['--get', 'merge.gsd-regen.driver'], repoRoot).status === 0;
|
||||
const { gitDir } = resolveGitDir(repoRoot);
|
||||
const marker = gitDir ? readMarker(path.join(gitDir, MARKER_NAME)) : null;
|
||||
return { registered, pendingCount: marker ? marker.count : 0 };
|
||||
}
|
||||
|
||||
function runStatus({ repoRoot = REPO_ROOT } = {}) {
|
||||
process.stdout.write(JSON.stringify(statusOf({ repoRoot }), null, 2) + '\n');
|
||||
return 0;
|
||||
}
|
||||
|
||||
function runDriver(argv) {
|
||||
const { gitDir } = resolveGitDir(process.cwd());
|
||||
const plan = resolveAndRecord({ argv, gitDir, now: Date.now() });
|
||||
|
||||
if (plan.action === ACTION.DECLINE) {
|
||||
process.stderr.write(
|
||||
`gsd-regen: declined (${plan.reason}) — leaving this path as a normal conflict.\n`,
|
||||
);
|
||||
return plan.exitCode;
|
||||
}
|
||||
if (plan.notice) {
|
||||
process.stderr.write(
|
||||
'gsd-regen: kept your branch’s copy of the generated parity/size artifacts.\n' +
|
||||
'gsd-regen: these files cannot be line-merged — their only correct value is recomputed.\n' +
|
||||
'gsd-regen: run `npm run regen:derived` before committing.\n' +
|
||||
'gsd-regen: (bridge for #2721; retired by #2724)\n',
|
||||
);
|
||||
}
|
||||
return plan.exitCode;
|
||||
}
|
||||
|
||||
function main() {
|
||||
const argv = process.argv.slice(2);
|
||||
const flags = argv.filter((a) => a.startsWith('--'));
|
||||
|
||||
if (flags.length > 0) {
|
||||
if (flags.length > 1 || argv.length > 1) {
|
||||
throw new ExitError(2, 'usage: git-merge-regen-driver.cjs [--install|--uninstall|--status]');
|
||||
}
|
||||
switch (flags[0]) {
|
||||
case '--install':
|
||||
return runInstall();
|
||||
case '--uninstall':
|
||||
return runUninstall();
|
||||
case '--status':
|
||||
return runStatus();
|
||||
default:
|
||||
throw new ExitError(
|
||||
2,
|
||||
`unknown flag ${flags[0]}\nusage: git-merge-regen-driver.cjs [--install|--uninstall|--status]`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
return runDriver(argv);
|
||||
}
|
||||
|
||||
module.exports = {
|
||||
ACTION,
|
||||
REASON,
|
||||
GITDIR_SOURCE,
|
||||
NOTICE_WINDOW_MS,
|
||||
MARKER_NAME,
|
||||
resolveGitDir,
|
||||
resolveAndRecord,
|
||||
planInstall,
|
||||
runInstall,
|
||||
runUninstall,
|
||||
runStatus,
|
||||
statusOf,
|
||||
};
|
||||
|
||||
if (require.main === module) runMain(main);
|
||||
@@ -19,11 +19,16 @@ const WORKFLOWS_DIR = path.join(__dirname, '..', 'gsd-core', 'workflows');
|
||||
/**
|
||||
* Byte size of a file, counted as on an LF (Unix) checkout.
|
||||
*
|
||||
* The size budget is calibrated against `wc -c` on a Unix (LF) checkout, but
|
||||
* these `.md` files have no `eol=lf` in `.gitattributes`, so Windows checks
|
||||
* them out as CRLF. Counting raw on-disk bytes there adds one byte per line,
|
||||
* a Windows-only false positive that diverges from the LF calibration basis
|
||||
* The size budget is calibrated against `wc -c` on a Unix (LF) checkout.
|
||||
* Counting raw on-disk bytes on a CRLF checkout adds one byte per line, a
|
||||
* Windows-only false positive that diverges from the LF calibration basis
|
||||
* (issue #683). Stripping CR yields the same LF byte count on every platform.
|
||||
*
|
||||
* `.gitattributes:2` (`* text=auto eol=lf`, added in #1088) now normalizes these
|
||||
* files to LF on checkout everywhere, so the CRLF case should not arise from a
|
||||
* normal clone — but this stays unconditional because it also covers a working
|
||||
* tree produced some other way (an unpacked archive, an editor that rewrites
|
||||
* line endings, a checkout predating that attribute).
|
||||
* This is still a raw byte count (not a trailing-newline-stripping line count).
|
||||
*
|
||||
* @param {string} filePath - Absolute or relative path to the file.
|
||||
|
||||
Reference in New Issue
Block a user