fix(#4112): portable grep in workflow markdown + submodule commit fix under diff.ignoreSubmodules (#4149)
* fix(#4112): ban GNU-only grep -P in workflow markdown to prevent macOS regressions Add scripts/lint-portable-grep.cjs, wire it into lint:ci, and add tests/lint-portable-grep.test.cjs. The #4112 shell-syntax fix newly exposed a pre-existing grep -oP invocation that silently resolves to "" on stock macOS's BSD grep (no -P support). This ratchet catches the same class of GNU-coreutils assumption before it merges, mirroring lint-portable-timeout.cjs. * fix(#4112): drop unneeded ls -l long-format that broke basename/dirname extraction The previous commit on this branch replaced grep -oP 'phases/\K[^/]+' (GNU-only, silently fails on macOS's BSD grep) with basename "$(dirname "$phase")"), but left ls -lt (long format, -l) in place. -l output is a full detail line (permissions, owner, size, date, path), not a bare path, so dirname on that string throws "illegal option -- r" (BSD) / errors under GNU coreutils too -- the extraction never produced a usable value, on any platform. -l was never needed here; only mtime-sort plus the bare path mattered. Dropping it to ls -t restores one-bare-path-per-line output, which basename/dirname actually requires. Confirmed on gsd-test's Linux bench: the #4112 regression test (tests/pause-work-context-detection.test.cjs) now passes phase, spike, and sketch resolution. Emitted-Drift-Ack-Growth: pause-work.md — portability fix (#4112): dropping grep -oP for a portable basename/dirname extraction is a few characters longer per line; no functional growth beyond the fix. Emitted-Drift-Ack-Growth: sync-skills.md — portability fix (#4112): replacing grep -oP '(?<=--from )\S+' with a portable sed -E capture-group equivalent (no PCRE lookbehind available) is a longer expression; growth is the direct cost of the fix, not new functionality. * fix(#3859): pin git commit itself against diff.ignoreSubmodules=all, not just the probe git 2.39.x (the exact version on the CI Linux bench) resolves a pathspec-scoped `git commit -- <path>` through the same diff.ignoreSubmodules-gated machinery as `git diff`, so a genuinely bumped submodule gitlink was silently dropped by the real commit even though the #3859 empty-diff probe correctly saw the change. The commit was misclassified as generic commit_failed because git's refusal text never says "nothing to commit". Prefix the pathspec-scoped commit invocation with -c diff.ignoreSubmodules=dirty, the same override the probe already carries, so the two can no longer disagree. Confirmed on git 2.50.1 (no-op) and git 2.39.5 via the actual gsd-tester-linux v1.8.0-node24 bench image (turns the silent refusal into a commit). * fix(#3859): carry the diff.ignoreSubmodules override via env, not argv The prior commit prepended `-c diff.ignoreSubmodules=dirty` to the real commit's argv, which shifted `commitArgs[0]` off `'commit'` and broke several pre-existing #3859 regression tests (tests/commit-files-pathspec.test.cjs) that assert on the raw argv captured at the execGit seam, e.g. `gitCalls.some((a) => a[0] === 'commit')`. Carry the same override via `GIT_CONFIG_COUNT` / `GIT_CONFIG_KEY_0` / `GIT_CONFIG_VALUE_0` env vars instead, which git honours identically and leaves argv untouched. Re-confirmed on the gsd-tester-linux v1.8.0-node24 bench (git 2.39.5): the submodule commit still succeeds. * test(#3859): pin the commitEnv GIT_CONFIG_* override to canScope directly The commitEnv/GIT_CONFIG_* override added in e935694fc/b3d37b929 was only exercised indirectly through pre-existing submodule integration tests. Add two dedicated regression tests that pin the canScope=false side of the scoping decision: a whole-index commit (no --files) and an --amend commit must both keep recording a bumped submodule gitlink under diff.ignoreSubmodules=all with no override applied, since neither shape carries a pathspec for that git internal check to consult. * fix(#4112): match grep invocations after then/do/else/elif shell keywords lint-portable-grep.cjs's GREP_INVOCATION_RE only anchored to line-start or right after `| & ; ( \` {`, so a grep call positioned right after `then`, `do`, `else`, or `elif` (e.g. `if x; then grep -oP '...'; fi`) was never flagged, letting the GNU-only grep -P defect this lint exists to catch reappear undetected in that shape. * fix(#3859): apply the diff.ignoreSubmodules override to every commit, not just scoped ones The commitEnv GIT_CONFIG_* override landed in e935694fc/b3d37b929 only when canScope was true, on the assumption that git 2.39.5's silent refusal of a bumped submodule gitlink under diff.ignoreSubmodules=all only affects a pathspec-limited `git commit -- <paths>`. Reproduced directly against the pinned CI tester image (ghcr.io/open-gsd/gsd-tester-linux:v1.8.0-node24, git 2.39.5): a bare whole-index `git commit -m ...` with no pathspec at all is refused identically when the only staged change is a submodule gitlink, since git's "nothing to commit" check is a real diff against HEAD that honours diff.ignoreSubmodules regardless of whether a pathspec narrows it. Apply the override unconditionally instead of gating it on canScope; it remains a confirmed no-op for --amend, which never hits this refusal at all. Caught by the new regression test added in 579ac9f8f, which failed against the real bench git version before this change. * fix(#3859): apply diff.ignoreSubmodules override to commit-to-subrepo and pr-subrepo cmdCommit already carries the GIT_CONFIG_* override that forces diff.ignoreSubmodules=dirty on the actual git commit call so a bumped submodule gitlink is not spuriously refused under git 2.39.5. The two sibling multi-repo commit paths, cmdCommitToSubrepo and cmdPrSubrepo, build the identical canScope-branched commit invocation but never carried the override, so the same refusal there surfaces as a generic commit_failed/error instead of a recorded gitlink. Apply the override unconditionally to both, matching the corrected cmdCommit shape. * fix(#3859): pin pr-subrepo's change detection against diff.ignoreSubmodules cmdPrSubrepo discovers what to commit via `git status --porcelain`, which (like the empty-diff probe fixed for cmdCommit) honors a local diff.ignoreSubmodules=all config. Under that config, a genuinely bumped submodule gitlink is invisible to the status scan, so changedFiles comes back empty and the function reports nothing_to_commit before ever reaching the commit call this same issue already fixed. Pin the status probe with --ignore-submodules=dirty, mirroring the flag cmdCommit's diff probe already uses, so a real gitlink bump is detected regardless of local config. Found while adding a regression test for the previous #3859 follow-up fix: the test failed not on the commit step but on this earlier detection step. * docs(#3859): update changeset to cover all three fixed commit sites * docs(#4112): backfill changeset PR number (pr:0 -> pr:4149) --------- Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
@@ -5737,8 +5737,180 @@ describe('#3859: the empty-diff probe is pinned against diff-only configuration'
|
||||
gitOrThrow(['show', 'HEAD:' + rel], { cwd: tmpDir }), 'B\n',
|
||||
'the new content must actually be recorded');
|
||||
});
|
||||
|
||||
// #3859 follow-up (e935694fc/b3d37b929, widened after a canScope gap found
|
||||
// reproducing live against the pinned CI tester image,
|
||||
// ghcr.io/open-gsd/gsd-tester-linux:v1.8.0-node24, which runs git 2.39.5):
|
||||
// the actual `git commit` call carries a `commitEnv` GIT_CONFIG_* override
|
||||
// forcing `diff.ignoreSubmodules=dirty`. This was originally scoped to only
|
||||
// fire when `canScope` was true (a pathspec-limited `git commit --
|
||||
// <paths>`), on the assumption that only a pathspec-limited commit
|
||||
// consults `diff.ignoreSubmodules` when deciding whether a bumped
|
||||
// submodule gitlink is a real change to record. That assumption was wrong:
|
||||
// on git 2.39.5 a bare WHOLE-INDEX `git commit -m ...` (no pathspec at
|
||||
// all, canScope=false) is refused identically when the only staged change
|
||||
// is a submodule gitlink under `diff.ignoreSubmodules=all` — git's
|
||||
// "nothing to commit" check is a real diff (HEAD vs. index) that honours
|
||||
// `diff.ignoreSubmodules` regardless of pathspec. The override is now
|
||||
// applied unconditionally (no `canScope` gate) to cover this shape too.
|
||||
// `--amend` is the one shape confirmed NOT to hit the refusal at all
|
||||
// (reproduced directly: it succeeds identically with or without the
|
||||
// override, since amend never runs the empty-diff check a plain `git
|
||||
// commit` does) — its test below pins that the override being applied
|
||||
// unconditionally is still harmless there.
|
||||
test('a whole-index commit (no --files, canScope=false) still records a bumped submodule under diff.ignoreSubmodules=all', () => {
|
||||
bumpedSubmodule();
|
||||
gitOrThrow(['config', 'diff.ignoreSubmodules', 'all'], { cwd: tmpDir });
|
||||
gitOrThrow(['add', '--', 'sub'], { cwd: tmpDir });
|
||||
const before = gitOrThrow(['rev-parse', 'HEAD:sub'], { cwd: tmpDir }).trim();
|
||||
|
||||
const result = runGsdTools('commit "m"', tmpDir);
|
||||
const payload = (result.output && result.output.trim()) ? result.output : result.error;
|
||||
const output = JSON.parse(payload);
|
||||
|
||||
assert.strictEqual(output.committed, true,
|
||||
'a whole-index commit hits the same git 2.39.5 refusal as a pathspec-limited one, so it needs the ' +
|
||||
'GIT_CONFIG_* override applied unconditionally, not gated on canScope, to record the bumped gitlink');
|
||||
assert.notStrictEqual(
|
||||
gitOrThrow(['rev-parse', 'HEAD:sub'], { cwd: tmpDir }).trim(), before,
|
||||
'the recorded gitlink must actually advance');
|
||||
});
|
||||
|
||||
test('an --amend commit (canScope=false) still records a bumped submodule under diff.ignoreSubmodules=all', () => {
|
||||
bumpedSubmodule();
|
||||
gitOrThrow(['config', 'diff.ignoreSubmodules', 'all'], { cwd: tmpDir });
|
||||
gitOrThrow(['add', '--', 'sub'], { cwd: tmpDir });
|
||||
const before = gitOrThrow(['rev-parse', 'HEAD:sub'], { cwd: tmpDir }).trim();
|
||||
|
||||
const result = runGsdTools('commit "m" --amend', tmpDir);
|
||||
const payload = (result.output && result.output.trim()) ? result.output : result.error;
|
||||
const output = JSON.parse(payload);
|
||||
|
||||
assert.strictEqual(output.committed, true,
|
||||
'--amend never hits the empty-diff refusal, so the now-unconditional GIT_CONFIG_* override must remain ' +
|
||||
'a harmless no-op here');
|
||||
assert.notStrictEqual(
|
||||
gitOrThrow(['rev-parse', 'HEAD:sub'], { cwd: tmpDir }).trim(), before,
|
||||
'the recorded gitlink must actually advance');
|
||||
});
|
||||
});
|
||||
|
||||
// #3859 follow-up: `cmdCommitToSubrepo` and `cmdPrSubrepo` carry the identical
|
||||
// structurally-shaped `canScope*`-branched `git commit` call as `cmdCommit`
|
||||
// above (see `COMMIT_TIMEOUT_MS`'s "three commit sites" comment in
|
||||
// src/commands.cts) and were missing the same `diff.ignoreSubmodules=dirty`
|
||||
// GIT_CONFIG_* override, applied unconditionally for the same reason.
|
||||
describe('#3859 follow-up: commit-to-subrepo and pr-subrepo also need the diff.ignoreSubmodules override', () => {
|
||||
const { createTempGitProject } = require('./helpers.cjs');
|
||||
let rootDir;
|
||||
let nestedSubmoduleSrc;
|
||||
|
||||
afterEach(() => {
|
||||
if (rootDir) cleanup(rootDir);
|
||||
if (nestedSubmoduleSrc) cleanup(nestedSubmoduleSrc);
|
||||
rootDir = undefined;
|
||||
nestedSubmoduleSrc = undefined;
|
||||
});
|
||||
|
||||
// A submodule nested inside `repoDir` whose recorded gitlink is AHEAD of
|
||||
// what `repoDir` has committed — same shape as `bumpedSubmodule()` above,
|
||||
// scoped to an arbitrary sub-repo directory instead of the project root.
|
||||
function bumpedSubmoduleIn(repoDir) {
|
||||
const subSrc = path.join(repoDir, '..', path.basename(repoDir) + '-nested-sub');
|
||||
nestedSubmoduleSrc = subSrc;
|
||||
fs.mkdirSync(subSrc, { recursive: true });
|
||||
gitOrThrow(['init', '-q', '.'], { cwd: subSrc });
|
||||
gitOrThrow(['config', 'user.email', 't@t'], { cwd: subSrc });
|
||||
gitOrThrow(['config', 'user.name', 't'], { cwd: subSrc });
|
||||
fs.writeFileSync(path.join(subSrc, 'f.txt'), 'v1\n');
|
||||
gitOrThrow(['add', 'f.txt'], { cwd: subSrc });
|
||||
gitOrThrow(['commit', '-m', 'v1'], { cwd: subSrc });
|
||||
|
||||
gitOrThrow(['-c', 'protocol.file.allow=always', 'submodule', 'add', '-q', subSrc, 'nested'], { cwd: repoDir });
|
||||
gitOrThrow(['commit', '-m', 'add nested submodule'], { cwd: repoDir });
|
||||
|
||||
fs.writeFileSync(path.join(subSrc, 'f.txt'), 'v2\n');
|
||||
gitOrThrow(['add', 'f.txt'], { cwd: subSrc });
|
||||
gitOrThrow(['commit', '-m', 'v2'], { cwd: subSrc });
|
||||
gitOrThrow(['-c', 'protocol.file.allow=always', 'submodule', 'update', '--remote', '--', 'nested'], { cwd: repoDir });
|
||||
}
|
||||
|
||||
test('commit-to-subrepo records a bumped nested submodule under diff.ignoreSubmodules=all', () => {
|
||||
rootDir = createTempGitProject();
|
||||
fs.writeFileSync(
|
||||
path.join(rootDir, '.planning', 'config.json'),
|
||||
JSON.stringify({ planning: { sub_repos: ['backend'] } }, null, 2),
|
||||
);
|
||||
const subDir = path.join(rootDir, 'backend');
|
||||
fs.mkdirSync(subDir, { recursive: true });
|
||||
gitOrThrow(['init', '-q', '.'], { cwd: subDir });
|
||||
gitOrThrow(['config', 'user.email', 't@t'], { cwd: subDir });
|
||||
gitOrThrow(['config', 'user.name', 't'], { cwd: subDir });
|
||||
fs.writeFileSync(path.join(subDir, 'seed.js'), '// seed\n');
|
||||
gitOrThrow(['add', 'seed.js'], { cwd: subDir });
|
||||
gitOrThrow(['commit', '-m', 'seed'], { cwd: subDir });
|
||||
|
||||
bumpedSubmoduleIn(subDir);
|
||||
gitOrThrow(['config', 'diff.ignoreSubmodules', 'all'], { cwd: subDir });
|
||||
const before = gitOrThrow(['rev-parse', 'HEAD:nested'], { cwd: subDir }).trim();
|
||||
|
||||
const res = runGsdTools(
|
||||
['commit-to-subrepo', 'chore: bump nested submodule', '--files', 'backend/nested'],
|
||||
rootDir,
|
||||
);
|
||||
assert.ok(res.success, `commit-to-subrepo failed: ${res.error}`);
|
||||
const result = JSON.parse(res.output);
|
||||
|
||||
assert.strictEqual(result.repos.backend.committed, true,
|
||||
`the gitlink moved and \`git commit -- nested\` records it on git 2.39.5 only with the ` +
|
||||
`GIT_CONFIG_* override applied, got ${JSON.stringify(result.repos.backend)}`);
|
||||
assert.notStrictEqual(result.repos.backend.reason, 'error');
|
||||
assert.notStrictEqual(
|
||||
gitOrThrow(['rev-parse', 'HEAD:nested'], { cwd: subDir }).trim(), before,
|
||||
'the recorded gitlink must actually advance');
|
||||
});
|
||||
|
||||
test('pr-subrepo records a bumped nested submodule under diff.ignoreSubmodules=all', () => {
|
||||
rootDir = createTempGitProject();
|
||||
fs.writeFileSync(
|
||||
path.join(rootDir, '.planning', 'config.json'),
|
||||
JSON.stringify({ planning: { sub_repos: ['backend'] } }, null, 2),
|
||||
);
|
||||
const subDir = path.join(rootDir, 'backend');
|
||||
const bareDir = path.join(rootDir, '_bare-backend.git');
|
||||
fs.mkdirSync(subDir, { recursive: true });
|
||||
gitOrThrow(['init', '-q', '.'], { cwd: subDir });
|
||||
gitOrThrow(['config', 'user.email', 't@t'], { cwd: subDir });
|
||||
gitOrThrow(['config', 'user.name', 't'], { cwd: subDir });
|
||||
fs.writeFileSync(path.join(subDir, 'seed.js'), '// seed\n');
|
||||
gitOrThrow(['add', 'seed.js'], { cwd: subDir });
|
||||
gitOrThrow(['commit', '-m', 'seed'], { cwd: subDir });
|
||||
fs.mkdirSync(bareDir, { recursive: true });
|
||||
gitOrThrow(['init', '--bare', '-q'], { cwd: bareDir });
|
||||
gitOrThrow(['remote', 'add', 'origin', bareDir], { cwd: subDir });
|
||||
const branch = gitOrThrow(['branch', '--show-current'], { cwd: subDir }).trim();
|
||||
gitOrThrow(['push', 'origin', branch], { cwd: subDir });
|
||||
|
||||
bumpedSubmoduleIn(subDir);
|
||||
gitOrThrow(['config', 'diff.ignoreSubmodules', 'all'], { cwd: subDir });
|
||||
const before = gitOrThrow(['rev-parse', 'HEAD:nested'], { cwd: subDir }).trim();
|
||||
|
||||
const res = runGsdTools(
|
||||
['query', 'pr-subrepo', 'fix(backend): bump nested submodule',
|
||||
'--repo', 'backend', '--branch', 'fix-3859-nested-submodule-pr'],
|
||||
rootDir,
|
||||
);
|
||||
assert.ok(res.success, `pr-subrepo failed: ${res.error}`);
|
||||
const result = JSON.parse(res.output);
|
||||
|
||||
assert.strictEqual(result.committed, true,
|
||||
`the gitlink moved and \`git commit -- nested\` records it on git 2.39.5 only with the ` +
|
||||
`GIT_CONFIG_* override applied, got ${JSON.stringify(result)}`);
|
||||
assert.notStrictEqual(
|
||||
gitOrThrow(['rev-parse', 'HEAD:nested'], { cwd: subDir }).trim(), before,
|
||||
'the recorded gitlink must actually advance');
|
||||
});
|
||||
});
|
||||
|
||||
describe('#2279: map-codebase date stamp instructions overwrite existing dates', () => {
|
||||
const REPO_ROOT = path.join(__dirname, '..');
|
||||
|
||||
95
tests/lint-portable-grep.test.cjs
Normal file
95
tests/lint-portable-grep.test.cjs
Normal file
@@ -0,0 +1,95 @@
|
||||
'use strict';
|
||||
|
||||
/**
|
||||
* Tests for scripts/lint-portable-grep.cjs — the ratchet that bans GNU-only
|
||||
* `grep -P`/`--perl-regexp` in gsd workflow / agent / reference / command
|
||||
* markdown (#4112 macOS regression: a `grep -oP` left behind after the
|
||||
* `pause-work.md` `$((` shell-syntax fix silently resolved phase/spike/sketch
|
||||
* detection to "" on stock macOS's BSD grep).
|
||||
*
|
||||
* Tests the PURE check logic (findPerlGrepInvocations) directly, on in-memory
|
||||
* string fixtures, so the suite is fast, hermetic, and never reads real repo
|
||||
* files (that integration concern is already covered by `npm run lint:ci`).
|
||||
*/
|
||||
|
||||
const { describe, test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
|
||||
const { findPerlGrepInvocations } = require('../scripts/lint-portable-grep.cjs');
|
||||
|
||||
describe('lint-portable-grep: findPerlGrepInvocations pure logic', () => {
|
||||
test('flags `grep -oP` with a lookahead pattern', () => {
|
||||
const findings = findPerlGrepInvocations(String.raw`grep -oP 'x\K[^/]+'`);
|
||||
assert.strictEqual(findings.length, 1);
|
||||
assert.strictEqual(findings[0].line, 1);
|
||||
});
|
||||
|
||||
test('flags bare `grep -P`', () => {
|
||||
const findings = findPerlGrepInvocations(`grep -P 'x'`);
|
||||
assert.strictEqual(findings.length, 1);
|
||||
});
|
||||
|
||||
test('flags `grep --perl-regexp` long form', () => {
|
||||
const findings = findPerlGrepInvocations(`grep --perl-regexp 'x'`);
|
||||
assert.strictEqual(findings.length, 1);
|
||||
});
|
||||
|
||||
test('flags `egrep -P`', () => {
|
||||
const findings = findPerlGrepInvocations(`egrep -P 'x'`);
|
||||
assert.strictEqual(findings.length, 1);
|
||||
});
|
||||
|
||||
test('flags `fgrep -P`', () => {
|
||||
const findings = findPerlGrepInvocations(`fgrep -P 'x'`);
|
||||
assert.strictEqual(findings.length, 1);
|
||||
});
|
||||
|
||||
test('passes `grep -o` (no P flag)', () => {
|
||||
const findings = findPerlGrepInvocations(`grep -o 'x'`);
|
||||
assert.strictEqual(findings.length, 0);
|
||||
});
|
||||
|
||||
test('passes `grep -E` (POSIX ERE, no perl flag)', () => {
|
||||
const findings = findPerlGrepInvocations(`grep -E 'x'`);
|
||||
assert.strictEqual(findings.length, 0);
|
||||
});
|
||||
|
||||
test('passes a line containing `--path` (lowercase p, not a -P cluster)', () => {
|
||||
const findings = findPerlGrepInvocations(`some-cmd --path /foo | grep -o 'x'`);
|
||||
assert.strictEqual(findings.length, 0);
|
||||
});
|
||||
|
||||
test('segment-scoping: an unrelated earlier -P-bearing command does not taint a later plain grep', () => {
|
||||
const findings = findPerlGrepInvocations(`foo -P | grep -o 'x'`);
|
||||
assert.strictEqual(findings.length, 0, 'the -P belongs to `foo`, not to the grep invocation after the pipe');
|
||||
});
|
||||
|
||||
test('multi-line input: finding reports the correct 1-indexed line number', () => {
|
||||
const text = ['line one is clean', 'line two is also clean', `grep -oP 'x\\K.*'`, 'line four is clean'].join(
|
||||
'\n',
|
||||
);
|
||||
const findings = findPerlGrepInvocations(text);
|
||||
assert.strictEqual(findings.length, 1);
|
||||
assert.strictEqual(findings[0].line, 3);
|
||||
});
|
||||
|
||||
test('empty string input produces no findings', () => {
|
||||
const findings = findPerlGrepInvocations('');
|
||||
assert.strictEqual(findings.length, 0);
|
||||
});
|
||||
|
||||
test('flags `grep -oP` right after a `then` shell keyword', () => {
|
||||
const findings = findPerlGrepInvocations(`if [ -n "$x" ]; then grep -oP 'x' ; fi`);
|
||||
assert.strictEqual(findings.length, 1);
|
||||
});
|
||||
|
||||
test('flags `grep -oP` right after a `do` shell keyword', () => {
|
||||
const findings = findPerlGrepInvocations(`for f in *.md; do grep -oP 'x' "$f"; done`);
|
||||
assert.strictEqual(findings.length, 1);
|
||||
});
|
||||
|
||||
test('a bare mention of the word "then" with no following grep is not flagged', () => {
|
||||
const findings = findPerlGrepInvocations(`this sentence mentions the word then but nothing else`);
|
||||
assert.strictEqual(findings.length, 0);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user