golem15/msd-core
3
0
Fork 0
Code Issues Pull Requests Actions 4 Packages Projects Releases Wiki Activity
3,057 Commits 3 Branches 1 Tag
3002bd73e0cb29ea36dee6ddb49d4e2cce0fbb8b
Commit Graph

7 Commits

Author SHA1 Message Date
Tom Boucher
7bd77d6268 fix(116): locale-safe base64-scan with portable timeout and partial-scan signaling (#132) 
* test(116): reproduce base64-scan illegal byte sequence on non-UTF8 fixtures

Adds regression fixtures and failing tests for #116. Empirically verified
on macOS 26.5 (BSD tr) that `tr -cd '[:print:]'` under LC_CTYPE=en_US.UTF-8
exits non-zero with "Illegal byte sequence" when its input contains bytes
that are not valid UTF-8 start sequences (e.g. lone continuation bytes 0x80–0x9F).

The base64-scan.sh root cause is a known bash pitfall: the assignment
  `local printable_count=$(... | tr -cd '[:print:]' | ...)`
uses `local` on the same line, which always returns exit 0, masking the tr
failure. Result: tr errors surface only on stderr; the scan exits 0 with
incomplete coverage (false-clean signal).

Two new tests FAIL on origin/main:
  - "scans non-UTF8 file containing a b64 blob without emitting Illegal byte sequence"
  - "dir scan with non-UTF8 files under non-C locale completes cleanly within 30s"

Fixtures in tests/fixtures/base64-locale/:
  utf8-with-injection.md       — UTF-8 + base64-encoded injection (positive control)
  non-utf8-with-b64blob.bin    — raw 0x80-0x9F bytes + b64 blob that decodes to
                                 binary (this is the reproducer that triggers tr error)
  mixed-encoding.txt           — valid UTF-8 + lone continuation bytes
  clean-text.md                — negative control (must not be flagged)

Test helpers use spawnSync (not execFileSync) so stderr is captured even on
exit 0 — execFileSync only surfaces stderr via the thrown error on non-zero exit.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(116): locale-safe base64-scan with portable timeout and partial-scan signaling

Root cause: BSD tr(1) on macOS rejects input bytes that are not valid UTF-8
start sequences with "Illegal byte sequence" when LC_CTYPE is set to any
UTF-8 locale (e.g. en_US.UTF-8). Empirically verified on macOS 26.5 using
`man tr` (ENVIRONMENT section) and direct testing:
  printf '\x80\x81hello' | LC_ALL=en_US.UTF-8 tr -cd '[:print:]'
  → tr: Illegal byte sequence (exit 1)

The error is silently masked because base64-scan.sh uses `local` on the same
line as the tr assignment. Bash's `local` built-in always returns 0 regardless
of the subshell's exit code — so the tr failure never propagates under
`set -euo pipefail`. Result: the script exits 0 with truncated printable_count,
causing binary-decoded blobs to be skipped (false-clean, security gap).

Fix: `export LC_ALL=C` at script level (line 33).
  - LC_ALL=C forces the POSIX C locale throughout: tr treats every byte 0x00–0xFF
    as a valid character, never rejects high bytes.
  - Safe for all script operations: all injection patterns are ASCII, grep POSIX
    classes ([:space:], [:print:]) behave correctly in C locale, base64 -d is
    locale-independent.
  - Script-level export is appropriate because all operations in this script are
    byte-level; no multi-byte character handling is needed.

Additional hardening:
  - MAX_LINE_BYTES=1048576 guard in extract_and_check_blobs: lines longer than
    1 MiB are skipped with an explicit "partial scan" warning to stderr. This
    bounds grep -oE cost on pathological inputs (minified JS, single-line binary
    blobs) and satisfies the "partial-scan failure signaling" requirement.
  - Portable run_with_timeout + is_timeout_exit: probes for GNU timeout,
    gtimeout (homebrew), and falls back to perl alarm(N)+exec. Defined for
    future use guarding external sub-commands. Verified: no timeout binary on
    this macOS host, perl alarm fallback works correctly (exit 142 on SIGALRM).

Test-rigor fixes applied per test-rigor skill review:
  - Fixture validity check: assert `isInvalidUtf8` (round-trip length difference)
    rather than checking for a specific byte range — the property that matters is
    "file is not valid UTF-8", not "file has bytes in 0x80–0x9F".
  - FAIL assertions: assert `FAIL: ${INJECTION_FIXTURE}` (specific filepath) not
    `result.stdout.includes('FAIL')` — rules out false-positives on other fixtures.
  - Test name: renamed "mixed-encoding file does not cause scan to abort or hang"
    to accurately describe what is tested (no extractable blobs → exits clean).

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(116): fix shellcheck warnings in base64-scan.sh

Address SC2034 (unused variables) and SC2329 (functions never invoked)
warnings flagged by shellcheck after the locale-hardening changes.

SC2034 fixes (pre-existing):
  - Remove unused SCRIPT_DIR variable (set but never referenced)
  - Remove unused printable_ratio local (declared but no assignment or use)

SC2329 fixes (new functions from this PR):
  - Add shellcheck disable=SC2329 annotations on run_with_timeout,
    _init_timeout_cmd, and is_timeout_exit — these are intentionally
    defined as infrastructure helpers, not called from the main loop.
    The line-length guard (MAX_LINE_BYTES) is the primary runtime
    protection; the timeout helpers are available for future use.

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

* fix(#116): exclude scanner fixtures from base64-scan diff mode

Add tests/fixtures/* to should_skip_file() so deliberate prompt-injection
samples in scanner fixture directories are never flagged in CI diff-mode.

Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
 2026-05-23 10:14:48 -04:00
Tom Boucher
02d3cf3033 fix(3597): add Windows-safe rmSync retry budgets in lint/security tests 2026-05-16 13:33:24 -04:00
Tom Boucher
f55069ecbf test(#2974): migrate 8 test files to typed-IR assertions (#3016) 
* test(#2974): migrate 8 test files to typed-IR assertions

Replaces raw stdout/stderr substring matching with structured-field
assertions per CONTRIBUTING.md "Prohibited: Raw Text Matching on Test
Outputs". Adds shared infrastructure for typed error emission so this
pattern is the easy path going forward.

Shared infrastructure:
- core.cjs: ERROR_REASON frozen enum + setJsonErrorMode/getJsonErrorMode
- gsd-tools.cjs: --json-errors CLI flag, parsed before subcommand dispatch
- config.cjs: typed reasons at all 7 error sites
- graphify.cjs: GRAPHIFY_REASON enum + reason/timeout_ms in execGraphify result
- bin/install.js: pure buildSdkFailFastReport() IR builder + renderer
- hooks/gsd-session-state.sh, gsd-phase-boundary.sh: emit Claude Code
  hookSpecificOutput JSON envelope with typed state_present/config_mode/
  planning_modified/file_path fields (no-op when hooks.community is off)

Test migrations (all pass, 171 tests across the 8 files):
- bug-2649-sdk-fail-fast: assert on ir.reason / ir.context / ir.fix_command
- bug-2687-config-read-warning-parity: assert.equal stderr === ''
- bug-2796-arg-parsing-regression: assert on result.json.updated/.phase
- bug-2838-summary-rescue: parse rescue footer, assert mtime invariant
- bug-2943-config-get-context-window: parse JSON, assert ERROR_REASON.CONFIG_KEY_NOT_FOUND
- graphify: assert reason === GRAPHIFY_REASON.ENOENT/TIMEOUT
- hooks-opt-in: parse hookSpecificOutput, assert typed fields
- security-scan: reclassified as source-text-is-the-product (scan label
  output and CI workflow YAML ARE the deployed contract)

Verification: lint-no-source-grep clean (0 violations), full suite
6741/6741 pass.

Closes #2974

* test(#2974): address CR feedback — typed code field, robust idempotency

Two CodeRabbit findings on #3016 addressed:

1. tests/hooks-opt-in.test.cjs:355 (Minor, inline) —
   parsed.reason.includes('Conventional Commits') was still substring
   matching after the typed-IR migration. Fixed at the source: the
   gsd-validate-commit hook now emits a typed `code` field
   ('CONVENTIONAL_COMMITS_VIOLATION', 'COMMIT_SUBJECT_TOO_LONG')
   alongside the human-readable `reason`. Test asserts strictEqual
   on the code; the prose copy is no longer part of the test contract.

2. tests/bug-2838-summary-rescue-gitignored-planning.test.cjs:224-250
   (Outside-diff) — mtimeMs alone can stay unchanged on coarse-grained
   filesystems (HFS+, FAT) when two rewrites land within the same
   timestamp tick, falsely passing the idempotency assertion.
   Replaced with a full snapshot (mtimeMs, ctimeMs, size, ino, sha256
   of contents) compared via assert.deepStrictEqual — the hash
   catches any rewrite the timestamp would miss.

Verification: 30/30 pass on the two affected files; lint-no-source-grep
clean (0 violations across 368 test files).
 2026-05-02 09:27:23 -04:00
Tom Boucher
ef43f5161f fix(#2969): deterministic Step 5 verification gate for /gsd-reapply-patches (#2972) 
* fix(#2969): deterministic Step 5 verification gate for /gsd-reapply-patches

The prior Step 5 "Hunk Verification Gate" was prescribed correctly in the
workflow text — but executed laxly by the LLM, which filled in `verified: yes`
without actually checking content presence. The reporter observed three
distinct files (skills/gsd-discuss-phase/SKILL.md, skills/gsd-autonomous/
SKILL.md, get-shit-done/workflows/new-project.md) where archives contained
substantive user-added blocks that did not survive into the merged result, yet
the gate reported clean.

Move verification from LLM-driven prose into a deterministic Node script the
workflow calls. The script can't be shortcut.

Changes:

- scripts/verify-reapply-patches.cjs (new): pure Node, no external deps.
  For each file in the patches dir, computes user-added significant lines as
  the line-set diff between backup and pristine baseline (when available;
  falls back to "every significant backup line" when no pristine — over-broad
  but the safe direction for this bug class). Asserts each line appears
  literally in the merged installed file via String.prototype.includes.
  Filters trivial lines (length < 12 chars, pure punctuation, decorative
  comments) so harmless drift doesn't trigger false failures. Exits 0 on
  pass, 1 on any miss with per-file diagnostic, 2 on usage error.
  Supports --json for workflow consumption.

- get-shit-done/workflows/reapply-patches.md: rewrite Step 5 to call the
  script and parse its JSON output. The Step 4 Hunk Verification Table
  remains as advisory Claude-readable summary, but the gate is now the
  script's exit code.

- tests/bug-2969-verify-reapply-patches.test.cjs (new): 6 tests covering
  (a) pass when every line survives, (b) fail when a line is missing,
  (c) fail when the merged file is deleted entirely, (d) --json structured
  report shape, (e) backup-meta.json is correctly skipped as metadata,
  (f) no-pristine-dir fallback exercises the safe over-broad path. All pass.

Out of scope: the manifest-baseline tightening described in #2969 Failure 1
(saveLocalPatches comparing against the wrong baseline so prior silent wipes
poison subsequent updates). That's a separate, bigger architectural change
involving pristine-content infrastructure; this PR addresses the gate fidelity
half so users at least see the diagnostic when content goes missing.

Closes #2969 (partial — Failure 2 only)

* fix(#2969): preserve #1999 Hunk Verification Table assertions alongside new script gate

CI failure on PR #2972 surfaced that tests/reapply-patches.test.cjs (the
#1999 contract) asserts Step 5 references:
  - "Hunk Verification Table"
  - `verified: no` failure condition
  - explicit STOP/halt/abort directive
  - "table absent / missing" halt path

My initial Step 5 rewrite for #2969 substituted the deterministic script
for the table-based gate entirely, stripping those references. The script
is the strictly stronger gate, but the existing #1999 test enforces the
table-based safety net as a defense-in-depth contract.

Restore both gates as a layered Step 5:

  - 5a (binding): deterministic verifier script — script gate, exits
    non-zero on any miss, cannot be shortcut by the LLM
  - 5b (advisory): Hunk Verification Table review — preserved as
    redundant safety net for the case where the script has a bug or the
    pristine baseline is unavailable

Both gates must pass. Verified: tests/reapply-patches.test.cjs (5 tests
in the #1999 suite) and tests/bug-2969-verify-reapply-patches.test.cjs
(6 tests in the #2969 suite) all pass — 21/21 total in this fixture.

* fix(#2969): address CodeRabbit findings on workflow + script

Five CR findings on PR #2972, all valid; addressed in this commit:

1. (Major) Stderr was merged into VERIFY_OUTPUT via `2>&1`, so any Node
   warning, deprecation notice, or stack trace would corrupt the JSON
   parse downstream. Capture stdout only; stderr remains on the
   controlling terminal for operator visibility.

2. (Major) verifyFile() crashed with EISDIR/EACCES instead of producing
   a structured diagnostic when the installed path was a directory or
   unreadable. Wrap statSync/readFileSync in try/catch and emit a
   per-file fail row; the whole-run gate continues with structured
   output. Added test case asserting the directory-at-installed-path
   case fails with `not a regular file` diagnostic instead of crashing.

3. (Minor) PRISTINE_FLAG built as a single string + unquoted expansion
   would split paths with spaces. Switched to a bash array (VERIFY_ARGS)
   that preserves whitespace through expansion.

4. (Minor) Fenced code block missing language tag (markdownlint MD040).
   Added `text` tag to the error message block.

5. (Minor) Usage comment said pristine fallback was "backup-meta lookup"
   but the actual code path falls back to significant-line checks from
   backup content. Corrected the comment to match implementation.

Verified all 21 tests in tests/reapply-patches.test.cjs (#1999 contract)
+ tests/bug-2969-verify-reapply-patches.test.cjs (now 7 tests with the
new directory case) pass.

* test(#2969): structured JSON assertions, no substring matching on script output

Replace every assert.match(r.stdout, /pattern/) call with structured
assertions on the parsed JSON report from the script's own --json mode.
The script's --json contract IS the structured shape we test against —
the test author should never depend on the human-readable formatter
output, just as no test should depend on substring presence in source.

Changes:

  - All 7 tests now run the verifier with --json (via a runVerifier()
    helper) and parse the resulting JSON document into { status, report,
    stderr }. Diagnostic stderr is preserved as a separate channel for
    debug output but is not used for assertions.
  - Each previously substring-matched diagnostic ("Failures: 1",
    "not a regular file", "installed file missing after merge",
    file path, dropped line) is now a deepEqual / equal / Array.includes
    against typed report fields: report.failures, report.results[i].status,
    report.results[i].reason, report.results[i].file,
    report.results[i].missing[].
  - Added an explicit "documented shape" test asserting the JSON output
    has exactly the keys { file, missing, reason, status } per result —
    locks the public contract of the --json mode.
  - DRY'd up fixture reset into a resetFixture() helper since every test
    starts with a fresh patches/installed/pristine triple.

Linter: scripts/lint-no-source-grep.cjs reports 0 violations across 348
test files. Combined run of bug-2969-...test.cjs (7 tests) +
reapply-patches.test.cjs (5 tests in the #1999 suite) all pass —
22/22 in the relevant fixture.

* fix(#2969): typed REASON enum + raw-text-matching rule shipped repo-wide

This commit closes the loop on the no-source-grep discipline:

1. scripts/verify-reapply-patches.cjs:
   - Frozen REASON enum exposes the diagnostic surface as stable codes:
     OK_NO_USER_LINES_VS_PRISTINE, OK_NO_SIGNIFICANT_BACKUP_LINES,
     FAIL_INSTALLED_MISSING, FAIL_INSTALLED_NOT_REGULAR_FILE,
     FAIL_READ_ERROR, FAIL_USER_LINES_MISSING.
   - Each result.reason is now a code from this enum, not free text.
     Tests assert via REASON.X equality, not regex on prose.
   - REASON exported from module.exports.

2. tests/bug-2969-verify-reapply-patches.test.cjs:
   - Full rewrite. Every assertion on typed structured fields:
     report.results[0].status === 'fail',
     report.results[0].reason === REASON.FAIL_INSTALLED_NOT_REGULAR_FILE,
     report.results[0].missing.includes(droppedLine) (Array set membership,
     not String substring).
   - Locks the REASON enum surface via Object.keys(REASON).sort() deepEqual.
   - Locks the JSON report shape via Object.keys(report).sort() deepEqual.
   - Zero regex, zero String#includes, zero startsWith/endsWith on text.

3. CONTRIBUTING.md:
   - New section "Prohibited: Raw Text Matching on Test Outputs" with
     concrete BAD/GOOD examples (substring on file content; assert.match
     on stdout; "structured parser" hiding string ops; regex on free-form
     reason fields).
   - The rule statement: "Tests assert on typed structured values. If
     the code under test produces text, the code under test must also
     expose a structured intermediate representation, and the test must
     assert on that IR — never on the rendered text."
   - Required structured-surface table: file IR, --json mode, frozen
     enum, fs facts.
   - "Hiding grep behind a function is still grep" callout — the
     parser-wrapper anti-pattern.
   - New `pre-existing-text-matching` exemption category for the 8
     grandfathered files. Marked Transitional; new tests cannot use it.

4. scripts/lint-no-source-grep.cjs:
   - Three new patterns enforced (in addition to the existing .cjs-source
     readFileSync rule):
     - assert.match/doesNotMatch on .stdout/.stderr
     - .stdout/.stderr.<includes|startsWith|endsWith>(
     - readFileSync(...).<includes|startsWith|endsWith>(
   - Aggregated violations per file (multiple findings now report together).
   - Updated diagnostic message references both CONTRIBUTING.md sections.

5. 8 pre-existing tests annotated with `// allow-test-rule:
   pre-existing-text-matching` so the lint passes on this commit; each
   carries the prose "Tracked for migration to typed-IR assertions; do
   not copy this pattern." Files: bug-2649, bug-2687, bug-2796, bug-2838,
   bug-2943, graphify, hooks-opt-in, security-scan.

Verification: lint 0 violations across 348 test files; full suite passes.

* fix(#2969): rename exemption category to pending-migration-to-typed-ir + cite tracking issue

Per maintainer feedback:
1. "Grandfathered" / "legacy" framing is wrong — both terms imply
   permanent or condoned exemption. The 8 files are tracked for
   correction, not exempted.
2. Each annotated file must cite the tracking issue so the migration
   work is auditable.

Changes:
- CONTRIBUTING.md: rename exemption category from
  `pre-existing-text-matching` to `pending-migration-to-typed-ir`. Update
  prose to "Tracked for correction, not exempted" and require each
  annotation to cite the open migration issue (e.g.
  `// allow-test-rule: pending-migration-to-typed-ir [#NNNN]`).
- 8 test files: update annotation to cite #2974 (the tracking issue
  opened for migrating these files to typed-IR assertions).
 2026-05-01 16:14:39 -04:00
Tom Boucher
0213c9baf6 fix: skip bash script behavioral tests on Windows 
The prompt-injection, base64, and secret scan tests execute bash
scripts via execFileSync which doesn't work on Windows without
Git Bash. Use node:test's { skip: IS_WINDOWS } option to skip
entire describe blocks on win32 platform.

Structure/existence tests (shebang, permissions) still run on
all platforms. Behavioral tests only run on macOS/Linux.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-24 13:33:58 -04:00
Tom Boucher
98f05d43b8 fix: security scan self-detection and Windows test compatibility 
- Add base64-scan.sh and secret-scan.sh to prompt injection scanner
  allowlist (scanner was flagging its own pattern strings)
- Skip executable bit check on Windows (no Unix permissions)
- Skip bash script execution tests on Windows (requires Git Bash)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-24 13:30:15 -04:00
Tom Boucher
feec5a37a2 ci(security): add prompt injection, base64, and secret scanning 
Add CI security pipeline to catch prompt injection attacks, base64-obfuscated
payloads, leaked secrets, and .planning/ directory commits in PRs.

This is critical for get-shit-done because the entire codebase is markdown
prompts — a prompt injection in a workflow file IS the attack surface.

New files:
- scripts/prompt-injection-scan.sh: scans for instruction override, role
  manipulation, system boundary injection, DAN/jailbreak, and tool call
  injection patterns in changed files
- scripts/base64-scan.sh: extracts base64 blobs >= 40 chars, decodes them,
  and checks decoded content against injection patterns (skips data URIs
  and binary content)
- scripts/secret-scan.sh: detects AWS keys, OpenAI/Anthropic keys, GitHub
  PATs, Stripe keys, private key headers, and generic credential patterns
- .github/workflows/security-scan.yml: runs all three scans plus a
  .planning/ directory check on every PR
- .base64scanignore / .secretscanignore: per-repo false positive allowlists
- tests/security-scan.test.cjs: 51 tests covering script existence,
  pattern matching, false positive avoidance, and workflow structure

All scripts support --diff (CI), --file, and --dir modes. Cross-platform
(macOS + Linux). SHA-pinned actions. Environment variables used for
github context in run blocks (no direct interpolation).

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
 2026-03-24 13:23:51 -04:00