golem15/msd-core
3
0
Fork 0
Code Issues Pull Requests Actions 1 Packages Projects Releases Wiki Activity

fix(#3631): exclude only __pycache__-resident bytecode from the consent digest (#3650)

Browse Source 
* test(3631): failing-first coverage for bytecode-cache in the consent hash

bundleContentHash digests a walk with no exclusion, so a routine 'python3 -m unittest'
inside a Python-backed capability bundle writes __pycache__ under the bundle, the
recomputed hash stops matching the consent record, and the capability silently goes
inactive — no error, no warning, and loop render-hooks then omits its step and gate.

Two distinct triggers, and the second is the sharper one: collectBundleEntries pushes a
{kind:'dir'} entry for EVERY directory and the digest emits a TAG_DIR marker for it, so an
EMPTY __pycache__/ flips the hash before a single .pyc is written. A fix filtering only
*.pyc would leave that live. Verified by execution against the built lib: 5 of 7 probe
rows diverge from intent today, including the empty-directory row.

The anti-regression rows are the point of the shape: editing a real scripts/m.py and
adding node_modules/pkg/index.js must BOTH still change the hash. node_modules is
deliberately not excludable — its contents are required at runtime, so dropping it from
the digest would stop consent binding executable content. The symlink row pins ordering:
exclusion must apply after the lstat fail-closed rejection, never before.

Refs #3631

* fix(3631): exclude derived bytecode caches from the consent digest

RED proven at e5ba8f1fe on the remote runner: 8 failures, exactly the rows predicted to
fail, with the four anti-regression rows already green.

collectBundleEntries now skips a hardcoded, gitignore-independent set from the DIGEST:
basenames __pycache__, .pytest_cache, .DS_Store, and any .pyc/.pyo file. Matching is
byte-exact on the raw Buffer name (the walk never utf8-decodes) and case-sensitive, so the
digest does not vary with how a name happens to be spelled on a case-insensitive volume.

Three properties were preserved deliberately, each pinned by a test:

  - The filter runs AFTER the lstat symlink/non-regular fail-closed rejection. Filtering
    first would have turned the exclusion into a way to smuggle a symlink past the check;
    a symlink named x.pyc still throws.
  - Excluded entries still count toward BUNDLE_MAX_FILES and BUNDLE_MAX_TOTAL_BYTES. The
    caps guard the WALK; the digest answers a different question, and exclusion must not
    become an unbounded-bytes hole.
  - An excluded DIRECTORY is neither emitted as a TAG_DIR marker nor recursed into. The
    directory marker was the sharper half of this bug: an empty __pycache__ flipped the
    hash before any .pyc existed, so a *.pyc-only filter would have left it live.

The issue proposed either a gitignore-aware walk or a list including node_modules. Both
are rejected. A consent binding must not delegate its scope to a .gitignore the bundle
author does not control — one line there would drop arbitrary executable content out of
the hash. And node_modules holds code that is required at runtime; excluding it would stop
consent binding executable content, turning a usability bug into a supply-chain hole. What
makes __pycache__ different is that CPython validates each .pyc against its sibling
source, which remains hashed, so a real code change still invalidates consent.

Docs: CONTEXT.md's 'EVERY regular file AND directory' claim is corrected in place.
ADR-2363's residual-gap section said the walk had 'no exclusions' — per
docs/adr/README.md ('ADRs are append-only') that is corrected by a dated amendment rather
than an in-place edit. Its D4 argument is unaffected: skill bodies are .md and stay bound.

Fixes #3631

* fix(3631): narrow the digest exclusion after two isolated security reviews

The first cut of this fix passed the full suite and was still wrong. Both orthogonal
reviews rejected it, and the second one found a hole that has nothing to do with Python.

HIGH — an excluded DIRECTORY was 'continue'd before recursion, so its whole subtree was
permanently outside the digest. Declared hook script paths allow '_', '.' and '/' with no
directory or extension rule, so hooks:[{script:'__pycache__/run.js'}] installed, executed
via node, and its bytes could be rewritten forever without moving the hash. Ship benign
v1, collect consent, then own the machine. No Python involved.

FALSE RATIONALE — the justification I wrote into the code, CONTEXT.md, the ADR amendment
and the changeset claimed CPython validates a cached .pyc against its sibling source, so
the source staying hashed kept consent honest. That is not true, and I proved it by
execution rather than argument: default timestamp invalidation compares only the source's
mtime and size, both settable by anyone who can write the bundle. A forged pyc ran while
the .py was byte-identical.

Also wrong: '*.pyc' matched anywhere, but a legacy sourceless scripts/x.pyc IS importable,
so excluding it was a live vector.

Narrowed to what is actually defensible:
  - a DIRECTORY named __pycache__/.pytest_cache has only its TAG_DIR marker suppressed;
    the walk still recurses and hashes every non-excluded child.
  - .pyc/.pyo are excluded ONLY when the parent basename is exactly __pycache__.
  - a regular FILE named __pycache__, and a DIRECTORY named x.pyc, stay bound.
  - declared hook paths containing a __pycache__/.pytest_cache segment or a .pyc/.pyo
    basename are now rejected in both validator copies — a file named .pyc can contain
    perfectly valid JavaScript, so the exclusion must not be reachable from a declared
    surface.

Accepted residual risk, stated plainly in ADR-2363 and CONTEXT.md instead of explained
away: a forged __pycache__/mod.pyc matching an unmodified, still-hashed mod.py executes
without moving the digest. Before this change that write was detected. It is accepted to
stop routine bytecode caching from silently deactivating capabilities, and it is bounded —
the attacker needs post-consent write access, everything outside __pycache__/*.pyc stays
hashed, and no declared surface can point into the excluded space.

Known limitation, not papered over: .pytest_cache CONTENTS still move the digest. Only the
directory marker is suppressed. Excluding that subtree would reopen the HIGH finding.

Refs #3631

* fix(3631): drop the .DS_Store exclusion and pin what the caps actually bind

Second round of isolated review findings. The hardening closed the two original holes —
both re-reviews confirmed that by execution — but it introduced a new one of the same
shape, and left three claims unbacked.

HIGH, self-inflicted: .DS_Store was excluded from the digest at any depth, but the hook
path validator was hardened only for __pycache__/.pytest_cache/.pyc/.pyo. So
script:'hooks/.DS_Store' was ACCEPTED, runnableHookCommand emits the bare quoted path for
a non-.js name (the branch .sh hooks already use), and capability-source copies it with
its mode bit intact. Ship it +x with a benign shebang, take consent, then rewrite it
forever — the digest never moves. Fixed by DELETING the .DS_Store exclusion rather than
teaching the validator about it: .DS_Store has nothing to do with this issue's Python
bytecode symptom, and an excluded filename is a permanently unhashed name. The narrower
the exclusion, the smaller the hole.

The residual-risk bound in ADR-2363 and CONTEXT.md claimed declared surfaces cannot reach
excluded space. That is false and is now stated correctly: node resolves an unregistered
extension through the default .js handler, so a hashed, consent-covered hooks/run.js that
requires '../__pycache__/mod.pyc' reaches it in one hop. The validator guard raises the
bar for DECLARED surfaces; it does not contain the risk. The two bounds that are real —
post-consent write access required, everything outside __pycache__/*.pyc still hashed —
are kept.

The BUNDLE_MAX_FILES boundary test had gone vacuous: it padded with root-level *.pyc,
which the hardening made non-excluded, so it no longer proved anything about excluded
entries while the ADR claimed the caps were test-pinned. It now pads __pycache__/f{i}.pyc,
with the arithmetic re-derived by execution (capability.json + the still-counted
__pycache__ dir + N). BUNDLE_MAX_TOTAL_BYTES had zero coverage at all and is now pinned by
a sparse 32 MiB __pycache__/big.pyc that must still trip the size cap — the test that
proves exclusion did not become an unbounded-bytes hole.

Added the parity assertion CLAUDE.md's Generative Fix Divergence rule requires for the two
isSafeHookScriptPath copies, and proved it can fail: mutating one BUILT copy to drop .pyo
made the parity check report the divergence. Also pinned semantics that were correct but
untested and would have survived mutation — __pycache__/sub/x.pyc stays hashed (the parent
resets to sub, which is the recursion threading itself), .pytest_cache/y.pyc stays hashed,
and .pyo in both directions, which was a free surviving mutant.

Changeset rewritten: it still described the rejected wholesale-exclusion semantics.

Refs #3631

* chore(3631): backfill changeset PR number (#3650)

---------

Co-authored-by: sim <sim@local>
This commit is contained in:
Tom Boucher
2026-08-18 22:35:54 -04:00
committed by GitHub
parent 02a36d3db9
commit 9e4f0e99ad
8 changed files with 697 additions and 17 deletions
Download Patch File Download Diff File Expand all files Collapse all files

5
.changeset/agile-elks-sing.md Normal file
View File

@@ -0,0 +1,5 @@
---
type: Fixed
pr: 3650
---
**Running a capability's own test suite no longer silently deactivates it** — `bundleContentHash` digested every entry under a capability bundle with no exclusions, so ordinary Python bytecode caching (`__pycache__/*.pyc`, written by any plain `python3` run) changed the consent-binding hash. The capability then reported `inactive` with no error and no warning, and `loop render-hooks` quietly dropped its step and gate — indistinguishable from never having installed it. An *empty* `__pycache__` directory was enough to trigger it, since the digest binds directory existence. Only a `*.pyc`/`*.pyo` file sitting directly inside a `__pycache__` directory is now excluded from the digest; a `.pyc`/`.pyo` file anywhere else stays bound, since a sourceless legacy `.pyc` there is still importable and executable. A `__pycache__`/`.pytest_cache` directory has only its own marker suppressed — its contents still bind the digest normally. `node_modules` and other executable content stay bound, excluded entries still count toward the walk's caps, and the filter runs after the symlink rejection so it cannot smuggle one past. (#3631)

2
CONTEXT.md
View File

File diff suppressed because one or more lines are too long

78
docs/adr/2363-capability-instruction-surface-trust.md
View File

@@ -155,3 +155,81 @@ The README's first ratification trap is *"shipped code is necessary, not suffici
**Full consent parity with hooks** — require the same ceremony for a skill contribution as for a hook. Rejected as consent fatigue. [`capability-trust-model.md`](../explanation/capability-trust-model.md) already rejects a per-run egress prompt on exactly these grounds: inflating every contribution to hook-level ceremony trains users to click through, degrading the prompt that matters.
**Docs correction with no ADR** — rejected: `CONTRIBUTING.md` requires an ADR for an architectural decision, and a docs edit with no recorded decision reproduces the unrecorded posture this ADR exists to end.
## Amendment (2026-08-18): `bundleContentHash` is no longer exclusion-free
The residual-gap section above states that `bundleContentHash` "walks every entry under the
bundle with no exclusions." As of #3631 that is no longer literally true, so the sentence is
corrected here rather than edited in place.
The trigger was a usability defect, not a security one — running a Python-backed capability's
own test suite wrote `__pycache__` inside the bundle (an *empty* `__pycache__` directory was
enough, because the walk emits a typed DIR marker per directory), which changed the recomputed
hash and silently deactivated the capability. The FIRST fix shipped for this (basename-excluded
`__pycache__`/`.pytest_cache`/`.DS_Store`, plus any `.pyc`/`.pyo` file anywhere, with an excluded
DIRECTORY skipped from recursion entirely) was itself found UNSAFE by two orthogonal reviews and
was corrected before merge to next. That draft is not described further here; this section
describes the shipped exclusion. `.DS_Store` was part of that first draft and was later removed
from the exclusion set entirely (not merely narrowed) — see the amendment below.
`bundleContentHash` now excludes from the DIGEST ONLY:
- A `__pycache__` or `.pytest_cache` DIRECTORY's own marker (its bare existence no longer moves
the hash) — but the directory is ALWAYS recursed into; every non-excluded child underneath is
still hashed. (Skipping recursion was the unsafe draft's hole: an excluded directory became an
unbounded, permanently-unhashed region a manifest hook `script` could point into —
`__pycache__/run.js` — ship benign, get consent, then rewrite freely afterward.)
- A `.pyc`/`.pyo` FILE, but ONLY when its immediate parent directory's basename is exactly
`__pycache__`. A `.pyc`/`.pyo` anywhere else (bundle root, `scripts/`, a directory literally
named `cache.pyc`, etc.) stays hashed, because a sourceless legacy `.pyc` there is genuinely
importable/executable. (The unsafe draft matched the suffix anywhere in the tree.)
`.DS_Store` is deliberately NOT excluded (the first draft excluded it; that exclusion was removed
entirely, not narrowed). An excluded filename is a permanently unhashed name that a declared hook
`script` could still be pointed at — e.g. `hooks/.DS_Store` — and `isSafeHookScriptPath` (below)
was hardened only for the `__pycache__`/`.pytest_cache`/`.pyc`/`.pyo` shapes, not for `.DS_Store`.
It was also unrelated to #3631's reported symptom (Python bytecode caching from a test run), so it
was not worth carrying as a permanently unhashed name. `.DS_Store` now stays bound like any other
file.
`isSafeHookScriptPath` (`src/capability-lifecycle.cts` and its mirror
`gsd-core/bin/lib/capability-validator.cjs`) rejects any DECLARED script path containing a
`__pycache__`/`.pytest_cache` path segment, or whose basename ends `.pyc`/`.pyo` — a file named
e.g. `x.pyc` can contain perfectly valid JavaScript and would be executed by `node` regardless of
extension. This raises the bar for a manifest-declared hook `script`, but it is NOT a containment
bound on the excluded region: it only ever inspects the declared `script` path string itself, not
what that script `require`s/`import`s at runtime. A hashed, consent-covered `hooks/run.js`
containing `require('../__pycache__/mod.pyc')` reaches the excluded region in one hop — Node loads
an unregistered extension through its default `.js` handler — and the validator never sees that
reference. Once loaded that way, the referenced `.pyc` is free to be rewritten post-consent with
the digest unmoved. See the corrected bound below.
**D4's argument is unaffected.** The claim this ADR rests on is that a single changed byte in a
skill body deactivates a project-scoped capability until re-consent. Skill bodies are `.md` files
and are not in the exclusion set, so that still holds exactly as written.
**ACCEPTED RESIDUAL RISK — stated plainly, not glossed over.** An earlier version of this section
claimed CPython "validates [a cached `.pyc`] against its sibling source" before trusting it. That
claim is FALSE and was disproven by execution: CPython's default (timestamp-based) invalidation
compares the cached `.pyc` header's stored mtime and size against the CURRENT source file's mtime
and size — it does NOT check source content. Both mtime and size are ordinary file metadata an
attacker who can already write to the bundle can forge. A forged `__pycache__/mod.cpython-3XX.pyc`
whose header mtime/size were copied from an unmodified, still-hashed `mod.py` executes without
moving this digest. Before this change, any write under `__pycache__` (even an empty directory)
was detected; after it, a `__pycache__/*.pyc` matching that narrow shape is not. This is accepted
deliberately — it stops routine bytecode caching from silently deactivating capabilities, which is
the usability defect this exclusion exists to fix — and it is bounded by: (1) the attacker must
already have POST-CONSENT write access to the bundle (this is not a remote-exploit surface); (2)
everything outside `__pycache__/*.pyc` — including sourceless legacy `.pyc`/`.pyo` files anywhere
else in the bundle — remains hashed. NOT a bound: the excluded region IS reachable by indirection
from any hashed, consent-covered script — a `require`/`import` of a `__pycache__/*.pyc` path is one
hop, not only CPython's own bytecode loading path described above — so `isSafeHookScriptPath`
raises the bar for a DECLARED hook `script` surface but does not contain the risk. KNOWN
LIMITATION: `.pytest_cache`'s CONTENTS still change the digest as ordinary hashed files — only its
directory marker is suppressed, so this residual risk does not extend to `.pytest_cache`.
Two properties were preserved deliberately and are pinned by tests: the exclusion is applied
AFTER the symlink/non-regular fail-closed rejection (so a symlink named `x.pyc` still throws
rather than being silently skipped), and excluded entries still count toward the walk's
size/count caps. `node_modules`, `dist`, and `build` were considered and deliberately NOT
excluded: their contents are required/executed at runtime, so dropping them from the digest
would stop consent binding executable content.

29
gsd-core/bin/lib/capability-validator.cjs
View File

@@ -210,10 +210,35 @@ const SHA512_INTEGRITY_RE = /^sha512-[A-Za-z0-9+/]{86}==$/;
// hard validation error — fail closed so the capability install/load is rejected loudly.
const SAFE_HOOK_SCRIPT_RE = /^[A-Za-z0-9._/-]+$/;
// #3631 (defense-in-depth, mirrors capability-lifecycle.cts — KEEP BOTH IN SYNC): a declared script
// path must not point into the space bundleContentHash (capability-consent.cts) excludes from the
// consent-binding digest. A file whose basename ends `.pyc`/`.pyo` can contain perfectly valid
// JavaScript and would be executed by `node` regardless of extension, and a `__pycache__`/
// `.pytest_cache` segment marks a directory whose digest marker is suppressed — so a MANIFEST-DECLARED
// executable surface must never be able to reach either, or the exclusion becomes reachable from a
// path an attacker fully controls at declare-time rather than only via post-consent tamper.
// Regex asymmetry is DELIBERATE, KEEP BOTH RULES IN SYNC WITH capability-lifecycle.cts (byte-identical
// text, verified by the isSafeHookScriptPath parity test in tests/capability-registry.test.cjs):
// (i) PYCACHE_SUFFIX_RE is case-INSENSITIVE (`/i`) on purpose — a validator should be STRICTER than
// the digest it defends, so it rejects `x.PYC` too even though bundleContentHash's own suffix
// match (hasPycacheFileSuffix, capability-consent.cts) is byte-exact and would still hash it.
// (ii) The __pycache__/.pytest_cache SEGMENT match is case-SENSITIVE to match the digest's own
// byte-exact, case-sensitive directory-basename comparison (CPython always writes a lowercase
// `__pycache__`) — a validator segment match looser than the digest here would reject paths the
// digest would still hash, which is over-strict in the wrong direction for a defense-in-depth
// check layered on top of an already-correct digest.
// (iii) The `[/\\]` backslash alternations in both regexes are defensive/UNREACHABLE in practice:
// SAFE_HOOK_SCRIPT_RE (above) already rejects any backslash character outright, so a script
// string containing `\` never reaches either PYCACHE_*_RE check.
const PYCACHE_SEGMENT_RE = /(?:^|[/\\])(__pycache__|\.pytest_cache)(?:[/\\]|$)/;
const PYCACHE_SUFFIX_RE = /\.(pyc|pyo)$/i;
/**
* #1460 (R): true when a relative hook-script path is shell-safe (see SAFE_HOOK_SCRIPT_RE).
* Rejects absolute paths, `..` segments, a leading `-` on any path segment, and any char
* outside the allowlist (whitespace / shell metacharacters / control / NUL).
* outside the allowlist (whitespace / shell metacharacters / control / NUL). #3631: also rejects a
* path with a `__pycache__`/`.pytest_cache` segment or a `.pyc`/`.pyo` basename suffix (defense in
* depth — keeps a declared executable surface out of the digest-excluded space).
*/
function isSafeHookScriptPath(script) {
if (typeof script !== 'string' || script.length === 0) return false;
@@ -225,6 +250,8 @@ function isSafeHookScriptPath(script) {
for (const seg of segments) {
if (seg.startsWith('-')) return false;
}
if (PYCACHE_SEGMENT_RE.test(script)) return false;
if (PYCACHE_SUFFIX_RE.test(path.basename(script))) return false;
return true;
}

167
src/capability-consent.cts
View File

@@ -15,8 +15,9 @@
* project ledger — `'' === ''` is no binding) NOR to the `disclosureSignature` alone (which covers
* only executable surfaces, so a declarative-only cap has a constant signature and a repo-write
* attacker could swap `capability.json` for a malicious gate/contribution while consent still
* matched). `bundleContentHash` is recomputed by the loader at load over EVERY file in the bundle
* (manifest AND artifacts AND identity), so any tamper — declarative-only swap, hook-script edit,
* matched). `bundleContentHash` is recomputed by the loader at load over the bundle (manifest AND
* artifacts AND identity), excluding only derived Python bytecode/cache noise (#3631 — see
* `bundleContentHash`'s own doc comment), so any tamper — declarative-only swap, hook-script edit,
* empty-integrity local install — changes the hash and leaves the cap inactive. `integrity` and
* `disclosureSignature` remain on the record for the human disclosure + re-consent-on-executable-
* change UX (TRUST-2); they are NO LONGER the security binding.
@@ -265,10 +266,31 @@ function normalizeSepBytes(rel: Buffer): Buffer {
/**
* Recursively collect every REGULAR file AND every DIRECTORY under `absDir` as RAW-BYTE POSIX-relative
* paths (`rel`, relative to the bundle root), refusing to follow symlinks out of the bundle. Bounded:
* throws if the entry count or total byte size exceeds the caps (fail closed — a hostile/runaway tree
* never hashes unbounded content). A non-regular entry encountered IN the tree (FIFO/device) is a
* fail-closed throw — a bundle must be plain files and directories.
* paths (`rel`, relative to the bundle root), refusing to follow symlinks out of the bundle. Two
* NARROW, RECURSION-PRESERVING exclusions apply (#3631, amended after two orthogonal reviews found the
* original wholesale-skip UNSAFE — see the doc comment on `bundleContentHash` for the full rationale
* and the accepted residual risk):
*
* 1. A DIRECTORY whose basename is exactly `__pycache__` or `.pytest_cache`: its own TAG_DIR marker
* is SUPPRESSED (not pushed), but it is STILL RECURSED INTO — every child that is not itself
* excluded is still hashed. Suppressing only the marker is what keeps an empty/all-excluded such
* directory from moving the digest; recursing is what closes the original hole (H1): skipping
* recursion made an excluded directory an unbounded, permanently-unhashed region a manifest could
* point a hook `script` at (`__pycache__/run.js`) to install benign, then rewrite freely post-consent.
* 2. A FILE whose name ends `.pyc` or `.pyo` is excluded ONLY when its PARENT directory's basename is
* exactly `__pycache__` (checked byte-exact, never `.pytest_cache`) — a `.pyc`/`.pyo` anywhere else
* (bundle root, `scripts/`, a directory literally named `cache.pyc`, etc.) is hashed normally,
* because a sourceless legacy `.pyc` there IS importable and executable (H2).
*
* A regular FILE literally named `__pycache__` or `.pytest_cache` is NOT excluded (only a DIRECTORY of
* that basename gets marker-suppression) — it is hashed like any other file. A DIRECTORY literally
* named `x.pyc` is NOT excluded either — the suffix rule only ever applies to files. All exclusion
* checks run strictly AFTER the lstat-backed symlink/non-regular rejections below, so a symlink or
* device masquerading under an excluded name is still fail-closed rejected rather than silently
* skipped. Bounded: throws if the entry count or total byte size exceeds the caps (fail closed — a
* hostile/runaway tree never hashes unbounded content); an excluded entry still counts toward BOTH
* caps (the caps guard the walk itself, not the digest). A non-regular entry encountered IN the tree
* (FIFO/device) is a fail-closed throw — a bundle must be plain files and directories.
*
* #1459 finding 2 (MED/HIGH, ROUND 6): the enumeration ITSELF is bounded. Instead of
* `fs.readdirSync` (which loads + sorts a WHOLE directory before the count cap — so a malicious bundle
@@ -287,11 +309,76 @@ function normalizeSepBytes(rel: Buffer): Buffer {
* abs/rel paths are concatenated at the BYTE level, so an invalid-UTF-8 filename is never lossily
* decoded — two filenames that differ only in invalid bytes produce distinct rel byte strings.
*
* @param absDir the absolute directory to scan, as RAW BYTES (Buffer).
* @param relDir the relpath of `absDir` from the bundle root, as RAW BYTES (Buffer; empty at the root).
* @param count the CUMULATIVE entry counter shared across the whole recursive walk (fail-closed at the cap).
* @param absDir the absolute directory to scan, as RAW BYTES (Buffer).
* @param relDir the relpath of `absDir` from the bundle root, as RAW BYTES (Buffer; empty at the root).
* @param dirBasename the basename of `absDir` itself, as RAW BYTES (Buffer) — the PARENT basename every
* entry collected at this level shares. Threaded down so a `.pyc`/`.pyo` FILE can be
* excluded only when ITS parent is exactly `__pycache__` (empty at the bundle root).
* @param count the CUMULATIVE entry counter shared across the whole recursive walk (fail-closed at the cap).
*/
function collectBundleEntries(absDir: Buffer, relDir: Buffer, acc: BundleEntry[], total: { bytes: number }, count: { n: number }): void {
// #3631 (amended — see bundleContentHash's doc comment for the H1/H2 rationale and accepted residual
// risk): basename rules for what is excluded FROM THE DIGEST. They are NOT excluded from the walk's
// resource caps — see the count.n / total.bytes accounting below, which still sees every excluded
// entry.
//
// Exact byte comparison, case-sensitive, deliberately: CPython always writes a lowercase
// `__pycache__` directory, so byte-exact matching keeps the digest identical across case-insensitive
// filesystems (e.g. default macOS/Windows) instead of varying with how a name happens to be spelled
// on disk.
//
// DELIBERATELY NOT on this list: node_modules, dist, build, or similar. Those hold code that is
// actually required/executed at runtime, so excluding them from the digest would stop consent from
// binding executable content — turning a usability bug (noisy re-consent prompts) into a
// supply-chain hole (a swapped dependency that never re-triggers consent).
/** Directory basenames whose TAG_DIR marker is suppressed — but the directory is STILL RECURSED INTO. */
const PYCACHE_DIR_BASENAMES: readonly Buffer[] = [
Buffer.from('__pycache__'),
Buffer.from('.pytest_cache'),
];
/** FILE-name suffixes excluded ONLY when the file's parent directory basename is `__pycache__` (see below). */
const PYCACHE_FILE_SUFFIXES: readonly Buffer[] = [
Buffer.from('.pyc'),
Buffer.from('.pyo'),
];
/** The one parent directory basename that gates the `.pyc`/`.pyo` FILE-suffix exclusion — never `.pytest_cache`. */
const PYCACHE_PARENT_BASENAME = Buffer.from('__pycache__');
/**
* True when `name` (a raw-byte dirent basename) is a DIRECTORY whose TAG_DIR marker must be
* suppressed — `__pycache__` or `.pytest_cache`, exact byte match. The caller still recurses into it;
* this predicate answers ONLY "skip the marker", never "skip the subtree".
*/
function isPycacheDirBasename(name: Buffer): boolean {
for (const exact of PYCACHE_DIR_BASENAMES) {
if (Buffer.compare(name, exact) === 0) return true;
}
return false;
}
/** True when raw-byte basename `name` ends in `.pyc` or `.pyo` (byte-suffix match, never utf8-decoded). */
function hasPycacheFileSuffix(name: Buffer): boolean {
for (const suffix of PYCACHE_FILE_SUFFIXES) {
if (name.length >= suffix.length && Buffer.compare(name.subarray(name.length - suffix.length), suffix) === 0) {
return true;
}
}
return false;
}
/**
* True when a FILE with basename `name`, inside a directory whose OWN basename is `dirBasename`, must
* be excluded from the digest: a `.pyc`/`.pyo` suffix whose parent directory basename is exactly
* `__pycache__` (byte-exact — never `.pytest_cache`, so a `.pyc` sitting directly inside a
* `.pytest_cache` dir is still hashed). A `.pyc`/`.pyo` file anywhere else — bundle root, `scripts/`,
* a directory literally named `cache.pyc`, etc. — is NOT excluded (H2): a sourceless legacy `.pyc`
* there is importable and executable, so it must stay bound to consent.
*/
function isExcludedFileBasename(name: Buffer, dirBasename: Buffer): boolean {
return hasPycacheFileSuffix(name) && Buffer.compare(dirBasename, PYCACHE_PARENT_BASENAME) === 0;
}
function collectBundleEntries(absDir: Buffer, relDir: Buffer, dirBasename: Buffer, acc: BundleEntry[], total: { bytes: number }, count: { n: number }): void {
let dir: fs.Dir;
try {
// RAW-BYTE streaming open: dirent names are Buffers (encoding: 'buffer'), so an invalid-UTF-8
@@ -344,19 +431,34 @@ function collectBundleEntries(absDir: Buffer, relDir: Buffer, acc: BundleEntry[]
throw new Error(`bundleContentHash: refusing to hash a symlink in the bundle: "${abs.toString('utf8')}"`);
}
if (st.isDirectory()) {
// #3631 (amended, H1): exclusion is checked HERE — after the symlink fail-closed throw above —
// so a symlink named e.g. "__pycache__" or "x.pyc" is never silently skipped; only a REAL
// (lstat-confirmed) dir/file can be excluded. An excluded directory's own dirent was already
// counted toward count.n above (the cap guards the walk itself). Only the TAG_DIR MARKER is
// suppressed for `__pycache__`/`.pytest_cache` — the directory is ALWAYS recursed into so every
// non-excluded child underneath is still hashed (closing H1: no unbounded unhashed region).
if (isPycacheDirBasename(name)) {
collectBundleEntries(abs, rel, name, acc, total, count);
continue;
}
// Emit a typed DIR marker for THIS directory (so an empty dir is bound), then recurse into it.
acc.push({ abs, rel, kind: 'dir' });
collectBundleEntries(abs, rel, acc, total, count);
collectBundleEntries(abs, rel, name, acc, total, count);
continue;
}
if (!st.isFile()) {
throw new Error(`bundleContentHash: refusing to hash a non-regular file in the bundle: "${abs.toString('utf8')}"`);
}
acc.push({ abs, rel, kind: 'file' });
// #3631: an excluded FILE (a __pycache__/*.pyc or *.pyo) still counts its bytes toward the
// total-size cap below — exclusion answers "does this bind the digest?", not "is this safe to
// read unbounded?" — so it must never become a way to smuggle unbounded bytes past
// BUNDLE_MAX_TOTAL_BYTES.
total.bytes += st.size;
if (total.bytes > BUNDLE_MAX_TOTAL_BYTES) {
throw new Error(`bundleContentHash: bundle size exceeds ${BUNDLE_MAX_TOTAL_BYTES} bytes (refusing)`);
}
if (isExcludedFileBasename(name, dirBasename)) continue;
acc.push({ abs, rel, kind: 'file' });
}
}
@@ -383,8 +485,39 @@ const TAG_DIR = Buffer.from([0x02]);
/**
* The recomputed full-bundle content hash (#1459 CB-1/CB-2/TRUST2-5) — the SECURITY BINDING. A
* `sha512-<base64>` over a DETERMINISTIC, INJECTIVE, LOSSLESS serialization of EVERY regular file
* AND directory under `capDir` (recursively).
* `sha512-<base64>` over a DETERMINISTIC, INJECTIVE, LOSSLESS serialization of every regular file
* AND directory under `capDir` (recursively), with two NARROW exclusions (#3631, amended after two
* orthogonal reviews found the original wholesale directory-skip UNSAFE):
* - A `__pycache__` or `.pytest_cache` DIRECTORY's own TAG_DIR marker is suppressed, but it is
* ALWAYS recursed into — every non-excluded child underneath is still hashed.
* - A `.pyc`/`.pyo` FILE is excluded ONLY when its immediate parent directory's basename is exactly
* `__pycache__`; a `.pyc`/`.pyo` anywhere else (bundle root, `scripts/`, etc.) is hashed normally,
* since a sourceless legacy `.pyc` there is importable and executable.
* `node_modules`, `dist`, `build`, and similar are deliberately NOT excluded: their contents are
* executed/required at runtime, so dropping them from the digest would stop consent from binding
* executable content.
*
* ACCEPTED RESIDUAL RISK (documented, not a defect to re-litigate): CPython's default `.pyc`
* invalidation (timestamp-based) validates a cached bytecode file against its SOURCE's mtime and size
* only — NOT against the source's content — and both mtime and size are attacker-settable by whoever
* can already write to the bundle. So a forged `__pycache__/mod.cpython-3XX.pyc` whose header mtime/size
* were copied from an unmodified, still-hashed `mod.py` will execute without moving this digest. Before
* this exclusion existed that write was detected (any file under `__pycache__` moved the hash); after
* it, it is not, for files matching the narrow `__pycache__/*.pyc` shape above. This is accepted
* deliberately — the alternative (H1's original wholesale skip, or hashing every regenerated bytecode
* file) either reopens an unbounded unhashed region or makes consent fire on routine bytecode caching —
* and is bounded by: (1) the attacker must already have POST-CONSENT write access to the bundle; (2)
* everything outside `__pycache__/*.pyc` — including sourceless legacy `.pyc`/`.pyo` files anywhere
* else — remains hashed. NOT a bound, despite `isSafeHookScriptPath` (capability-lifecycle.cts /
* capability-validator.cjs) rejecting a manifest-declared hook `script` that names a
* `__pycache__`/`.pytest_cache` segment or ends `.pyc`/`.pyo`: the excluded region is still reachable
* by ONE HOP of indirection from any hashed, consent-covered script — a `require`/`import` of a
* `__pycache__/*.pyc` module path is not itself a declared script and the validator never sees it —
* so a hashed `hooks/run.js` can load a `__pycache__/mod.pyc` whose bytes are then free to change
* post-consent with the digest unmoved. The validator guard raises the bar for DECLARED surfaces; it
* does not contain the risk. KNOWN LIMITATION: `.pytest_cache`'s CONTENTS still change the digest as
* normal files — only its directory marker is suppressed, so this residual risk does NOT extend to
* `.pytest_cache`.
*
* Canonicalization (#1459 findings 1 + 4 — the prior `relpath + NUL + content + NUL` over utf8-decoded
* STRINGS was non-injective, lossy in CONTENT, AND lossy in the PATH component):
@@ -412,8 +545,12 @@ const TAG_DIR = Buffer.from([0x02]);
function bundleContentHash(capDir: string): string {
// Resolve to an absolute path, then carry it as RAW BYTES so the walk never lossily decodes a name.
const rootBytes = Buffer.from(path.resolve(capDir));
// The root's own basename is threaded as the initial `dirBasename` so the parent-basename check for
// a `.pyc`/`.pyo` FILE applies even to a file placed DIRECTLY at the bundle root (root literally
// named `__pycache__` is the only case this matters for, and it is a correct, if exotic, match).
const rootBasename = Buffer.from(path.basename(path.resolve(capDir)));
const entries: BundleEntry[] = [];
collectBundleEntries(rootBytes, Buffer.alloc(0), entries, { bytes: 0 }, { n: 0 });
collectBundleEntries(rootBytes, Buffer.alloc(0), rootBasename, entries, { bytes: 0 }, { n: 0 });
// Sort by the raw-byte (separator-normalized) relpath so the digest is identical on Windows and POSIX,
// and is independent of the on-disk creation/readdir order. Tie-break on kind so a (degenerate, never
// produced on a real fs) file-and-dir same-relpath pair still has a stable order.

24
src/capability-lifecycle.cts
View File

@@ -416,6 +416,28 @@ function confinedSharedFile(runtimeDir: string, relFile: unknown): string | null
// isSafeHookScriptPath; see confinedBundleScript for why). Only [A-Za-z0-9._/-], no leading
// `-` segment, no `..`, not absolute.
const SAFE_HOOK_SCRIPT_RE = /^[A-Za-z0-9._/-]+$/;
// #3631 (defense-in-depth, mirrors capability-validator.cjs — KEEP BOTH IN SYNC): a declared script
// path must not point into the space bundleContentHash (capability-consent.cts) excludes from the
// consent-binding digest. A file whose basename ends `.pyc`/`.pyo` can contain perfectly valid
// JavaScript and would be executed by `node` regardless of extension, and a `__pycache__`/
// `.pytest_cache` segment marks a directory whose digest marker is suppressed — so a MANIFEST-DECLARED
// executable surface must never be able to reach either, or the exclusion becomes reachable from a
// path an attacker fully controls at declare-time rather than only via post-consent tamper.
// Regex asymmetry is DELIBERATE, KEEP BOTH RULES IN SYNC WITH capability-validator.cjs (byte-identical
// text, verified by the isSafeHookScriptPath parity test in tests/capability-registry.test.cjs):
// (i) PYCACHE_SUFFIX_RE is case-INSENSITIVE (`/i`) on purpose — a validator should be STRICTER than
// the digest it defends, so it rejects `x.PYC` too even though bundleContentHash's own suffix
// match (hasPycacheFileSuffix, capability-consent.cts) is byte-exact and would still hash it.
// (ii) The __pycache__/.pytest_cache SEGMENT match is case-SENSITIVE to match the digest's own
// byte-exact, case-sensitive directory-basename comparison (CPython always writes a lowercase
// `__pycache__`) — a validator segment match looser than the digest here would reject paths the
// digest would still hash, which is over-strict in the wrong direction for a defense-in-depth
// check layered on top of an already-correct digest.
// (iii) The `[/\\]` backslash alternations in both regexes are defensive/UNREACHABLE in practice:
// SAFE_HOOK_SCRIPT_RE (above) already rejects any backslash character outright, so a script
// string containing `\` never reaches either PYCACHE_*_RE check.
const PYCACHE_SEGMENT_RE = /(?:^|[/\\])(__pycache__|\.pytest_cache)(?:[/\\]|$)/;
const PYCACHE_SUFFIX_RE = /\.(pyc|pyo)$/i;
function isSafeHookScriptPath(script: string): boolean {
if (typeof script !== 'string' || script.length === 0) return false;
if (!SAFE_HOOK_SCRIPT_RE.test(script)) return false;
@@ -425,6 +447,8 @@ function isSafeHookScriptPath(script: string): boolean {
for (const seg of segments) {
if (seg.startsWith('-')) return false;
}
if (PYCACHE_SEGMENT_RE.test(script)) return false;
if (PYCACHE_SUFFIX_RE.test(path.basename(script))) return false;
return true;
}

333
tests/capability-consent.test.cjs
View File

@@ -985,4 +985,337 @@ test('WIN-3: roots containing spaces are keyed unambiguously on disk (no collisi
}
});
// ---------------------------------------------------------------------------
// #3631: bundleContentHash excludes Python bytecode-cache noise from the DIGEST, so that
// running a Python-backed capability's own test suite — which writes __pycache__ inside
// the bundle — does not flip the recomputed hash and silently deactivate consent.
//
// The exclusion is deliberately NARROW, because this digest is a consent binding and every
// excluded byte is a byte that can change post-consent without detection:
// - a DIRECTORY named __pycache__ / .pytest_cache has only its TAG_DIR marker suppressed;
// the walk STILL RECURSES and hashes every non-excluded child (so __pycache__/run.js
// stays bound). Skipping recursion would make it a permanently unhashed region that a
// declared hook script could point into.
// - a FILE ending .pyc/.pyo is excluded ONLY when its parent basename is exactly
// __pycache__. Elsewhere it stays bound: a legacy sourceless .pyc IS importable.
// - a regular FILE named __pycache__, and a DIRECTORY named x.pyc, are ordinary content
// and stay bound — marker suppression is directory-only, the suffix rule file-only.
// Exclusion applies AFTER the lstat/symlink fail-closed rejection, and excluded entries
// still count toward BUNDLE_MAX_FILES / BUNDLE_MAX_TOTAL_BYTES.
//
// Deliberately NOT excluded: node_modules, dist, build — their contents ARE executed, so
// excluding them would break the consent binding for real executable surfaces.
// ACCEPTED RESIDUAL RISK (see the ADR-2363 amendment): CPython's default timestamp
// invalidation checks a cached pyc only against its source's mtime+size, both forgeable by
// anyone who can already write to the bundle — so a forged __pycache__/mod.pyc matching an
// unmodified mod.py executes without moving the digest. Accepted knowingly; NOT excused by
// any claim that CPython validates bytecode against source content (it does not).
// ---------------------------------------------------------------------------
test('#3631: an EMPTY __pycache__/ directory does not change the bundle hash', (t) => {
// This is the TAG_DIR-marker trigger: collectBundleEntries pushes a {kind:'dir'} entry for EVERY
// directory (including empty ones) and bundleContentHash emits a TAG_DIR marker for it. A fix that
// only filters *.pyc file CONTENT and still emits the DIR marker for an empty __pycache__/ leaves
// this red — the exclusion must be by basename, applied to the dir entry itself, not just its files.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, '__pycache__'), { recursive: true }); // EMPTY — no .pyc inside yet
const after = consent.bundleContentHash(dir);
assert.strictEqual(after, before, 'an empty __pycache__ directory must not change the hash');
});
test('#3631: hooks/__pycache__/check.cpython-313.pyc does not change the hash', (t) => {
const dir = makeBundle({ manifest: { id: 'cap', role: 'feature', version: '1.0.0', hooks: [{ event: 'PostToolUse', script: 'hooks/check.js' }] }, script: 'console.log(1)' });
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, 'hooks', '__pycache__'), { recursive: true });
fs.writeFileSync(path.join(dir, 'hooks', '__pycache__', 'check.cpython-313.pyc'), Buffer.from([5, 6, 7]));
const after = consent.bundleContentHash(dir);
assert.strictEqual(after, before, '__pycache__ nested under hooks/ must not change the hash');
});
test('#3631: __pycache__ under an existing tests/ dir does not change the hash (isolated from the tests/-dir-creation variable)', (t) => {
// Creating the NEW tests/ dir itself legitimately changes the hash (it is not excluded), so tests/ is
// pre-created WITH a real file BEFORE the baseline snapshot — only the __pycache__ add is under test.
const dir = makeBundle({});
t.after(() => cleanup(dir));
fs.mkdirSync(path.join(dir, 'tests'), { recursive: true });
fs.writeFileSync(path.join(dir, 'tests', 'test_x.py'), 'def test_x():\n assert True\n', 'utf8');
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, 'tests', '__pycache__'), { recursive: true });
fs.writeFileSync(path.join(dir, 'tests', '__pycache__', 'test_x.pyc'), Buffer.from([1, 2, 3]));
const after = consent.bundleContentHash(dir);
assert.strictEqual(after, before, '__pycache__ under an already-present tests/ dir must not change the hash');
});
test('#3631 (post-hardening): .pytest_cache/CACHEDIR.TAG DOES change the hash — only the dir MARKER is suppressed', (t) => {
// Hardened semantics: exclusion suppresses the TAG_DIR marker for a __pycache__/.pytest_cache
// directory, but the walk still RECURSES into it and hashes every non-excluded child. Suppressing
// the whole subtree would create an unhashed region a declared surface could point into (the HIGH
// finding closed below) — this is a deliberate, documented limitation, not a gap.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, '.pytest_cache'), { recursive: true });
fs.writeFileSync(path.join(dir, '.pytest_cache', 'CACHEDIR.TAG'), 'Signature: 8a477f597d28d172789f06886806bc55\n', 'utf8');
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, '.pytest_cache/CACHEDIR.TAG content is still bound to the hash even though the dir marker is suppressed');
});
test('#3631 (post-hardening): .DS_Store at the bundle root DOES change the hash — deliberately NOT excluded', (t) => {
// .DS_Store is deliberately NOT on the exclusion list. Excluding a filename means a permanently
// unhashed name that a declared hook `script` could point into (e.g. `hooks/.DS_Store`, which the
// validator's __pycache__/.pytest_cache/.pyc/.pyo rejection does not cover), and it is unrelated to
// #3631's reported symptom (Python bytecode caching from a test run). Stays bound like any other file.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.writeFileSync(path.join(dir, '.DS_Store'), Buffer.from([0, 0, 0, 1, 2, 3]));
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, '.DS_Store must still change the hash — it is not excluded');
});
test('#3631 (post-hardening): a REGULAR FILE literally named __pycache__ at the bundle root DOES change the hash', (t) => {
// Marker suppression applies to DIRECTORIES only. A file wearing the __pycache__ name is ordinary
// content — the walk must still bind it, and must not choke on the kind mismatch.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.writeFileSync(path.join(dir, '__pycache__'), 'not actually a directory', 'utf8');
let after;
assert.doesNotThrow(() => { after = consent.bundleContentHash(dir); }, 'a file named __pycache__ must not throw');
assert.notStrictEqual(after, before, 'a regular FILE named __pycache__ is ordinary content and must change the hash');
});
test('#3631 (post-hardening): stray.pyc at the bundle ROOT (not under any __pycache__) DOES change the hash', (t) => {
// A legacy sourceless .pyc outside __pycache__ IS importable by CPython, so it must stay bound to
// the digest. Only __pycache__-resident bytecode (whose parent dir basename is exactly
// __pycache__) is excluded by suffix; a stray .pyc elsewhere is hashed like any other file.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.writeFileSync(path.join(dir, 'stray.pyc'), Buffer.from([9, 9, 9]));
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, 'a root-level .pyc file (not under __pycache__) must still change the hash — it is legacy-importable content');
});
test('#3631: a bundle whose ONLY added content is __pycache__/mod.pyc hashes IDENTICALLY to before that dir existed, and does not throw', (t) => {
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, '__pycache__'), { recursive: true });
fs.writeFileSync(path.join(dir, '__pycache__', 'mod.pyc'), Buffer.from([1, 2, 3, 4]));
let after;
assert.doesNotThrow(() => { after = consent.bundleContentHash(dir); }, 'adding only __pycache__/mod.pyc must not throw');
assert.strictEqual(after, before, 'adding only __pycache__/mod.pyc must be a full no-op on the digest');
});
test('#3631 (GREEN — must stay green: anti-regression control): modifying a REAL source file still changes the hash', (t) => {
// Proves the fix NARROWS the hash rather than gutting it — an actual code-content edit must still be
// observable to the binding once the __pycache__/.pyc noise is excluded.
const dir = makeBundle({});
t.after(() => cleanup(dir));
fs.mkdirSync(path.join(dir, 'scripts'), { recursive: true });
fs.writeFileSync(path.join(dir, 'scripts', 'm.py'), 'print("v1")\n', 'utf8');
const before = consent.bundleContentHash(dir);
fs.writeFileSync(path.join(dir, 'scripts', 'm.py'), 'print("v2")\n', 'utf8');
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, 'a real source-file content edit must still change the hash');
});
test('#3631 (GREEN — must stay green): adding node_modules/pkg/index.js changes the hash (node_modules is deliberately NOT excluded)', (t) => {
// node_modules holds code that is actually executed/required at runtime — excluding it would break the
// consent binding for real executable content. Only __pycache__/.pytest_cache/*.pyc/*.pyo (the latter
// two only when nested directly under __pycache__) are excluded; node_modules, dist, and build are NOT
// on that list and must keep binding the hash.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, 'node_modules', 'pkg'), { recursive: true });
fs.writeFileSync(path.join(dir, 'node_modules', 'pkg', 'index.js'), 'module.exports = 1;\n', 'utf8');
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, 'node_modules content must still bind the hash');
});
test('#3631 (GREEN — must stay green, boundary): excluded .pyc entries still count toward BUNDLE_MAX_FILES (limit-1 / limit / limit+1)', (t) => {
// The digest excludes __pycache__/*.pyc CONTENT, but the walk's entry-count cap must still see every
// entry (excluded or not) BEFORE exclusion is applied — the caps guard the walk itself (DoS/memory
// bound), the digest answers a separate question. Padding must be GENUINELY excluded to prove that
// property: a ROOT-level f{i}.pyc is NOT excluded (its parent is not __pycache__ — see the sibling
// "stray.pyc at the bundle ROOT" test above), so it would prove nothing about excluded entries. The
// padding here lives under __pycache__/f{i}.pyc, which the digest never binds, while the count cap
// still sees it. The real cap is BUNDLE_MAX_FILES (default BUNDLE_MAX_FILES_DEFAULT = 100_000 in
// src/capability-consent.cts:117) — too large to materialize cheaply in a test, so this drives the
// exported `_setBundleMaxFilesForTest` seam (the same seam the existing "finding 2" cap tests above
// use) down to a small CAP and proves the exact boundary.
//
// Entry arithmetic (confirmed by executing the built lib, not by reasoning): a bundle built from
// capability.json + an empty __pycache__/ dir + N excluded .pyc files inside it has
// 1 [capability.json] + 1 [the __pycache__ DIRECTORY dirent itself — its marker is suppressed but the
// walk still counts the dirent and still recurses] + N [f{i}.pyc files] = N + 2 total entries.
const CAP = 5;
const restore = consent._setBundleMaxFilesForTest(CAP);
t.after(restore);
const build = (pycCount) => {
const bdir = makeBundle({});
fs.mkdirSync(path.join(bdir, '__pycache__'), { recursive: true });
for (let i = 0; i < pycCount; i++) {
fs.writeFileSync(path.join(bdir, '__pycache__', `f${i}.pyc`), Buffer.from([i]));
}
return bdir;
};
const dirBelow = build(CAP - 3); // total entries = 1 + 1 + (CAP-3) = CAP-1
t.after(() => cleanup(dirBelow));
assert.doesNotThrow(() => consent.bundleContentHash(dirBelow), 'limit-1 total entries (capability.json + __pycache__ dir + genuinely-excluded .pyc padding) must not throw');
const dirAt = build(CAP - 2); // total entries = 1 + 1 + (CAP-2) = CAP
t.after(() => cleanup(dirAt));
assert.doesNotThrow(() => consent.bundleContentHash(dirAt), 'exactly-limit total entries (capability.json + __pycache__ dir + genuinely-excluded .pyc padding) must not throw');
const dirOver = build(CAP - 1); // total entries = 1 + 1 + (CAP-1) = CAP+1
t.after(() => cleanup(dirOver));
assert.throws(
() => consent.bundleContentHash(dirOver),
/exceeds|refusing/i,
'limit+1 total entries must still throw even though every .pyc padding entry is excluded from the digest — caps guard the WALK, not the digest',
);
});
test('#3631 (GREEN — must stay green, boundary): an EXCLUDED __pycache__/*.pyc file still trips BUNDLE_MAX_TOTAL_BYTES', (t) => {
// Exclusion answers "does this bind the digest?", never "is this safe to read unbounded?" — an
// excluded file's bytes must still count toward BUNDLE_MAX_TOTAL_BYTES (src/capability-consent.cts:105,
// 16 MiB), or exclusion becomes an unbounded-bytes DoS hole. Uses a SPARSE file (fs.truncateSync) well
// beyond any plausible cap so the assertion never hardcodes the exact byte constant — it costs no real
// disk or CPU time (no 32 MiB buffer is ever written).
const dir = makeBundle({});
t.after(() => cleanup(dir));
fs.mkdirSync(path.join(dir, '__pycache__'), { recursive: true });
const bigPath = path.join(dir, '__pycache__', 'big.pyc');
fs.writeFileSync(bigPath, Buffer.alloc(0));
fs.truncateSync(bigPath, 32 * 1024 * 1024); // sparse — well past the 16 MiB cap, no real bytes written
assert.throws(
() => consent.bundleContentHash(dir),
/bundle size exceeds \d+ bytes \(refusing\)/,
'an EXCLUDED __pycache__/*.pyc file must still trip BUNDLE_MAX_TOTAL_BYTES even though its content never reaches the digest',
);
});
test('#3631 (security pin — parent-threading precision): __pycache__/sub/x.pyc stays HASHED — its parent is "sub", not "__pycache__"', (t) => {
// The single most valuable missing pin (isolated review finding): the .pyc/.pyo suffix exclusion is
// gated on the IMMEDIATE parent directory basename, threaded down one level at a time through the
// recursive walk. A .pyc two levels under __pycache__ must NOT be excluded — only a .pyc whose direct
// parent is __pycache__ itself is.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, '__pycache__', 'sub'), { recursive: true });
fs.writeFileSync(path.join(dir, '__pycache__', 'sub', 'x.pyc'), Buffer.from([1, 2, 3]));
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, '__pycache__/sub/x.pyc must still change the hash — its parent basename is "sub", not "__pycache__"');
});
test('#3631 (security pin): .pytest_cache/y.pyc stays HASHED — the .pyc suffix rule never gates on .pytest_cache', (t) => {
// The PYCACHE_PARENT_BASENAME gate for the .pyc/.pyo suffix rule is exactly "__pycache__", never
// ".pytest_cache" — a .pyc sitting directly inside a .pytest_cache dir is ordinary content.
const dir = makeBundle({});
t.after(() => cleanup(dir));
fs.mkdirSync(path.join(dir, '.pytest_cache'), { recursive: true });
const before = consent.bundleContentHash(dir);
fs.writeFileSync(path.join(dir, '.pytest_cache', 'y.pyc'), Buffer.from([1, 2, 3]));
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, '.pytest_cache/y.pyc must still change the hash — the .pyc suffix rule only ever gates on a __pycache__ parent');
});
test('#3631 (security pin — closes the untested .pyo mutant): __pycache__/m.pyo is excluded exactly like a .pyc sibling', (t) => {
// .pyo is on PYCACHE_FILE_SUFFIXES alongside .pyc but was never independently exercised anywhere in
// the suite — a surviving mutant could delete the ".pyo" entry with nothing failing. Pins both halves:
// excluded under __pycache__/, and still bound everywhere else.
const dir = makeBundle({});
t.after(() => cleanup(dir));
fs.mkdirSync(path.join(dir, '__pycache__'), { recursive: true });
const before = consent.bundleContentHash(dir);
fs.writeFileSync(path.join(dir, '__pycache__', 'm.pyo'), Buffer.from([1, 2, 3]));
const after = consent.bundleContentHash(dir);
assert.strictEqual(after, before, '__pycache__/m.pyo must be excluded from the digest exactly like __pycache__/m.pyc');
});
test('#3631 (security pin — closes the untested .pyo mutant): a root-level m.pyo DOES change the hash', (t) => {
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.writeFileSync(path.join(dir, 'm.pyo'), Buffer.from([1, 2, 3]));
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, 'a root-level m.pyo (not under __pycache__) must still change the hash — it is legacy-importable content, same as a stray .pyc');
});
test('#3631 (GREEN — already passes today, pins ordering post-fix): a SYMLINK named x.pyc still makes bundleContentHash THROW (fail-closed)', { skip: process.platform === 'win32' }, (t) => {
// Pins that the exclusion match must be applied AFTER the existing lstat/symlink rejection, never
// before — a symlinked *.pyc is exactly the shape a naive "skip by suffix before lstat" fix would
// silently pass through instead of rejecting.
const dir = makeBundle({});
t.after(() => cleanup(dir));
fs.symlinkSync('/etc/passwd', path.join(dir, 'x.pyc'));
assert.throws(() => consent.bundleContentHash(dir), /symlink/i, 'a symlinked *.pyc must still be rejected fail-closed');
});
// ---------------------------------------------------------------------------
// #3631 hardening — security regression pins. Two independent reviews found holes in the original
// exclusion: (HIGH) suppressing recursion into an excluded dir left it a permanently-unhashed region a
// declared surface could point into, and (the sourceless-legacy-.pyc vector) a bare .pyc anywhere was
// excluded by suffix alone even though CPython can import a sourceless .pyc outside __pycache__. Both
// holes are now closed: marker suppression is directory-only and never stops recursion, and the .pyc/
// .pyo suffix exclusion applies ONLY when the parent directory basename is exactly __pycache__.
// ---------------------------------------------------------------------------
test('#3631 (security pin — closes HIGH: excluded-dir recursion skip): __pycache__/run.js DOES change the hash', (t) => {
// Pins the HIGH finding: skipping recursion into __pycache__ made it a permanently-unhashed region
// that a declared hook script could point into. A non-.pyc file inside __pycache__ must still bind.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, '__pycache__'), { recursive: true });
fs.writeFileSync(path.join(dir, '__pycache__', 'run.js'), 'module.exports = 1;\n', 'utf8');
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, 'a non-.pyc file inside __pycache__ must still change the hash');
});
test('#3631 (security pin — closes sourceless-legacy-.pyc vector): scripts/x.pyc DOES change the hash', (t) => {
// A .pyc whose parent is NOT __pycache__ is a legacy sourceless bytecode file CPython can still
// import directly — it must stay bound to the digest, regardless of how deep it is nested.
const dir = makeBundle({});
t.after(() => cleanup(dir));
fs.mkdirSync(path.join(dir, 'scripts'), { recursive: true });
const before = consent.bundleContentHash(dir);
fs.writeFileSync(path.join(dir, 'scripts', 'x.pyc'), Buffer.from([1, 2, 3]));
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, 'scripts/x.pyc (parent is not __pycache__) must still change the hash');
});
test('#3631 (security pin — suffix rule is file-only): a DIRECTORY named cache.pyc/ containing inner.js DOES change the hash', (t) => {
// Pins that the .pyc/.pyo suffix rule never applies to directories — a directory literally named
// cache.pyc gets no marker suppression and no exclusion; its contents bind normally.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, 'cache.pyc'), { recursive: true });
fs.writeFileSync(path.join(dir, 'cache.pyc', 'inner.js'), 'module.exports = 1;\n', 'utf8');
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, 'a directory named cache.pyc must be hashed normally (marker + recursion), not excluded');
});
test('#3631 (security pin — recursion is unbounded depth): __pycache__/sub/deep.js DOES change the hash', (t) => {
// Proves recursion into an excluded dir goes all the way down, not just one level — a nested
// non-.pyc file several directories under __pycache__ must still bind.
const dir = makeBundle({});
t.after(() => cleanup(dir));
const before = consent.bundleContentHash(dir);
fs.mkdirSync(path.join(dir, '__pycache__', 'sub'), { recursive: true });
fs.writeFileSync(path.join(dir, '__pycache__', 'sub', 'deep.js'), 'module.exports = 1;\n', 'utf8');
const after = consent.bundleContentHash(dir);
assert.notStrictEqual(after, before, 'a nested non-.pyc file under __pycache__ must still change the hash, at any depth');
});
void crypto; // reserved import; keep explicit.

76
tests/capability-registry.test.cjs
View File

@@ -1918,6 +1918,82 @@ describe('C4: description and hooks validation', () => {
assert.deepEqual(hookErrors, [], 'Expected a normal nested relative script to be accepted, got: ' + JSON.stringify(hookErrors));
});
// ─── #3631 (defense-in-depth): a declared hook script path must not point into the space
// bundleContentHash excludes from the consent digest. A file named `x.pyc` can contain valid
// JavaScript and would be executed by `node` regardless of extension, and a __pycache__/
// .pytest_cache path segment marks digest-excluded space — so a manifest-declared executable
// surface must never be able to reach either. ───
for (const [label, script] of [
['__pycache__ path segment', '__pycache__/run.js'],
['.pytest_cache path segment', '.pytest_cache/run.js'],
['.pyc basename suffix', 'hooks/x.pyc'],
]) {
test(`hook script pointing into digest-excluded space is rejected (${label})`, () => {
const cap = { ...UI_CAP, hooks: [{ event: 'PostToolUse', script }] };
const errors = validateCapability(cap, 'ui');
const hookErrors = errors.filter((e) => e.includes('hooks[0].script'));
assert.ok(
hookErrors.length > 0,
`Expected a hooks[0].script rejection for ${label} (script=${JSON.stringify(script)}), got: ` + JSON.stringify(errors),
);
});
}
test('hook script not pointing into digest-excluded space is still accepted (hooks/check.js)', () => {
const cap = { ...UI_CAP, hooks: [{ event: 'PostToolUse', script: 'hooks/check.js' }] };
const errors = validateCapability(cap, 'ui');
const hookErrors = errors.filter((e) => e.includes('hooks[0].script'));
assert.deepEqual(hookErrors, [], 'Expected a normal .js hook script to be accepted, got: ' + JSON.stringify(hookErrors));
});
// ─── Generative-fix-divergence parity: `isSafeHookScriptPath` is duplicated hand-maintained
// logic in src/capability-lifecycle.cts (via `confinedBundleScript`, the nearest exported
// consumer) and gsd-core/bin/lib/capability-validator.cjs (via `validateCapability`, the
// nearest exported consumer). There is no src/capability-validator.cts — the .cjs is hand
// -maintained — so this drives BOTH real copies behaviorally (never source-greps either file)
// and asserts they agree on every path, catching the two implementations drifting apart. #3631
// finding 1 removed the `.DS_Store` DIGEST exclusion, but the validator never rejected
// `.DS_Store` on either side — `hooks/.DS_Store` is expected to be ACCEPTED by both.
test('isSafeHookScriptPath parity: capability-lifecycle.cts and capability-validator.cjs agree on every path', () => {
const { confinedBundleScript } = require('../gsd-core/bin/lib/capability-lifecycle.cjs');
// A capDir that does not exist on disk: confinedBundleScript falls into its lexical
// (does-not-exist-yet) branch, so the verdict reflects ONLY isSafeHookScriptPath — never a
// realpath/confinement side effect unrelated to what is under test here.
const fakeCapDir = path.join(os.tmpdir(), 'gsd-parity-probe-nonexistent-cap-dir');
const cases = [
['hooks/check.js', true],
['__pycache__/run.js', false],
['hooks/__pycache__/run.js', false],
['.pytest_cache/run.js', false],
['hooks/x.pyc', false],
['x.pyo', false],
['hooks/x.PYC', false],
['hooks/.DS_Store', true],
['hooks/../__pycache__/run.js', false],
['hooks/__pycache__./run.js', true],
['__PYCACHE__/run.js', true],
['hooks\\__pycache__\\run.js', false],
];
for (const [script, expectedAccept] of cases) {
const cap = { ...UI_CAP, hooks: [{ event: 'PostToolUse', script }] };
const errors = validateCapability(cap, 'ui');
const cjsAccept = errors.filter((e) => e.includes('hooks[0].script')).length === 0;
const ctsAccept = confinedBundleScript(fakeCapDir, script) !== null;
assert.strictEqual(
cjsAccept,
ctsAccept,
`capability-validator.cjs (accept=${cjsAccept}) and capability-lifecycle.cts (accept=${ctsAccept}) disagree on ${JSON.stringify(script)}`,
);
assert.strictEqual(
cjsAccept,
expectedAccept,
`expected accept=${expectedAccept} for ${JSON.stringify(script)}, both copies returned accept=${cjsAccept}`,
);
}
});
test('description present in UI_CAP passes validation', () => {
const errors = validateCapability(UI_CAP, 'ui');
const descErrors = errors.filter((e) => e.includes('description'));