workflow.security_asvs_level was display-only — the planner hardcoded 'mitigate if ASVS L1 requires it' and the auditor only echoed the level, so L2/L3 behaved identically to L1. - New reference gsd-core/references/security-asvs-levels.md defines L1 (opportunistic), L2 (standard), L3 (comprehensive) for both planner threat disposition and auditor verification depth (higher = superset). - planner: disposition now scales with the configured ASVS level (no hardcoded L1) + @-pointer to the reference. - auditor: verification depth scales with asvs_level (L1 grep-presence, L2 boundary/vector check, L3 end-to-end trace + bypass check). - planning-config.md + INVENTORY updated; planner kept under its 48K cap by extracting the goal-backward worked example to planner-guidance.md. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
5
.changeset/sunny-newts-dance.md
Normal file
5
.changeset/sunny-newts-dance.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Fixed
|
||||
pr: 1636
|
||||
---
|
||||
**`workflow.security_asvs_level` now actually scales security rigor** — it was display-only (the planner hardcoded ASVS L1 and the auditor only echoed the level), so L2/L3 behaved identically to L1. The configured ASVS level now scales both planner threat-disposition rigor and auditor verification depth (L1 grep-presence → L2 boundary/vector checks → L3 end-to-end trace), defined in a new `references/security-asvs-levels.md`; the secure-phase clean-phase short-circuit now spawns the auditor at L2/L3 so deep verification runs even when the preliminary grep classification is clean. (#1627)
|
||||
@@ -459,7 +459,7 @@ Only include what Claude literally cannot do.
|
||||
**Step 0: Extract Requirement IDs**
|
||||
Read ROADMAP.md `**Requirements:**` line for this phase. Strip brackets if present (e.g., `[AUTH-01, AUTH-02]` → `AUTH-01, AUTH-02`). Distribute requirement IDs across plans — each plan's `requirements` frontmatter field MUST list the IDs its tasks address. **CRITICAL:** Every requirement ID MUST appear in at least one plan. Plans with an empty `requirements` field are invalid.
|
||||
|
||||
**Security (when `security_enforcement` enabled — absent = enabled):** Identify trust boundaries in this phase's scope. Map STRIDE categories to applicable tech stack from RESEARCH.md security domain. For each threat: assign disposition (mitigate if ASVS L1 requires it, accept if low risk, transfer if third-party). Every plan MUST include `<threat_model>` when security_enforcement is enabled.
|
||||
**Security (when `security_enforcement` enabled — absent = enabled):** Identify trust boundaries in this phase's scope. Map STRIDE categories to applicable tech stack from RESEARCH.md security domain. For each threat: assign disposition (`mitigate`/`accept`/`transfer`) per the configured OWASP ASVS level — see @~/.claude/gsd-core/references/security-asvs-levels.md. Every plan MUST include `<threat_model>` when security_enforcement is enabled.
|
||||
|
||||
**Package legitimacy gate (npm/pip/cargo only):**
|
||||
- Require RESEARCH.md `## Package Legitimacy Audit` before package-manager install tasks.
|
||||
@@ -477,66 +477,16 @@ Take phase goal from ROADMAP.md. Must be outcome-shaped, not task-shaped.
|
||||
**Step 2: Derive Observable Truths**
|
||||
"What must be TRUE for this goal to be achieved?" List 3-7 truths from USER's perspective.
|
||||
|
||||
For "working chat interface":
|
||||
- User can see existing messages
|
||||
- User can type a new message
|
||||
- User can send the message
|
||||
- Sent message appears in the list
|
||||
- Messages persist across page refresh
|
||||
|
||||
**Test:** Each truth verifiable by a human using the application.
|
||||
|
||||
**Step 3: Derive Required Artifacts**
|
||||
For each truth: "What must EXIST for this to be true?"
|
||||
|
||||
"User can see existing messages" requires:
|
||||
- Message list component (renders Message[])
|
||||
- Messages state (loaded from somewhere)
|
||||
- API route or data source (provides messages)
|
||||
- Message type definition (shapes the data)
|
||||
|
||||
**Test:** Each artifact = a specific file or database object.
|
||||
|
||||
**Step 4: Derive Required Wiring**
|
||||
For each artifact: "What must be CONNECTED for this to function?"
|
||||
|
||||
Message list component wiring:
|
||||
- Imports Message type (not using `any`)
|
||||
- Receives messages prop or fetches from API
|
||||
- Maps over messages to render (not hardcoded)
|
||||
- Handles empty state (not just crashes)
|
||||
|
||||
**Step 5: Identify Key Links**
|
||||
"Where is this most likely to break?" Key links = critical connections where breakage causes cascading failures.
|
||||
|
||||
## Must-Haves Output Format
|
||||
|
||||
```yaml
|
||||
must_haves:
|
||||
truths:
|
||||
- "User can see existing messages"
|
||||
- "User can send a message"
|
||||
- "Messages persist across refresh"
|
||||
artifacts:
|
||||
- path: "src/components/Chat.tsx"
|
||||
provides: "Message list rendering"
|
||||
min_lines: 30
|
||||
- path: "src/app/api/chat/route.ts"
|
||||
provides: "Message CRUD operations"
|
||||
exports: ["GET", "POST"]
|
||||
- path: "prisma/schema.prisma"
|
||||
provides: "Message model"
|
||||
contains: "model Message"
|
||||
key_links:
|
||||
- from: "src/components/Chat.tsx"
|
||||
to: "src/app/api/chat/route.ts"
|
||||
via: "fetch in useEffect — calls /api/chat endpoint"
|
||||
pattern: "fetch.*api/chat"
|
||||
- from: "src/app/api/chat/route.ts"
|
||||
to: "prisma/schema.prisma"
|
||||
via: "database query via prisma.message"
|
||||
pattern: "prisma\\.message\\.(find|create)"
|
||||
```
|
||||
See @~/.claude/gsd-core/references/planner-guidance.md for a worked example and the `must_haves` YAML format.
|
||||
|
||||
</goal_backward>
|
||||
|
||||
|
||||
@@ -69,10 +69,15 @@ For each threat in `<threat_model>`, determine verification method by dispositio
|
||||
| `transfer` | Verify transfer documentation present (insurance, vendor SLA, etc.) |
|
||||
|
||||
Classify each threat before verification. Record classification for every threat — no threat skipped.
|
||||
|
||||
**Verification depth scales with `asvs_level`** (see @~/.claude/gsd-core/references/security-asvs-levels.md for full definitions):
|
||||
- L1: verify mitigation is PRESENT in the cited file (grep-level — pattern exists).
|
||||
- L2: verify the mitigation ADDRESSES the threat vector and is placed at the correct boundary (a check in the wrong layer does not close the threat).
|
||||
- L3: deep trace — follow the data flow end-to-end, check edge cases and ordering, confirm no bypass path exists.
|
||||
</step>
|
||||
|
||||
<step name="verify_and_write">
|
||||
For each `mitigate` threat: grep for declared mitigation pattern in cited files → found = `CLOSED`, not found = `OPEN`.
|
||||
For each `mitigate` threat: grep for declared mitigation pattern in cited files → found = `CLOSED`, not found = `OPEN`. Apply depth per `asvs_level` (see analyze_threats step).
|
||||
For `accept` threats: check SECURITY.md accepted risks log → entry present = `CLOSED`, absent = `OPEN`.
|
||||
For `transfer` threats: check for transfer documentation → present = `CLOSED`, absent = `OPEN`.
|
||||
|
||||
|
||||
@@ -250,6 +250,7 @@
|
||||
"research-verification-protocol.md",
|
||||
"revision-loop.md",
|
||||
"scout-codebase.md",
|
||||
"security-asvs-levels.md",
|
||||
"skeleton-template.md",
|
||||
"sketch-interactivity.md",
|
||||
"sketch-theme-system.md",
|
||||
|
||||
@@ -283,6 +283,7 @@ Full roster at `gsd-core/references/*.md`. References are shared knowledge docum
|
||||
| `verification-patterns.md` | How to verify different artifact types. |
|
||||
| `verification-overrides.md` | Per-artifact verification override rules. |
|
||||
| `planning-config.md` | Full config schema and behavior. |
|
||||
| `security-asvs-levels.md` | OWASP ASVS level definitions for GSD threat modeling — per-level planner disposition rigor and auditor verification depth (L1 opportunistic, L2 standard, L3 comprehensive). |
|
||||
| `git-integration.md` | Git commit, branching, and history patterns. |
|
||||
| `git-planning-commit.md` | Planning directory commit conventions. |
|
||||
| `questioning.md` | Dream-extraction philosophy for project initialization. |
|
||||
|
||||
@@ -184,3 +184,69 @@ Execute: `/gsd:execute-phase {phase} --gaps-only`
|
||||
## Checkpoint Reached / Revision Complete
|
||||
|
||||
Follow templates in checkpoints and revision_mode sections respectively.
|
||||
|
||||
---
|
||||
|
||||
## Goal-Backward Worked Example
|
||||
|
||||
### Step 2: Derive Observable Truths
|
||||
|
||||
For "working chat interface":
|
||||
- User can see existing messages
|
||||
- User can type a new message
|
||||
- User can send the message
|
||||
- Sent message appears in the list
|
||||
- Messages persist across page refresh
|
||||
|
||||
**Test:** Each truth verifiable by a human using the application.
|
||||
|
||||
### Step 3: Derive Required Artifacts
|
||||
|
||||
"User can see existing messages" requires:
|
||||
- Message list component (renders Message[])
|
||||
- Messages state (loaded from somewhere)
|
||||
- API route or data source (provides messages)
|
||||
- Message type definition (shapes the data)
|
||||
|
||||
**Test:** Each artifact = a specific file or database object.
|
||||
|
||||
### Step 4: Derive Required Wiring
|
||||
|
||||
Message list component wiring:
|
||||
- Imports Message type (not using `any`)
|
||||
- Receives messages prop or fetches from API
|
||||
- Maps over messages to render (not hardcoded)
|
||||
- Handles empty state (not just crashes)
|
||||
|
||||
### Step 5: Identify Key Links
|
||||
|
||||
"Where is this most likely to break?" Key links = critical connections where breakage causes cascading failures.
|
||||
|
||||
### Must-Haves Output Format
|
||||
|
||||
```yaml
|
||||
must_haves:
|
||||
truths:
|
||||
- "User can see existing messages"
|
||||
- "User can send a message"
|
||||
- "Messages persist across refresh"
|
||||
artifacts:
|
||||
- path: "src/components/Chat.tsx"
|
||||
provides: "Message list rendering"
|
||||
min_lines: 30
|
||||
- path: "src/app/api/chat/route.ts"
|
||||
provides: "Message CRUD operations"
|
||||
exports: ["GET", "POST"]
|
||||
- path: "prisma/schema.prisma"
|
||||
provides: "Message model"
|
||||
contains: "model Message"
|
||||
key_links:
|
||||
- from: "src/components/Chat.tsx"
|
||||
to: "src/app/api/chat/route.ts"
|
||||
via: "fetch in useEffect — calls /api/chat endpoint"
|
||||
pattern: "fetch.*api/chat"
|
||||
- from: "src/app/api/chat/route.ts"
|
||||
to: "prisma/schema.prisma"
|
||||
via: "database query via prisma.message"
|
||||
pattern: "prisma\\.message\\.(find|create)"
|
||||
```
|
||||
|
||||
@@ -275,7 +275,7 @@ Set via `workflow.*` namespace in config.json (e.g., `"workflow": { "research":
|
||||
| `workflow.code_review_depth` | string | `"standard"` | `"light"`, `"standard"`, `"deep"` | Depth level for code review analysis in the ship workflow |
|
||||
| `workflow._auto_chain_active` | boolean | `false` | `true`, `false` | Internal: tracks whether autonomous chaining is active |
|
||||
| `workflow.security_enforcement` | boolean | `true` | `true`, `false` | Enable threat-model-anchored security verification via `/gsd:secure-phase`. When `false`, security checks are skipped entirely |
|
||||
| `workflow.security_asvs_level` | number | `1` | `1`, `2`, `3` | OWASP ASVS verification level. Level 1 = opportunistic, Level 2 = standard, Level 3 = comprehensive |
|
||||
| `workflow.security_asvs_level` | number | `1` | `1`, `2`, `3` | OWASP ASVS verification level. Level 1 = opportunistic, Level 2 = standard, Level 3 = comprehensive. Scales both planner threat-disposition rigor (which threats must be mitigated vs. accepted) and auditor verification depth (grep-level → boundary-placement check → full data-flow trace). See `gsd-core/references/security-asvs-levels.md`. |
|
||||
| `workflow.security_block_on` | string | `"high"` | `"high"`, `"medium"`, `"low"` | Minimum severity that blocks phase advancement |
|
||||
| `workflow.post_planning_gaps` | boolean | `true` | `true`, `false` | Post-planning gap report (#2493). After plans are generated, scans REQUIREMENTS.md and CONTEXT.md `<decisions>` against all PLAN.md files and emits a unified `Source \| Item \| Status` table. Non-blocking. Set to `false` to skip Step 13e of plan-phase. _Alias:_ `post_planning_gaps` is the flat-key form used in `CONFIG_DEFAULTS`; `workflow.post_planning_gaps` is the canonical namespaced form. |
|
||||
|
||||
|
||||
27
gsd-core/references/security-asvs-levels.md
Normal file
27
gsd-core/references/security-asvs-levels.md
Normal file
@@ -0,0 +1,27 @@
|
||||
# Security ASVS Levels
|
||||
|
||||
GSD threat modeling maps OWASP ASVS levels to planner disposition rigor and auditor verification depth. Higher levels are supersets of lower — L3 includes all L2 and L1 requirements.
|
||||
|
||||
## L1 — Opportunistic (default)
|
||||
|
||||
**Scope:** Cover threats on primary trust boundaries and high-impact components.
|
||||
|
||||
**Planner disposition:** `mitigate` critical/high-severity threats. `mitigate` medium-severity threats if they occur on a primary trust boundary; otherwise `accept` with documented rationale explaining the specific risk tolerance. `accept` low-risk threats with a rationale statement. `transfer` when threat is third-party responsibility.
|
||||
|
||||
**Auditor verification depth:** Verify each declared mitigation is PRESENT in the cited file (grep-level check — find the pattern, confirm the call exists).
|
||||
|
||||
## L2 — Standard
|
||||
|
||||
**Scope:** Map ALL applicable STRIDE categories for every in-scope component.
|
||||
|
||||
**Planner disposition:** `mitigate` medium-severity-and-above threats. Every `accept` MUST have explicit documented rationale explaining why the risk is tolerable for this specific context.
|
||||
|
||||
**Auditor verification depth:** Verify the mitigation ACTUALLY ADDRESSES the threat vector (not just that some pattern is present) and is placed at the correct trust boundary. A login check in the wrong layer does not close the threat.
|
||||
|
||||
## L3 — Comprehensive
|
||||
|
||||
**Scope:** Exhaustive STRIDE × all components; defense-in-depth for critical threats.
|
||||
|
||||
**Planner disposition:** `mitigate` all threats except those explicitly accepted with documented sign-off. Defense-in-depth layers required for critical threats (multiple independent controls).
|
||||
|
||||
**Auditor verification depth:** Deep verification — trace data flow end-to-end, check edge cases and ordering, confirm the mitigation cannot be bypassed via alternate code paths or parameter manipulation.
|
||||
@@ -77,7 +77,8 @@ Classify each threat:
|
||||
Build: `{ threat_id, category, component, disposition, status, evidence }`
|
||||
|
||||
**Short-circuit rule:**
|
||||
- If `threats_open: 0 AND register_authored_at_plan_time: true` → skip to Step 6 directly. All plan-time threats are verified CLOSED.
|
||||
- If `threats_open: 0 AND register_authored_at_plan_time: true AND asvs_level == 1` → skip to Step 6 directly. All plan-time threats are verified CLOSED at L1 grep-depth; no deeper verification required.
|
||||
- If `threats_open: 0 AND register_authored_at_plan_time: true AND asvs_level >= 2` → **do NOT skip**. The preliminary threat classification is grep-level (L1 depth) and is insufficient for L2/L3. Proceed to Step 5 (spawn the auditor) so that L2 boundary-placement checks and L3 end-to-end trace checks are performed. Skipping the auditor here would defeat ASVS level scaling for "clean" phases.
|
||||
- If `threats_open: 0 AND register_authored_at_plan_time: false` → **do NOT skip**. Empty-by-no-planning must not rubber-stamp a clean SECURITY.md. Proceed to Step 5 in **retroactive-STRIDE mode** — the auditor builds a register from implementation files first, then verifies mitigations.
|
||||
- If `threats_open > 0` → proceed to Step 4 (present threat plan to user).
|
||||
|
||||
@@ -177,7 +178,8 @@ Display `/clear` reminder.
|
||||
- [ ] Input state detected (A/B/C) — state C exits cleanly
|
||||
- [ ] PLAN.md threat model parsed, register built
|
||||
- [ ] SUMMARY.md threat flags incorporated
|
||||
- [ ] threats_open: 0 AND register_authored_at_plan_time: true → skip directly to Step 6
|
||||
- [ ] threats_open: 0 AND register_authored_at_plan_time: true AND asvs_level == 1 → skip directly to Step 6 (L1 grep-depth sufficient)
|
||||
- [ ] threats_open: 0 AND register_authored_at_plan_time: true AND asvs_level >= 2 → do NOT skip; auditor spawned for L2/L3 deep verification
|
||||
- [ ] threats_open: 0 AND register_authored_at_plan_time: false → retroactive-STRIDE mode (Step 5), not skipped
|
||||
- [ ] User gate with threat table presented
|
||||
- [ ] Auditor spawned with complete context
|
||||
|
||||
@@ -23,11 +23,11 @@
|
||||
"gsd-pattern-mapper.md": 12487,
|
||||
"gsd-phase-researcher.md": 40638,
|
||||
"gsd-plan-checker.md": 44646,
|
||||
"gsd-planner.md": 49306,
|
||||
"gsd-planner.md": 47865,
|
||||
"gsd-project-researcher.md": 22014,
|
||||
"gsd-research-synthesizer.md": 13653,
|
||||
"gsd-roadmapper.md": 22183,
|
||||
"gsd-security-auditor.md": 6226,
|
||||
"gsd-security-auditor.md": 6767,
|
||||
"gsd-ui-auditor.md": 17159,
|
||||
"gsd-ui-checker.md": 11088,
|
||||
"gsd-ui-researcher.md": 19272,
|
||||
|
||||
233
tests/fix-1627-asvs-level-scaling.test.cjs
Normal file
233
tests/fix-1627-asvs-level-scaling.test.cjs
Normal file
@@ -0,0 +1,233 @@
|
||||
// allow-test-rule: source-text-is-the-product #1627
|
||||
// Agent .md / reference .md files — their text IS what the runtime loads.
|
||||
// Testing text content tests the deployed contract.
|
||||
// Per CONTRIBUTING.md exception matrix.
|
||||
|
||||
/**
|
||||
* Fix #1627 — ASVS level scaling
|
||||
*
|
||||
* Asserts that `workflow.security_asvs_level` now scales both planner
|
||||
* threat-disposition rigor and auditor verification depth rather than
|
||||
* being display-only.
|
||||
*/
|
||||
|
||||
'use strict';
|
||||
|
||||
const { describe, test } = require('node:test');
|
||||
const assert = require('node:assert/strict');
|
||||
const fs = require('node:fs');
|
||||
const path = require('node:path');
|
||||
|
||||
const ROOT = path.join(__dirname, '..');
|
||||
const AGENTS_DIR = path.join(ROOT, 'agents');
|
||||
const REFS_DIR = path.join(ROOT, 'gsd-core', 'references');
|
||||
const MANIFEST_PATH = path.join(ROOT, 'docs', 'INVENTORY-MANIFEST.json');
|
||||
|
||||
describe('SECURE: ASVS level scaling (#1627)', () => {
|
||||
// ── 1. New reference file ────────────────────────────────────────────────
|
||||
|
||||
describe('security-asvs-levels.md reference', () => {
|
||||
const refPath = path.join(REFS_DIR, 'security-asvs-levels.md');
|
||||
|
||||
test('file exists', () => {
|
||||
assert.ok(fs.existsSync(refPath), 'gsd-core/references/security-asvs-levels.md must exist');
|
||||
});
|
||||
|
||||
test('defines all three levels', () => {
|
||||
const content = fs.readFileSync(refPath, 'utf-8');
|
||||
assert.ok(content.includes('L1'), 'must define L1');
|
||||
assert.ok(content.includes('L2'), 'must define L2');
|
||||
assert.ok(content.includes('L3'), 'must define L3');
|
||||
});
|
||||
|
||||
test('L1 describes opportunistic scope and planner disposition', () => {
|
||||
const content = fs.readFileSync(refPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.toLowerCase().includes('opportunistic'),
|
||||
'L1 must be described as opportunistic'
|
||||
);
|
||||
assert.ok(
|
||||
content.includes('mitigate') && content.includes('accept'),
|
||||
'must describe mitigate/accept dispositions'
|
||||
);
|
||||
});
|
||||
|
||||
test('L2 requires explicit rationale for accepted threats', () => {
|
||||
const content = fs.readFileSync(refPath, 'utf-8');
|
||||
// L2 must require documented rationale for accepted risks
|
||||
assert.ok(
|
||||
content.includes('rationale') || content.includes('documented'),
|
||||
'L2 must require documented rationale for accepted threats'
|
||||
);
|
||||
});
|
||||
|
||||
test('L3 describes deep/comprehensive verification', () => {
|
||||
const content = fs.readFileSync(refPath, 'utf-8');
|
||||
const lower = content.toLowerCase();
|
||||
assert.ok(
|
||||
lower.includes('deep') || lower.includes('comprehensive') || lower.includes('exhaustive'),
|
||||
'L3 must describe deep/comprehensive verification'
|
||||
);
|
||||
});
|
||||
|
||||
test('mentions that higher levels are supersets of lower', () => {
|
||||
const content = fs.readFileSync(refPath, 'utf-8');
|
||||
const lower = content.toLowerCase();
|
||||
assert.ok(
|
||||
lower.includes('superset') || lower.includes('higher level') || lower.includes('includes all'),
|
||||
'must note that higher levels are supersets of lower'
|
||||
);
|
||||
});
|
||||
|
||||
test('describes distinct auditor verification depth for each level', () => {
|
||||
const content = fs.readFileSync(refPath, 'utf-8');
|
||||
// All three audit depth keywords should appear
|
||||
assert.ok(content.includes('grep') || content.includes('PRESENT'), 'L1 audit depth must mention grep/presence check');
|
||||
assert.ok(content.includes('boundary') || content.includes('addresses'), 'L2 audit depth must mention boundary/addresses');
|
||||
assert.ok(content.includes('end-to-end') || content.includes('bypass'), 'L3 audit depth must mention end-to-end or bypass check');
|
||||
});
|
||||
});
|
||||
|
||||
// ── 2. gsd-planner.md — no hardcoded L1 in disposition ──────────────────
|
||||
|
||||
describe('gsd-planner.md security disposition', () => {
|
||||
const plannerPath = path.join(AGENTS_DIR, 'gsd-planner.md');
|
||||
|
||||
test('planner security instruction does not hardcode "ASVS L1"', () => {
|
||||
const content = fs.readFileSync(plannerPath, 'utf-8');
|
||||
// The old bug: "mitigate if ASVS L1 requires it" — must be gone
|
||||
assert.ok(
|
||||
!content.includes('ASVS L1 requires it'),
|
||||
'planner must not hardcode "ASVS L1 requires it"; it must reference the configured level'
|
||||
);
|
||||
});
|
||||
|
||||
test('planner references the configured OWASP ASVS level', () => {
|
||||
const content = fs.readFileSync(plannerPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('OWASP ASVS level') || content.includes('configured OWASP'),
|
||||
'planner must reference the configured OWASP ASVS level'
|
||||
);
|
||||
});
|
||||
|
||||
test('planner @-references security-asvs-levels.md', () => {
|
||||
const content = fs.readFileSync(plannerPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('security-asvs-levels.md'),
|
||||
'planner must @-reference security-asvs-levels.md'
|
||||
);
|
||||
});
|
||||
|
||||
test('planner is under the 49152-char cap', () => {
|
||||
const content = fs.readFileSync(plannerPath, 'utf-8').replace(/\r\n/g, '\n').replace(/\r/g, '\n');
|
||||
assert.ok(
|
||||
content.length < 49152,
|
||||
`gsd-planner.md must be < 49152 chars (LF-normalized); got ${content.length}`
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 3. gsd-security-auditor.md — scaled verification depth ──────────────
|
||||
|
||||
describe('gsd-security-auditor.md verification depth', () => {
|
||||
const auditorPath = path.join(AGENTS_DIR, 'gsd-security-auditor.md');
|
||||
|
||||
test('auditor scales verification depth by asvs_level', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('asvs_level') || content.includes('ASVS level'),
|
||||
'auditor must reference asvs_level to scale verification'
|
||||
);
|
||||
});
|
||||
|
||||
test('auditor describes L1/L2/L3 depth differences', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
// All three levels must appear in context of depth scaling
|
||||
assert.ok(content.includes('L1'), 'auditor must mention L1 depth');
|
||||
assert.ok(content.includes('L2'), 'auditor must mention L2 depth');
|
||||
assert.ok(content.includes('L3'), 'auditor must mention L3 depth');
|
||||
});
|
||||
|
||||
test('auditor @-references security-asvs-levels.md', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('security-asvs-levels.md'),
|
||||
'auditor must @-reference security-asvs-levels.md'
|
||||
);
|
||||
});
|
||||
|
||||
test('auditor still echoes ASVS Level in structured output', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('ASVS Level:') && content.includes('{1/2/3}'),
|
||||
'auditor must still emit ASVS Level in SECURED/OPEN_THREATS output'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 4. secure-phase.md — ASVS-aware short-circuit ──────────────────────
|
||||
|
||||
describe('secure-phase.md short-circuit conditioned on asvs_level', () => {
|
||||
const wfPath = path.join(ROOT, 'gsd-core', 'workflows', 'secure-phase.md');
|
||||
|
||||
test('short-circuit to Step 6 is gated on asvs_level == 1', () => {
|
||||
const content = fs.readFileSync(wfPath, 'utf-8');
|
||||
// The condition must reference asvs_level so that L2/L3 don't skip the auditor
|
||||
assert.ok(
|
||||
content.includes('asvs_level == 1'),
|
||||
'secure-phase.md must gate the skip-to-Step-6 short-circuit on asvs_level == 1'
|
||||
);
|
||||
});
|
||||
|
||||
test('auditor runs at L2/L3 even when threats_open is 0 (asvs_level >= 2 branch present)', () => {
|
||||
const content = fs.readFileSync(wfPath, 'utf-8');
|
||||
// The >= 2 branch must explicitly say the auditor is spawned for L2/L3 deep verification
|
||||
assert.ok(
|
||||
content.includes('asvs_level >= 2'),
|
||||
'secure-phase.md must include asvs_level >= 2 branch that does NOT skip the auditor'
|
||||
);
|
||||
// The >= 2 branch must make clear the auditor is spawned (not skipped)
|
||||
assert.ok(
|
||||
content.includes('L2/L3 deep verification') || content.includes('L2 boundary') || content.includes('L3 end-to-end'),
|
||||
'secure-phase.md asvs_level >= 2 branch must reference L2/L3 deep verification'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 5. security-asvs-levels.md — L1 medium-severity gap closed ──────────
|
||||
|
||||
describe('security-asvs-levels.md L1 medium-severity is specified', () => {
|
||||
const refPath = path.join(REFS_DIR, 'security-asvs-levels.md');
|
||||
|
||||
test('L1 explicitly handles medium-severity threats (no gap)', () => {
|
||||
const content = fs.readFileSync(refPath, 'utf-8');
|
||||
// L1 section must say something about medium-severity
|
||||
assert.ok(
|
||||
content.includes('medium-severity') || content.includes('medium severity'),
|
||||
'L1 must explicitly specify disposition for medium-severity threats (no ambiguity gap)'
|
||||
);
|
||||
});
|
||||
|
||||
test('L1 medium-severity disposition is conditional (trust-boundary-aware)', () => {
|
||||
const content = fs.readFileSync(refPath, 'utf-8');
|
||||
// L1 must distinguish between medium on primary trust boundary vs not
|
||||
assert.ok(
|
||||
content.includes('trust boundary') || content.includes('primary trust'),
|
||||
'L1 medium-severity rule must reference trust boundary to disambiguate disposition'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ── 6. Inventory manifest ─────────────────────────────────────────────────
|
||||
|
||||
describe('inventory manifest', () => {
|
||||
test('security-asvs-levels.md is registered in INVENTORY-MANIFEST.json', () => {
|
||||
const manifest = JSON.parse(fs.readFileSync(MANIFEST_PATH, 'utf-8'));
|
||||
const refs = (manifest.families || {}).references || [];
|
||||
assert.ok(
|
||||
refs.includes('security-asvs-levels.md'),
|
||||
'security-asvs-levels.md must appear in families.references of INVENTORY-MANIFEST.json'
|
||||
);
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -65,7 +65,7 @@
|
||||
"resume-project.md": 17226,
|
||||
"review.md": 39404,
|
||||
"scan.md": 7688,
|
||||
"secure-phase.md": 12656,
|
||||
"secure-phase.md": 13315,
|
||||
"session-report.md": 4044,
|
||||
"settings-advanced.md": 39666,
|
||||
"settings-integrations.md": 15848,
|
||||
|
||||
Reference in New Issue
Block a user