workflow.security_block_on was documented as the minimum threat severity that blocks advancement, but threats carried no severity and the auditor's threats_open count (the SECURITY.md gate field) counted every open threat regardless of severity — so the threshold had no effect, and the auditor's block_on vocabulary (open/unregistered/none) did not even match the config enum (critical/high/medium/low/none). - planner: add a Severity column to the STRIDE threat register; assign severity per threat. - auditor: read severity; reconcile the <config> block_on domain to the severity enum; redefine threats_open as the count of OPEN threats whose severity is at or above block_on (none => 0). Below-threshold opens are reported as non-blocking and excluded from threats_open. - SECURITY.md template + planning-config.md reconciled. No gate-check site changed: threats_open == 0 stays the gate everywhere; only its computation is now severity-filtered. Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Tom Boucher
5
.changeset/kind-ravens-hum.md
Normal file
5
.changeset/kind-ravens-hum.md
Normal file
@@ -0,0 +1,5 @@
|
||||
---
|
||||
type: Fixed
|
||||
pr: 1635
|
||||
---
|
||||
**The security audit gate now respects `workflow.security_block_on` severity** — `/gsd:secure-phase` previously blocked phase advancement on *any* open threat regardless of severity, so the documented `security_block_on` threshold had no effect (and the auditor's block vocabulary didn't even match the config enum). Threats now carry a per-threat **Severity** (critical|high|medium|low), and only open threats at or above the configured `security_block_on` severity count toward the blocking gate (`SECURITY.md threats_open`); `none` disables blocking, and a missing/unparseable severity fails closed as critical. (#1626)
|
||||
@@ -380,11 +380,11 @@ Output: [Artifacts created]
|
||||
|
||||
## STRIDE Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|-------------|-----------------|
|
||||
| T-{phase}-01 | {S/T/R/I/D/E} | {function/endpoint/file} | mitigate | {specific: e.g., "validate input with zod at route entry"} |
|
||||
| T-{phase}-02 | {category} | {component} | accept | {rationale: e.g., "no PII, low-value target"} |
|
||||
| T-{phase}-SC | Tampering | npm/pip/cargo installs | mitigate | slopcheck + blocking human checkpoint for [ASSUMED]/[SUS] |
|
||||
| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |
|
||||
|-----------|----------|-----------|----------|-------------|-----------------|
|
||||
| T-{phase}-01 | {S/T/R/I/D/E} | {function/endpoint/file} | {critical\|high\|medium\|low} | mitigate | {specific mitigation action} |
|
||||
| T-{phase}-02 | {category} | {component} | low | accept | {rationale for acceptance} |
|
||||
| T-{phase}-SC | Tampering | npm/pip/cargo installs | high | mitigate | slopcheck + blocking human checkpoint for [ASSUMED]/[SUS] |
|
||||
</threat_model>
|
||||
|
||||
<verification>
|
||||
@@ -459,7 +459,7 @@ Only include what Claude literally cannot do.
|
||||
**Step 0: Extract Requirement IDs**
|
||||
Read ROADMAP.md `**Requirements:**` line for this phase. Strip brackets if present (e.g., `[AUTH-01, AUTH-02]` → `AUTH-01, AUTH-02`). Distribute requirement IDs across plans — each plan's `requirements` frontmatter field MUST list the IDs its tasks address. **CRITICAL:** Every requirement ID MUST appear in at least one plan. Plans with an empty `requirements` field are invalid.
|
||||
|
||||
**Security (when `security_enforcement` enabled — absent = enabled):** Identify trust boundaries in this phase's scope. Map STRIDE categories to applicable tech stack from RESEARCH.md security domain. For each threat: assign disposition (`mitigate`/`accept`/`transfer`) per the configured OWASP ASVS level — see @~/.claude/gsd-core/references/security-asvs-levels.md. Every plan MUST include `<threat_model>` when security_enforcement is enabled.
|
||||
**Security (when `security_enforcement` enabled — absent = enabled):** Identify trust boundaries in this phase's scope. Map STRIDE categories to applicable tech stack from RESEARCH.md security domain. For each threat: assign a **severity** (critical|high|medium|low) based on impact × likelihood, and a disposition (`mitigate`/`accept`/`transfer`) per the configured OWASP ASVS level — see @~/.claude/gsd-core/references/security-asvs-levels.md. Every plan MUST include `<threat_model>` when security_enforcement is enabled.
|
||||
|
||||
**Package legitimacy gate (npm/pip/cargo only):**
|
||||
- Require RESEARCH.md `## Package Legitimacy Audit` before package-manager install tasks.
|
||||
@@ -987,6 +987,7 @@ Phase planning complete when:
|
||||
- [ ] User knows next steps and wave structure
|
||||
- [ ] `<threat_model>` present with STRIDE register (when `security_enforcement` enabled)
|
||||
- [ ] Every threat has a disposition (mitigate / accept / transfer)
|
||||
- [ ] Every threat has a Severity (critical|high|medium|low)
|
||||
- [ ] Mitigations reference specific implementation (not generic advice)
|
||||
|
||||
## Gap Closure Mode
|
||||
|
||||
@@ -33,18 +33,19 @@ Does NOT scan blindly for new vulnerabilities. Verifies each threat in `<threat_
|
||||
- Marking CLOSED based on code structure ("looks like it validates input") without finding the actual validation call
|
||||
|
||||
**Required finding classification:**
|
||||
- **BLOCKER** — `OPEN_THREATS`: a declared mitigation is absent in implemented code; phase must not ship
|
||||
- **BLOCKER** — `OPEN_THREATS`: a declared mitigation is absent in implemented code AND the threat's severity ≥ `block_on` threshold; phase must not ship until resolved
|
||||
- **OPEN — non-blocking** — mitigation absent BUT the threat's severity is below the `block_on` threshold; tracked in SECURITY.md, does NOT count toward `threats_open`, does not block ship
|
||||
- **WARNING** — `unregistered_flag`: new attack surface appeared during implementation with no threat mapping
|
||||
Every threat must resolve to CLOSED, OPEN (BLOCKER), or documented accepted risk.
|
||||
Every threat must resolve to CLOSED, OPEN-blocking (severity ≥ block_on), OPEN-non-blocking (severity below block_on), or documented accepted risk.
|
||||
</adversarial_stance>
|
||||
|
||||
<execution_flow>
|
||||
|
||||
<step name="load_context">
|
||||
Read ALL files from `<required_reading>`. Extract:
|
||||
- PLAN.md `<threat_model>` block: full threat register with IDs, categories, dispositions, mitigation plans
|
||||
- PLAN.md `<threat_model>` block: full threat register with IDs, categories, severities, dispositions, mitigation plans
|
||||
- SUMMARY.md `## Threat Flags` section: new attack surface detected by executor during implementation
|
||||
- `<config>` block: `asvs_level` (1/2/3), `block_on` (open / unregistered / none)
|
||||
- `<config>` block: `asvs_level` (1/2/3), `block_on` (critical | high | medium | low | none) — severity ordering: critical > high > medium > low; none = never block
|
||||
- Implementation files: exports, auth patterns, input handling, data flows
|
||||
|
||||
**Context budget:** Load project skills first (lightweight). Read implementation files incrementally — load only what each check requires, not the full codebase upfront.
|
||||
@@ -60,7 +61,7 @@ This ensures project-specific patterns, conventions, and best practices are appl
|
||||
</step>
|
||||
|
||||
<step name="analyze_threats">
|
||||
For each threat in `<threat_model>`, determine verification method by disposition:
|
||||
For each threat in `<threat_model>`, read its `severity` field (critical|high|medium|low). If building the register retroactively (no `<threat_model>` in PLAN.md), assign a severity to each threat you construct based on impact × likelihood. Determine verification method by disposition:
|
||||
|
||||
| Disposition | Verification Method |
|
||||
|-------------|---------------------|
|
||||
@@ -83,7 +84,13 @@ For `transfer` threats: check for transfer documentation → present = `CLOSED`,
|
||||
|
||||
For each `threat_flag` in SUMMARY.md `## Threat Flags`: if maps to existing threat ID → informational. If no mapping → log as `unregistered_flag` in SECURITY.md (not a blocker).
|
||||
|
||||
Write SECURITY.md. Set `threats_open` count. Return structured result.
|
||||
**Severity-aware `threats_open` computation (severity order: critical > high > medium > low):**
|
||||
`threats_open` (the SECURITY.md frontmatter gate field) = the count of threats whose status is OPEN AND whose severity rank ≥ the `block_on` rank. `block_on: none` ⇒ 0 (nothing ever blocks). `block_on: low` ⇒ all open threats block. `block_on: high` (default) ⇒ only high and critical open threats block.
|
||||
Open threats BELOW the block threshold are recorded in SECURITY.md as **open — below {block_on} threshold (non-blocking)** and MUST NOT be counted in `threats_open`.
|
||||
|
||||
**Fail-closed for missing severity:** if an OPEN threat has no severity or an unparseable severity (e.g. a legacy register predating the Severity column), treat it as `critical` for this computation — it COUNTS toward `threats_open` (blocking). Never silently drop an unranked open threat.
|
||||
|
||||
Write SECURITY.md. Set `threats_open` to the severity-filtered count. Return structured result.
|
||||
</step>
|
||||
|
||||
</execution_flow>
|
||||
@@ -100,9 +107,9 @@ Write SECURITY.md. Set `threats_open` count. Return structured result.
|
||||
**ASVS Level:** {1/2/3}
|
||||
|
||||
### Threat Verification
|
||||
| Threat ID | Category | Disposition | Evidence |
|
||||
|-----------|----------|-------------|----------|
|
||||
| {id} | {category} | {mitigate/accept/transfer} | {file:line or doc reference} |
|
||||
| Threat ID | Category | Severity | Disposition | Evidence |
|
||||
|-----------|----------|----------|-------------|----------|
|
||||
| {id} | {category} | {critical\|high\|medium\|low} | {mitigate/accept/transfer} | {file:line or doc reference} |
|
||||
|
||||
### Unregistered Flags
|
||||
{none / list from SUMMARY.md ## Threat Flags with no threat mapping}
|
||||
@@ -120,14 +127,21 @@ SECURITY.md: {path}
|
||||
**ASVS Level:** {1/2/3}
|
||||
|
||||
### Closed
|
||||
| Threat ID | Category | Disposition | Evidence |
|
||||
|-----------|----------|-------------|----------|
|
||||
| {id} | {category} | {disposition} | {evidence} |
|
||||
| Threat ID | Category | Severity | Disposition | Evidence |
|
||||
|-----------|----------|----------|-------------|----------|
|
||||
| {id} | {category} | {critical\|high\|medium\|low} | {disposition} | {evidence} |
|
||||
|
||||
### Open
|
||||
| Threat ID | Category | Mitigation Expected | Files Searched |
|
||||
|-----------|----------|---------------------|----------------|
|
||||
| {id} | {category} | {pattern not found} | {file paths} |
|
||||
### Open (blocking — severity ≥ block_on threshold)
|
||||
| Threat ID | Category | Severity | Mitigation Expected | Files Searched |
|
||||
|-----------|----------|----------|---------------------|----------------|
|
||||
| {id} | {category} | {critical\|high\|medium\|low} | {pattern not found} | {file paths} |
|
||||
|
||||
### Open (non-blocking — severity below block_on threshold)
|
||||
| Threat ID | Category | Severity | Mitigation Expected | Files Searched |
|
||||
|-----------|----------|----------|---------------------|----------------|
|
||||
| {id} | {category} | {critical\|high\|medium\|low} | {pattern not found} | {file paths} |
|
||||
|
||||
*Only blocking-open threats count toward `threats_open` in SECURITY.md frontmatter.*
|
||||
|
||||
Next: Implement mitigations or document as accepted in SECURITY.md accepted risks log, then re-run /gsd:secure-phase.
|
||||
|
||||
|
||||
@@ -830,7 +830,7 @@ These keys live under `workflow.*` — that is where the workflows and installer
|
||||
|---------|------|---------|-------------|
|
||||
| `workflow.security_enforcement` | boolean | `true` | Enable threat-model-anchored security verification via `/gsd-secure-phase`. When `false`, security checks are skipped entirely |
|
||||
| `workflow.security_asvs_level` | number (1-3) | `1` | OWASP ASVS verification level. Level 1 = opportunistic, Level 2 = standard, Level 3 = comprehensive |
|
||||
| `workflow.security_block_on` | string | `"high"` | Minimum severity that blocks phase advancement. Options: `"high"`, `"medium"`, `"low"` |
|
||||
| `workflow.security_block_on` | string | `"high"` | Minimum threat severity that blocks phase advancement. The auditor counts only open threats at or above this severity toward the blocking gate; `none` disables severity blocking. Options: `"critical"`, `"high"`, `"medium"`, `"low"`, `"none"` |
|
||||
|
||||
---
|
||||
|
||||
|
||||
@@ -276,7 +276,7 @@ Set via `workflow.*` namespace in config.json (e.g., `"workflow": { "research":
|
||||
| `workflow._auto_chain_active` | boolean | `false` | `true`, `false` | Internal: tracks whether autonomous chaining is active |
|
||||
| `workflow.security_enforcement` | boolean | `true` | `true`, `false` | Enable threat-model-anchored security verification via `/gsd:secure-phase`. When `false`, security checks are skipped entirely |
|
||||
| `workflow.security_asvs_level` | number | `1` | `1`, `2`, `3` | OWASP ASVS verification level. Level 1 = opportunistic, Level 2 = standard, Level 3 = comprehensive. Scales both planner threat-disposition rigor (which threats must be mitigated vs. accepted) and auditor verification depth (grep-level → boundary-placement check → full data-flow trace). See `gsd-core/references/security-asvs-levels.md`. |
|
||||
| `workflow.security_block_on` | string | `"high"` | `"high"`, `"medium"`, `"low"` | Minimum severity that blocks phase advancement |
|
||||
| `workflow.security_block_on` | string | `"high"` | `"critical"`, `"high"`, `"medium"`, `"low"`, `"none"` | Minimum threat severity that blocks phase advancement. The auditor counts only open threats at or above this severity toward the blocking gate (SECURITY.md `threats_open`); `none` disables severity blocking. |
|
||||
| `workflow.post_planning_gaps` | boolean | `true` | `true`, `false` | Post-planning gap report (#2493). After plans are generated, scans REQUIREMENTS.md and CONTEXT.md `<decisions>` against all PLAN.md files and emits a unified `Source \| Item \| Status` table. Non-blocking. Set to `false` to skip Step 13e of plan-phase. _Alias:_ `post_planning_gaps` is the flat-key form used in `CONFIG_DEFAULTS`; `workflow.post_planning_gaps` is the canonical namespaced form. |
|
||||
|
||||
### Ship Fields
|
||||
|
||||
@@ -2,6 +2,7 @@
|
||||
phase: {N}
|
||||
slug: {phase-slug}
|
||||
status: draft
|
||||
# threats_open = count of OPEN threats at or above workflow.security_block_on severity (the blocking gate)
|
||||
threats_open: 0
|
||||
asvs_level: 1
|
||||
created: {date}
|
||||
@@ -23,11 +24,12 @@ created: {date}
|
||||
|
||||
## Threat Register
|
||||
|
||||
| Threat ID | Category | Component | Disposition | Mitigation | Status |
|
||||
|-----------|----------|-----------|-------------|------------|--------|
|
||||
| T-{N}-01 | {STRIDE category} | {component} | {mitigate / accept / transfer} | {control or reference} | open |
|
||||
| Threat ID | Category | Component | Severity | Disposition | Mitigation | Status |
|
||||
|-----------|----------|-----------|----------|-------------|------------|--------|
|
||||
| T-{N}-01 | {STRIDE category} | {component} | {critical / high / medium / low} | {mitigate / accept / transfer} | {control or reference} | open |
|
||||
|
||||
*Status: open · closed*
|
||||
*Status: open · closed · open — below {block_on} threshold (non-blocking)*
|
||||
*Severity: critical > high > medium > low — only open threats at or above workflow.security_block_on count toward threats_open*
|
||||
*Disposition: mitigate (implementation required) · accept (documented risk) · transfer (third-party)*
|
||||
|
||||
---
|
||||
|
||||
@@ -53,7 +53,7 @@ SUMMARY_FILES=$(ls "${PHASE_DIR}"/*-SUMMARY.md 2>/dev/null)
|
||||
|
||||
### 2a. Read Phase Artifacts
|
||||
|
||||
Read PLAN.md — extract `<threat_model>` block: trust boundaries, STRIDE register (`threat_id`, `category`, `component`, `disposition`, `mitigation_plan`).
|
||||
Read PLAN.md — extract `<threat_model>` block: trust boundaries, STRIDE register (`threat_id`, `category`, `component`, `severity`, `disposition`, `mitigation_plan`).
|
||||
|
||||
### 2b. Read Summary Threat Flags
|
||||
|
||||
@@ -61,7 +61,7 @@ Read SUMMARY.md — extract `## Threat Flags` entries.
|
||||
|
||||
### 2c. Build Threat Register
|
||||
|
||||
Per threat: `{ threat_id, category, component, disposition, mitigation_pattern, files_to_check }`
|
||||
Per threat: `{ threat_id, category, component, severity, disposition, mitigation_pattern, files_to_check }`
|
||||
|
||||
Also set `register_authored_at_plan_time: true` if **at least one** PLAN file contained a parseable `<threat_model>` block; `false` if no PLAN files had any `<threat_model>` block (legacy phase authored before formal threat modelling was standard).
|
||||
|
||||
@@ -74,10 +74,10 @@ Classify each threat:
|
||||
| CLOSED | mitigation found OR accepted risk documented in SECURITY.md OR transfer documented |
|
||||
| OPEN | none of the above |
|
||||
|
||||
Build: `{ threat_id, category, component, disposition, status, evidence }`
|
||||
Build: `{ threat_id, category, component, severity, disposition, status, evidence }`
|
||||
|
||||
**Short-circuit rule:**
|
||||
- If `threats_open: 0 AND register_authored_at_plan_time: true AND asvs_level == 1` → skip to Step 6 directly. All plan-time threats are verified CLOSED at L1 grep-depth; no deeper verification required.
|
||||
- If `threats_open: 0 AND register_authored_at_plan_time: true AND asvs_level == 1` → skip to Step 6 directly. No open threats at or above the block threshold remain (threats_open: 0); below-threshold open threats may remain and are non-blocking. L1 grep-depth is sufficient; no deeper verification required.
|
||||
- If `threats_open: 0 AND register_authored_at_plan_time: true AND asvs_level >= 2` → **do NOT skip**. The preliminary threat classification is grep-level (L1 depth) and is insufficient for L2/L3. Proceed to Step 5 (spawn the auditor) so that L2 boundary-placement checks and L3 end-to-end trace checks are performed. Skipping the auditor here would defeat ASVS level scaling for "clean" phases.
|
||||
- If `threats_open: 0 AND register_authored_at_plan_time: false` → **do NOT skip**. Empty-by-no-planning must not rubber-stamp a clean SECURITY.md. Proceed to Step 5 in **retroactive-STRIDE mode** — the auditor builds a register from implementation files first, then verifies mitigations.
|
||||
- If `threats_open > 0` → proceed to Step 4 (present threat plan to user).
|
||||
@@ -146,7 +146,7 @@ Handle return:
|
||||
|
||||
```
|
||||
GSD > PHASE {N} SECURITY BLOCKED
|
||||
{K} threats open — phase advancement blocked until threats_open: 0
|
||||
{K} blocking threats open — phase advancement blocked until threats_open: 0
|
||||
▶ Fix mitigations then re-run: /gsd:secure-phase {N}
|
||||
▶ Or document accepted risks in SECURITY.md and re-run.
|
||||
```
|
||||
@@ -164,7 +164,7 @@ gsd_run query commit "docs(phase-${PHASE}): add/update security threat verificat
|
||||
**Secured (threats_open: 0):**
|
||||
```
|
||||
GSD > PHASE {N} THREAT-SECURE
|
||||
threats_open: 0 — all threats have dispositions.
|
||||
threats_open: 0 — no blocking threats remain (threats_open: 0).
|
||||
▶ /gsd:validate-phase {N} validate test coverage
|
||||
▶ /gsd:verify-work {N} run UAT
|
||||
```
|
||||
|
||||
@@ -23,11 +23,11 @@
|
||||
"gsd-pattern-mapper.md": 12487,
|
||||
"gsd-phase-researcher.md": 40638,
|
||||
"gsd-plan-checker.md": 44646,
|
||||
"gsd-planner.md": 47865,
|
||||
"gsd-planner.md": 48023,
|
||||
"gsd-project-researcher.md": 22014,
|
||||
"gsd-research-synthesizer.md": 13653,
|
||||
"gsd-roadmapper.md": 22183,
|
||||
"gsd-security-auditor.md": 6767,
|
||||
"gsd-security-auditor.md": 8891,
|
||||
"gsd-ui-auditor.md": 17159,
|
||||
"gsd-ui-checker.md": 11088,
|
||||
"gsd-ui-researcher.md": 19272,
|
||||
|
||||
@@ -395,7 +395,9 @@ describe('gsd-planner.md — supply-chain row in threat_model template', () => {
|
||||
const supplyChainRow = strideTable.rows.find((row) => hasAllTokens(row.cells[0] || '', ['t-{phase}-sc']));
|
||||
assert.ok(supplyChainRow, 'threat_model must include T-{phase}-SC supply-chain row');
|
||||
|
||||
const disposition = supplyChainRow.cells[3] || '';
|
||||
const dispoIdx = strideTable.headers.findIndex((h) => /disposition/i.test(String(h)));
|
||||
assert.ok(dispoIdx >= 0, 'STRIDE table must have a Disposition column');
|
||||
const disposition = supplyChainRow.cells[dispoIdx] || '';
|
||||
assert.ok(hasAllTokens(disposition, ['mitigate']), 'supply-chain threat disposition must be mitigate');
|
||||
});
|
||||
});
|
||||
|
||||
@@ -399,7 +399,185 @@ describe('SECURE: VALIDATION.md security columns', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ─── 7. Threat-model-anchored behaviour (structural) ────────────────────────
|
||||
// ─── 7. Per-threat severity gate (#1626) ────────────────────────────────────
|
||||
|
||||
describe('SECURE: per-threat severity gate (#1626)', () => {
|
||||
const plannerPath = path.join(AGENTS_DIR, 'gsd-planner.md');
|
||||
const auditorPath = path.join(AGENTS_DIR, 'gsd-security-auditor.md');
|
||||
const tplPath = path.join(TEMPLATES_DIR, 'SECURITY.md');
|
||||
const configDocPath = path.join(REPO_ROOT, 'gsd-core', 'references', 'planning-config.md');
|
||||
|
||||
// ── planner: Severity column in threat register header ──────────────────
|
||||
test('gsd-planner.md threat_model register header has Severity column', () => {
|
||||
const content = fs.readFileSync(plannerPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('| Threat ID | Category | Component | Severity | Disposition | Mitigation Plan |'),
|
||||
'planner STRIDE Threat Register header must include a Severity column'
|
||||
);
|
||||
});
|
||||
|
||||
test('gsd-planner.md security instruction assigns severity to each threat', () => {
|
||||
const content = fs.readFileSync(plannerPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('severity') && content.includes('critical|high|medium|low'),
|
||||
'planner security instruction must tell agents to assign a severity (critical|high|medium|low) to each threat'
|
||||
);
|
||||
});
|
||||
|
||||
test('gsd-planner.md checklist has Severity item', () => {
|
||||
const content = fs.readFileSync(plannerPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('Every threat has a Severity (critical|high|medium|low)'),
|
||||
'planner success_criteria checklist must include a Severity checklist item'
|
||||
);
|
||||
});
|
||||
|
||||
// ── auditor: block_on uses severity vocabulary ───────────────────────────
|
||||
test('gsd-security-auditor.md block_on domain is severity vocabulary', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('block_on') && content.includes('critical') && content.includes('none'),
|
||||
'auditor <config> block_on must use severity vocabulary (critical ... none), not the old open/unregistered/none'
|
||||
);
|
||||
});
|
||||
|
||||
test('gsd-security-auditor.md defines severity ordering critical > high > medium > low', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('critical > high > medium > low'),
|
||||
'auditor must define the severity ordering: critical > high > medium > low'
|
||||
);
|
||||
});
|
||||
|
||||
test('gsd-security-auditor.md threats_open counts only open threats at or above block_on', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('threats_open') && content.includes('severity rank') && content.includes('block_on'),
|
||||
'auditor must state that threats_open counts only open threats whose severity rank >= block_on rank'
|
||||
);
|
||||
});
|
||||
|
||||
test('gsd-security-auditor.md documents non-blocking below-threshold opens', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('non-blocking') && content.includes('below'),
|
||||
'auditor must state that open threats below the block_on threshold are non-blocking and must not count toward threats_open'
|
||||
);
|
||||
});
|
||||
|
||||
// ── SECURITY.md template: Severity column ───────────────────────────────
|
||||
test('SECURITY.md template Threat Register has Severity column', () => {
|
||||
const content = fs.readFileSync(tplPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('Severity'),
|
||||
'SECURITY.md Threat Register table must include a Severity column'
|
||||
);
|
||||
});
|
||||
|
||||
// ── planning-config.md: security_block_on reconciled enum ───────────────
|
||||
test('planning-config.md security_block_on row lists critical', () => {
|
||||
const content = fs.readFileSync(configDocPath, 'utf-8');
|
||||
const blockOnLineIdx = content.indexOf('security_block_on');
|
||||
assert.ok(blockOnLineIdx > -1, 'planning-config.md must have security_block_on row');
|
||||
const lineEnd = content.indexOf('\n', blockOnLineIdx);
|
||||
const row = content.slice(blockOnLineIdx, lineEnd);
|
||||
assert.ok(
|
||||
row.includes('critical'),
|
||||
'security_block_on allowed values must include "critical"'
|
||||
);
|
||||
});
|
||||
|
||||
test('planning-config.md security_block_on row lists none', () => {
|
||||
const content = fs.readFileSync(configDocPath, 'utf-8');
|
||||
const blockOnLineIdx = content.indexOf('security_block_on');
|
||||
assert.ok(blockOnLineIdx > -1, 'planning-config.md must have security_block_on row');
|
||||
const lineEnd = content.indexOf('\n', blockOnLineIdx);
|
||||
const row = content.slice(blockOnLineIdx, lineEnd);
|
||||
assert.ok(
|
||||
row.includes('none'),
|
||||
'security_block_on allowed values must include "none"'
|
||||
);
|
||||
});
|
||||
|
||||
// ── auditor: classification vocabulary is severity-conditioned (not all-open-blocks) ──
|
||||
test('gsd-security-auditor.md BLOCKER classification conditions blocking on severity threshold (no unconditional all-open-blocks language)', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
// The reworded classification must include both the blocking condition (severity >= block_on)
|
||||
// AND the non-blocking category for below-threshold threats.
|
||||
// These substrings only appear in the reworded classification block.
|
||||
assert.ok(
|
||||
content.includes('severity ≥ `block_on`'),
|
||||
'BLOCKER classification must condition blocking on "severity ≥ `block_on`" threshold'
|
||||
);
|
||||
assert.ok(
|
||||
content.includes('OPEN-non-blocking (severity below block_on)'),
|
||||
'classification must include OPEN-non-blocking category for below-threshold threats'
|
||||
);
|
||||
// The old unconditional language said "phase must not ship" without a severity qualifier.
|
||||
// After the fix, every "phase must not ship" must be paired with a severity condition.
|
||||
// Find all occurrences of "must not ship" and verify none appear without "severity" nearby.
|
||||
const lines = content.split('\n');
|
||||
for (const line of lines) {
|
||||
if (line.includes('must not ship') && !line.includes('severity')) {
|
||||
assert.fail(
|
||||
`Found "must not ship" without a severity condition on line: ${line.trim()}`
|
||||
);
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
// ── auditor: fail-closed for missing/unranked severity (Finding 1) ─────────
|
||||
test('gsd-security-auditor.md states fail-closed rule for missing/unranked severity', () => {
|
||||
const content = fs.readFileSync(auditorPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('Fail-closed') && content.includes('missing') && content.includes('critical'),
|
||||
'auditor must state that open threats with missing or unparseable severity are treated as critical (fail-closed / blocking)'
|
||||
);
|
||||
});
|
||||
|
||||
// ── secure-phase workflow: blocking-threshold semantics in prose (Finding 2)
|
||||
test('secure-phase.md prose reflects blocking-threshold semantics for threats_open', () => {
|
||||
const wfPath = path.join(WORKFLOWS_DIR, 'secure-phase.md');
|
||||
const content = fs.readFileSync(wfPath, 'utf-8');
|
||||
assert.ok(
|
||||
content.includes('blocking threats') || content.includes('block threshold'),
|
||||
'secure-phase.md must use "blocking threats" or "block threshold" language when describing the threats_open gate'
|
||||
);
|
||||
});
|
||||
|
||||
// ── secure-phase workflow: severity field in register shapes (#1626) ────────
|
||||
test('secure-phase.md Step 2c per-threat shape includes severity', () => {
|
||||
const wfPath = path.join(WORKFLOWS_DIR, 'secure-phase.md');
|
||||
const content = fs.readFileSync(wfPath, 'utf-8');
|
||||
// Step 2c defines the per-threat object shape — must carry severity so the
|
||||
// auditor's fail-closed rule can rank it rather than defaulting to critical.
|
||||
assert.ok(
|
||||
content.includes('threat_id, category, component, severity, disposition, mitigation_pattern'),
|
||||
'secure-phase.md Step 2c per-threat shape must include severity field'
|
||||
);
|
||||
});
|
||||
|
||||
// ── docs/CONFIGURATION.md: security_block_on full enum (Finding 3) ─────────
|
||||
test('docs/CONFIGURATION.md security_block_on mentions critical and none', () => {
|
||||
const docsConfigPath = path.join(REPO_ROOT, 'docs', 'CONFIGURATION.md');
|
||||
const content = fs.readFileSync(docsConfigPath, 'utf-8');
|
||||
// Find the markdown table row (starts with '| `workflow.security_block_on`')
|
||||
const tableRowIdx = content.indexOf('| `workflow.security_block_on`');
|
||||
assert.ok(tableRowIdx > -1, 'docs/CONFIGURATION.md must have a workflow.security_block_on table row');
|
||||
const lineEnd = content.indexOf('\n', tableRowIdx);
|
||||
const row = content.slice(tableRowIdx, lineEnd);
|
||||
assert.ok(
|
||||
row.includes('critical'),
|
||||
'docs/CONFIGURATION.md security_block_on row must include "critical"'
|
||||
);
|
||||
assert.ok(
|
||||
row.includes('none'),
|
||||
'docs/CONFIGURATION.md security_block_on row must include "none"'
|
||||
);
|
||||
});
|
||||
});
|
||||
|
||||
// ─── 8. Threat-model-anchored behaviour (structural) ────────────────────────
|
||||
|
||||
describe('SECURE: threat-model-anchored behaviour', () => {
|
||||
const agentPath = path.join(AGENTS_DIR, 'gsd-security-auditor.md');
|
||||
|
||||
@@ -65,7 +65,7 @@
|
||||
"resume-project.md": 17226,
|
||||
"review.md": 39404,
|
||||
"scan.md": 7688,
|
||||
"secure-phase.md": 13315,
|
||||
"secure-phase.md": 13476,
|
||||
"session-report.md": 4044,
|
||||
"settings-advanced.md": 39666,
|
||||
"settings-integrations.md": 15848,
|
||||
|
||||
Reference in New Issue
Block a user