docs(phase-31): add validation strategy
This commit is contained in:
@@ -0,0 +1,97 @@
|
||||
---
|
||||
phase: "31"
|
||||
slug: "chat-polish-against-wavepath-clients"
|
||||
# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6)
|
||||
# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117)
|
||||
status: draft
|
||||
nyquist_compliant: false
|
||||
wave_0_complete: false
|
||||
created: "2026-11-13"
|
||||
---
|
||||
|
||||
# Phase 31 — Validation Strategy
|
||||
|
||||
> Per-phase validation contract for feedback sampling during execution.
|
||||
> Source: `31-RESEARCH.md`, section "Validation Architecture".
|
||||
|
||||
---
|
||||
|
||||
## Test Infrastructure
|
||||
|
||||
| Property | Value |
|
||||
|----------|-------|
|
||||
| **Framework** | Go `testing` (testcontainers Postgres for the parity suite); Vitest in the vue layers |
|
||||
| **Config file** | none in `sm-wavepath-app` yet — Wave 0 creates the repo |
|
||||
| **Quick run command** | `go vet ./... && go test -short ./...` in each touched repo |
|
||||
| **Full suite command** | `go vet ./... && go test ./...` in `summercms.go`, `sm-wavepath-app` and each changed plugin; `scripts/check-parity.sh` in `sm-wavepath-app` |
|
||||
| **Estimated runtime** | quick: under 60 seconds per repo; full with Docker: several minutes |
|
||||
|
||||
---
|
||||
|
||||
## Sampling Rate
|
||||
|
||||
- **After every task commit:** Run `go vet ./... && go test -short ./...` in the touched repo
|
||||
- **After every plan wave:** Run `go test ./...` (Docker) in the touched repos and `go test ./parity` in `sm-wavepath-app`
|
||||
- **Before `/msd-verify-work`:** `scripts/check-parity.sh` in `sm-wavepath-app` must be green
|
||||
- **Max feedback latency:** 60 seconds for the quick run
|
||||
|
||||
---
|
||||
|
||||
## Per-Task Verification Map
|
||||
|
||||
Task IDs are filled in by the planner. The rows below are the behaviours every plan set must cover.
|
||||
|
||||
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|
||||
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
|
||||
| TBD | TBD | TBD | SC-1 every corpus route recorded from PHP | — | fixtures carry no secrets (`--check-secrets`) | corpus check | `go run ./parity/check_corpus.go --manifest parity/manifest.yaml --routes <routes.php> --require-recorded --check-secrets` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | SC-1 Go replays every ported route; pending never passes | — | denial cases (401, 403, 404) replay | integration | `go test ./parity -run TestParityCorpus` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | SC-1 zero pending at close (D-15) | — | fail-closed gate | gate | `scripts/check-parity.sh` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | SC-2 publications of each writing case match the golden (D-10) | — | no publication reaches a non-member channel | integration | `go test ./parity -run TestBroadcasts` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | SC-2 channel authorization routes replay | — | subscribe proxy and `realtime/token` deny non-members | integration | `go test ./parity -run 'TestParityCorpus/.*realtime'` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | D-11 vendor calls match the sidecars | — | N/A | integration | `go test ./parity -run TestUpstreamSidecarsAreReplayed` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | SC-3 `sm-bm-app` builds and passes with the new plugin commits (D-14) | — | N/A | integration | `cd ../sm-bm-app && go build ./... && go test ./...` | ✅ | ⬜ pending |
|
||||
| TBD | TBD | TBD | SC-3 `sm-demo-app` builds and passes | — | N/A | integration | `cd ../sm-demo-app && go test ./...` | ✅ | ⬜ pending |
|
||||
| TBD | TBD | TBD | SC-3 framework docs name no application | — | N/A | docs check | `summer docs:build --check` and `go test ./cmd/summer -run TestDocsTree` | ✅ | ⬜ pending |
|
||||
| TBD | TBD | TBD | SC-4 tide additions | — | N/A | unit | `go test ./modules/tide/...` | ✅ (extend) | ⬜ pending |
|
||||
| TBD | TBD | TBD | SC-4 app plugin host services | — | guest, bot and token auth, webhook, access resolvers | unit | `go test -short ./plugins/golem15/wavepath/...` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | D-12 attachment Range, 206 and 416 | — | N/A | unit and parity | `go test -short ./... -run TestAttachmentRange` in `sm-chat-plugin` | ❌ W0 | ⬜ pending |
|
||||
| TBD | TBD | TBD | Todo: OIDC parity | — | N/A | integration | `PHASE194_REQUIRE_PARITY=1 go test -count=1 -run '^(TestPHPParityReplay\|TestIDTokenShapeParity)$' ./...` in `sm-oauthserver-plugin` | ✅ (skips today) | ⬜ pending |
|
||||
| TBD | TBD | TBD | Todo: moderation seams fire and reject | — | with no listener, behaviour is unchanged | unit | `go test -short ./...` in user, userfriends, chat; `go test ./... -run TestModeration` in `sm-demo-app` | partly | ⬜ pending |
|
||||
| TBD | TBD | TBD | Todo: slash handling in the layer | — | N/A | unit | the layer's Vitest suite (`pnpm --dir ../vue-chat-layer test`) | ✅ (extend) | ⬜ pending |
|
||||
|
||||
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
||||
|
||||
---
|
||||
|
||||
## Wave 0 Requirements
|
||||
|
||||
- [ ] `sm-wavepath-app` repo: `go.mod`, `go.work`, submodules, `summer.yaml`
|
||||
- [ ] `parity/parity_test.go` with a `TestMain` that calls `flag.Parse()` and checks `testing.Short()`
|
||||
- [ ] `parity/check_corpus.go` and `routes.snapshot`
|
||||
- [ ] `parity/fixtureplugin` and `wavepath_reset.php` for the scenario
|
||||
- [ ] `scripts/parity-stack.sh` and `parity/php_parity.sh`
|
||||
- [ ] `scripts/check-parity.sh`
|
||||
- [ ] tide tests for each addition (T-1 to T-10 in `31-RESEARCH.md`)
|
||||
|
||||
---
|
||||
|
||||
## Manual-Only Verifications
|
||||
|
||||
| Behavior | Requirement | Why Manual | Test Instructions |
|
||||
|----------|-------------|------------|-------------------|
|
||||
| PHP self-replay is green | SC-1 | needs the local PHP stack running; not part of `go test` | bring the stack up with `scripts/parity-stack.sh`, then `summer parity:replay --manifest parity/manifest.yaml --self-check true` |
|
||||
| Proxy sessions of the real clients | D-07 | done once by a person driving the clients through `summer parity:proxy` | follow the recording runbook in `sm-wavepath-app/parity/README.md` |
|
||||
| Accepted-differences sign-off | D-13 | each entry needs the user's signature | review the accepted-differences file before the gate is run |
|
||||
|
||||
---
|
||||
|
||||
## Validation Sign-Off
|
||||
|
||||
- [ ] All tasks have `<automated>` verify or Wave 0 dependencies
|
||||
- [ ] Sampling continuity: no 3 consecutive tasks without automated verify
|
||||
- [ ] Wave 0 covers all MISSING references
|
||||
- [ ] No watch-mode flags
|
||||
- [ ] Feedback latency < 60s
|
||||
- [ ] `nyquist_compliant: true` set in frontmatter
|
||||
|
||||
**Approval:** pending
|
||||
Reference in New Issue
Block a user