119 lines
11 KiB
Markdown
119 lines
11 KiB
Markdown
---
|
|
phase: "20"
|
|
slug: "demo-instance-with-every-shared-plugin"
|
|
# status lifecycle: draft (seeded by plan-phase) → validated (set by validate-phase §6)
|
|
# audit-milestone §5.5 distinguishes NOT-VALIDATED (draft) from PARTIAL (validated + nyquist_compliant: false) (#2117)
|
|
status: draft
|
|
nyquist_compliant: false
|
|
wave_0_complete: false
|
|
created: "2026-10-28"
|
|
---
|
|
|
|
# Phase 20 — Validation Strategy
|
|
|
|
> Per-phase validation contract for feedback sampling during execution.
|
|
|
|
---
|
|
|
|
## Test Infrastructure
|
|
|
|
| Property | Value |
|
|
|----------|-------|
|
|
| **Framework** | go test (plugins, sm-demo-app; testcontainers Postgres on full runs); vitest 3.2.7 (vue-demo-app, vue-chat-layer); node --test or vitest (other layers) |
|
|
| **Config file** | `../sm-demo-app/vue-demo-app/vitest.config.ts`; new layers: none — Wave 0 installs |
|
|
| **Quick run command** | `go vet ./... && go test -short ./...` in the touched Go repo; `pnpm test` in the touched layer |
|
|
| **Full suite command** | `cd ../sm-demo-app && scripts/check-all.sh` |
|
|
| **Estimated runtime** | ~30 seconds quick, ~300 seconds full |
|
|
|
|
---
|
|
|
|
## Sampling Rate
|
|
|
|
- **After every task commit:** Run the quick run command of the repo touched
|
|
- **After every plan wave:** Run sm-demo-app `go test ./...` (Docker) + `pnpm --dir vue-demo-app test` + `scripts/check-pins.sh`
|
|
- **Before `/msd-verify-work`:** Full suite must be green
|
|
- **Max feedback latency:** 60 seconds
|
|
|
|
---
|
|
|
|
## Per-Task Verification Map
|
|
|
|
Filled by the planner from the plans; source map in `20-RESEARCH.md` § Validation Architecture.
|
|
|
|
| Task ID | Plan | Wave | Requirement | Threat Ref | Secure Behavior | Test Type | Automated Command | File Exists | Status |
|
|
|---------|------|------|-------------|------------|-----------------|-----------|-------------------|-------------|--------|
|
|
| 20-01-01 | 01 | 1 | D-18, SC-1 | T-20-01, T-20-02 | a rejecting MessageSaving listener gives 422 and stores nothing; no listener changes nothing | integration (Docker) | `go -C ../sm-chat-plugin test -count=1 -run '^TestMessageSavingTracer$' -v .` | ❌ created in task (saving_smoke_test.go) | ⬜ pending |
|
|
| 20-01-02 | 01 | 1 | D-18 | T-20-01 | forum thread, reply and edit refused before render | integration (Docker) | `go -C ../sm-forum-plugin test -count=1 -run '^TestPostSavingTracer$' -v .` | ❌ created in task | ⬜ pending |
|
|
| 20-01-03 | 01 | 1 | D-17 | T-20-03, T-20-04 | word and link matching; list caps; settings behind a permission | unit | `go -C ../sm-moderation-plugin test -count=1 -run '^TestFilterSmoke$' -v ./classes` | ❌ new repo | ⬜ pending |
|
|
| 20-01-04 | 01 | 1 | SC-1 | T-20-05 | modules resolve by tag only | build | `for r in …8 repos; do go -C ../$r vet ./... && go -C ../$r test -short ./...; done` + `GOWORK=off go build` of 5 dependants | ✅ | ⬜ pending |
|
|
| 20-01-05 | 01 | 1 | — | T-20-05 | go-ahead before one-way tags | checkpoint | — | — | ⬜ pending |
|
|
| 20-01-06 | 01 | 1 | SC-1 | T-20-05 | remote tags equal verified HEADs | script | `git ls-remote origin refs/tags/<tag>^{}` loop | ✅ | ⬜ pending |
|
|
| 20-02-01 | 02 | 2 | SC-1, D-06, D-07 | T-20-06, T-20-07 | member posts in #general; channel creation refused | integration (Docker) | `go -C ../sm-demo-app test -count=1 -run '^(TestDemoChatTracer\|TestGoModRequiresTags\|TestBootAllPlugins)$' -v .` + `scripts/check-pins.sh` | ❌ created in task (chat_smoke_test.go) | ⬜ pending |
|
|
| 20-02-02 | 02 | 2 | D-05, D-08, D-12, D-14, D-17, D-22, D-23 | T-20-09, T-20-10 | banned words and links refused in chat and forum | integration (Docker) | `go -C ../sm-demo-app test -count=1 -run '^TestDemoModerationSmoke$' -v .` + `scripts/build.sh dev` | ❌ created in task | ⬜ pending |
|
|
| 20-02-03 | 02 | 2 | D-09 | T-20-08, T-20-13 | personas cannot log in; reset empties every chat-family table | integration (Docker) | `go -C ../sm-demo-app test -count=1 -run '^(TestSeedChatSmoke\|TestResetKeepsPlans\|TestSeedRoadmapUpsert)$' -v .` | ❌ / ✅ | ⬜ pending |
|
|
| 20-02-04 | 02 | 2 | D-06, D-10 | T-20-11, T-20-12 | realtime token only for live members; no secret committed | unit + script | `go -C ../sm-demo-app test -short .` + `centrifugo checkconfig` + `scripts/dev.sh up` + `scripts/check-local.sh realtime` | ✅ | ⬜ pending |
|
|
| 20-03-01 | 03 | 3 | D-01, D-04, D-07, SC-2 | T-20-15 | anonymous prompt; Bearer only to same-origin APIs | unit + browser | `pnpm --dir ../vue-chat-layer test` + vue-demo-app build + `scripts/check-local.sh chat` | ❌ created in task | ⬜ pending |
|
|
| 20-03-02 | 03 | 3 | D-02, SC-2 | T-20-14, T-20-19 | text escaped; one realtime connection | unit + browser | layer and vue-demo-app tests + `scripts/check-local.sh chat realtime remote` | ✅ (app) | ⬜ pending |
|
|
| 20-03-03 | 03 | 3 | D-02, D-05, D-22, D-23 | T-20-14, T-20-16, T-20-17 | linkify on escaped HTML; authorised downloads; ephemeral answers stay local | unit + build | `pnpm --dir ../vue-chat-layer test` + `build:playground` + `scripts/check-local.sh chat` | ✅ | ⬜ pending |
|
|
| 20-03-04 | 03 | 3 | D-03, D-04 | — | floating panel hidden for anonymous visitors | browser | `scripts/check-local.sh tour chat` | ✅ extend | ⬜ pending |
|
|
| 20-03-05 | 03 | 3 | D-01 | — | go-ahead before the public tag | checkpoint | — | — | ⬜ pending |
|
|
| 20-03-06 | 03 | 3 | D-01 | T-20-18 | layer pinned by tag with a committed lockfile | script | remote tag equals HEAD + pin grep + vue-demo-app test and build | ✅ | ⬜ pending |
|
|
| 20-04-01 | 04 | 4 | D-14, D-21, D-22 | T-20-20 | posts render only server content_html | unit + script | `pnpm --dir ../vue-forum-layer test` + build + `scripts/check-local.sh forum` | ❌ created in task | ⬜ pending |
|
|
| 20-04-02 | 04 | 4 | D-15 | T-20-25 | friend refusals stay generic | unit + script | `pnpm --dir ../vue-userfriends-layer test` + `scripts/check-local.sh friends forum chat` | ❌ created in task | ⬜ pending |
|
|
| 20-04-03 | 04 | 4 | D-13 | T-20-SC | livekit-client legitimacy confirmed before install | checkpoint (blocking-human) | — | — | ⬜ pending |
|
|
| 20-04-04 | 04 | 4 | D-11, D-12, D-13, D-23 | T-20-21, T-20-22, T-20-23, T-20-24 | server-minted tokens; call button only in DMs; media only after start or accept | unit + script | `pnpm --dir ../vue-chatvideo-layer test` + build + `scripts/check-local.sh call chat` | ❌ created in task | ⬜ pending |
|
|
| 20-04-05 | 04 | 4 | D-16 | T-20-26 | VAPID keys only in .env | script | `scripts/dev.sh up` + `scripts/check-local.sh herald tour` | ❌ check added in task | ⬜ pending |
|
|
| 20-04-06 | 04 | 4 | D-14, D-15, D-13 | — | go-ahead before public tags | checkpoint | — | — | ⬜ pending |
|
|
| 20-04-07 | 04 | 4 | SC-2 | T-20-SC | layers pinned by tag | script | remote tags equal HEAD + vue-demo-app test, build + `scripts/check-local.sh forum friends call chat tour` | ✅ | ⬜ pending |
|
|
| 20-05-01 | 05 | 5 | D-10, D-12, SC-3 | T-20-31 | exact Go allow-list; /_herald unrouted; ports and caps agree | script | `scripts/check-deploy.sh` (last line `check-deploy: ok`) | ✅ extend | ⬜ pending |
|
|
| 20-05-02 | 05 | 5 | SC-1 | T-20-33 | release build from tags, smoke clean | script | `scripts/check-pins.sh` + `scripts/build.sh release` + `scripts/smoke.sh` | ✅ | ⬜ pending |
|
|
| 20-05-03 | 05 | 5 | SC-3 | T-20-27 | go-ahead before rome changes | checkpoint | — | — | ⬜ pending |
|
|
| 20-05-04 | 05 | 5 | SC-2, SC-3, D-10, D-16 | T-20-27, T-20-28, T-20-34 | WavePath untouched; secrets generated on rome | live | `scripts/check-live.sh https://demo.summercms.io` | ✅ extend | ⬜ pending |
|
|
| 20-05-05 | 05 | 5 | D-10 | T-20-29, T-20-32 | go-ahead before the firewall change | checkpoint | — | — | ⬜ pending |
|
|
| 20-05-06 | 05 | 5 | D-10 | T-20-29, T-20-32 | only 7891/tcp and 7892/udp opened, through Vuurmuur | live | `ssh … iptables -S` count + `nc -zv … 7891` | ✅ | ⬜ pending |
|
|
| 20-06-01 | 06 | 6 | D-17, D-18, SC-4 | T-20-10 | settings to filter to listener to 422 | unit + integration | `go -C ../sm-moderation-plugin test ./...` + `-run '^TestDemoModeration$'` | ❌ created in task | ⬜ pending |
|
|
| 20-06-02 | 06 | 6 | D-18, D-17 | T-20-01, T-20-02 | parity without listeners; refusal on every write path | unit + integration | chat, forum and moderation full suites | ❌ created in task | ⬜ pending |
|
|
| 20-06-03 | 06 | 6 | D-05 to D-09, D-11, D-12, D-22, D-23, SC-1 | T-20-06, T-20-07, T-20-09, T-20-11 | membership, refused channel create, moderator scope, member-only tokens | integration | `go -C ../sm-demo-app test -count=1 . ./plugins/demo/...` + named tests | ❌ created in task | ⬜ pending |
|
|
| 20-06-04 | 06 | 6 | D-01, D-02, D-04, D-05, D-13, D-14, D-15, D-21, D-23 | T-20-14 | escaping and DM-only slot pinned | unit | `pnpm test` in four layers + vue-demo-app test and build | ❌ created in task | ⬜ pending |
|
|
| 20-06-05 | 06 | 6 | SC-2, D-11, D-13, D-23 | — | two-browser realtime; DM call with fake media | browser E2E | `scripts/check-local.sh chat chatlive callmedia forum friends call herald tour realtime remote` | ❌ created in task | ⬜ pending |
|
|
| 20-06-06 | 06 | 6 | — | — | go-ahead before pushes and fix tags | checkpoint | — | — | ⬜ pending |
|
|
| 20-06-07 | 06 | 6 | SC-2, SC-3, SC-4 | T-20-36 | full gate including check-live | gate | `scripts/check-all.sh --local` (last line `check-all: ok`) | ✅ extend | ⬜ pending |
|
|
| 20-06-08 | 06 | 6 | SC-2, SC-3 | — | two-browser chat and DM call on the live demo | manual (UAT) | — | — | ⬜ pending |
|
|
|
|
*Status: ⬜ pending · ✅ green · ❌ red · ⚠️ flaky*
|
|
|
|
---
|
|
|
|
## Wave 0 Requirements
|
|
|
|
Each gap is closed inside the plan that creates the code (a smoke test or check in the same task), and the full tests come in plan 06 (project rule: unit tests are the last plan).
|
|
|
|
- [ ] `sm-demo-app/seed_chat_test.go`, `chat_test.go` (context, channel adapter, resolvers, DM calls, bootstrap JSON): smoke in 20-02 (`chat_smoke_test.go`), full in 20-06 Task 3
|
|
- [ ] `sm-chat-plugin` and `sm-forum-plugin` pre-save seam tests: smoke in 20-01 Tasks 1-2, full in 20-06 Task 2
|
|
- [ ] `sm-moderation-plugin` package and tests (new repo): smoke in 20-01 Task 3, full in 20-06 Tasks 1-2
|
|
- [ ] `vue-chat-layer` vitest config and ported bm chat tests (composable store setup): config and smoke in 20-03 Task 1, ported tests in 20-06 Task 4
|
|
- [ ] `vue-chatvideo-layer`, `vue-forum-layer`, `vue-userfriends-layer` `tests/*.test.ts`: smoke in 20-04 Tasks 1, 2 and 4, full in 20-06 Task 4
|
|
- [ ] `scripts/dev.sh` local livekit-server (20-02 Task 4); `scripts/check-local.sh chat` (20-03 Task 1), `forum`, `friends`, `call`, `herald` (20-04), `chatlive` and `callmedia` (20-06 Task 5)
|
|
|
|
---
|
|
|
|
## Manual-Only Verifications
|
|
|
|
| Behavior | Requirement | Why Manual | Test Instructions |
|
|
|----------|-------------|------------|-------------------|
|
|
| Two-browser chat realtime and a DM video call on the live demo | SC-2, D-13 | Real media over NAT'd browsers and the rome firewall | Two accounts on https://demo.summercms.io: send a message, see it arrive live; start a DM call, see and hear both tiles, mute, leave |
|
|
| Vuurmuur firewall rule on rome | D-10 | Shared production host change | Apply with the user's go-ahead; confirm with `/livekit/rtc/validate` and a call |
|
|
|
|
---
|
|
|
|
## Validation Sign-Off
|
|
|
|
- [x] All tasks have `<automated>` verify or Wave 0 dependencies (checkpoints excepted)
|
|
- [x] Sampling continuity: no 3 consecutive tasks without automated verify
|
|
- [ ] Wave 0 covers all MISSING references
|
|
- [x] No watch-mode flags (vitest run, node --test)
|
|
- [ ] Feedback latency < 60s
|
|
- [ ] `nyquist_compliant: true` set in frontmatter
|
|
|
|
**Approval:** pending
|