enhance(#2671): brand raw vs calibrated token types so double-application is a compile error (#2676)
* test(#2671): add failing-first brand-typing compile fixtures * feat(#2671): brand raw vs calibrated token types * refactor(#2671): hoist type-compile into a before() hook Two review responses: - The fixture compile ran in the describe() body, so it executed at collection time even when the block was filtered out, and a failed precondition collapsed eight independent assertions into one opaque describe-level failure. A before() hook is this repo's documented idiom and preserves per-test granularity. - parseTokensFlag now records WHY it returns an unbranded number: it validates the magnitude of --tokens, but the basis is decided by --calibrated, so branding here would be wrong for half its callers. The assertion belongs to cmdEstimateCheck, its only caller. * test(#2671): pin each brand diagnostic to its OFFENDING marker Adversarial review demonstrated that asserting only exactly-one-diagnostic- at-code-N is not airtight. Repairing a fixture's brand violation while injecting an unrelated error of the same code (a string passed as the budget argument) still yielded exactly one TS2345, so the fixture would have reported green while no longer testing its regression at all. Each bad-* fixture now routes its violating value through a const named OFFENDING, and the test asserts the diagnostic's start offset falls inside that node — located through the AST, so it survives reformatting and never pattern-matches source text. Replaying the proof-of-concept against the new assertion rejects it: the diagnostic lands on the budget literal, not the marker. Also corrects a doc comment that claimed the program type-checks all of src/; it covers phase-estimation.cts and its transitive dependencies. * chore(#2671): backfill changeset PR number (#2676)
This commit is contained in:
@@ -76,6 +76,7 @@ factor = 1.0 when n < 3
|
||||
- **`n >= 3` before any correction applies** — below that, the sample says more about variance than about bias.
|
||||
- **The denominator is the RAW projection, not the emitted (already-corrected) figure** — amended #2632. Measuring `actual / calibrated` is self-defeating: once the correction works the observed ratio approaches 1, which drags the median back toward 1 and un-corrects the next estimate. Simulated over 10 phases against a true 2x miss it oscillates and settles near 1.41 instead of converging on 2.0. Plans therefore record `estimate.raw_tokens` alongside the calibrated `estimate.tokens`, and `calibrationBasis()` prefers it (falling back to `tokens` for plans written before #2632, where no factor had yet been applied).
|
||||
- **Samples are per PLAN, not per phase** — amended #2632. A phase holds several `<NN>-<PP>-PLAN.md` files; pairing at phase granularity cross-pairs one plan's projection with another's cost and discards the rest.
|
||||
- **The raw/calibrated distinction is enforced by the type system, not by convention** — amended #2671. `--calibrated` and `raw_tokens` fix the two known call sites but remain conventions the *next* caller must also remember, and the failure mode is silent: both states are positive integers of the same magnitude, so no runtime check can tell them apart. `src/phase-estimation.cts` therefore gives them distinct branded types, `RawTokens` and `CalibratedTokens`, so `applyCalibration(alreadyCalibrated, factor)` and `{ estimateTokens: estimate.tokens }` are compile errors under `npm run build:lib` rather than review findings. The brands erase at compile time — no `.cjs` behavior change, no wire-format change, and untyped `.cjs` callers are unaffected, which is why every runtime guard in the module stays in place. Compile fixtures live in `tests/fixtures/brand-typing/` and are driven through the TypeScript compiler API by `tests/phase-estimation.test.cjs`.
|
||||
|
||||
Persisted to `.planning/estimation-calibration.json` with a `schema_version` field, written by `extract-learnings`, read at plan time. Versioned from the first write so the schema can migrate without a silent misread.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user