f4d6747d21
* fix(#2738): report budget outcome from graphify query and stop over-trimming between tiers applyBudget retains seed nodes unconditionally, so the seed set is a floor the edge-tier reduction cannot go below — a --budget 500 request could return the full seed payload (~119k tokens measured) with no signal of the miss. Add budget_met + budget_estimate to the budget result and surface them through graphifyQuery when a budget was requested. Secondary: the tier loop estimated against the full pre-filter node set, so a tier removal that already satisfied the budget (once its orphaned nodes were excluded) still triggered the next, higher-confidence tier drop. Recompute reachability and the estimate after each tier and break as soon as the pruned result fits. Adjacent same-class instance from pre-submit review: the CLI forwards --budget 0 but truthiness checks silently treated it as no budget and returned the unbounded result. Test budget presence with != null so a zero budget is honored and reported as an unmeetable miss. * docs(changeset): backfill PR number for #2738 fragment * fix(#2738): estimate the payload as emitted, not a private compact form budget_met measured a different payload than the caller receives. The estimator serialized a compact `{nodes, edges}`, while output() emits the whole response pretty-printed (2-space indent, plus the term/total_*/trimmed wrapper keys). Measured on the repo's own SAMPLE_GRAPH fixture: reported 183 tokens against 302 actually emitted — 1.65x — so `--budget 200` returned budget_met: true while handing back 302 tokens. That is worse than the old silent miss: an automated consumer stops checking a signal that is confidently wrong. Fix the basis rather than the number: - io.cts gains serializeForOutput(), the single definition of the wire form. output() now calls it, so the estimator and the emitter cannot drift on indentation or shape. Pure extraction; output()'s behaviour is unchanged. - graphify builds its response through one buildQueryResponse() used by both the emitter and the estimator, so the estimate describes exactly the bytes returned. - The tier loop estimates on that same basis, so it keeps trimming until the real payload fits instead of stopping at a smaller internal measure. This makes budget_met === (budget_estimate <= budget) true by construction. - Drop the module-private chars/4 helper for prompt-budget's estimateTokens — the repo's single token scale, per the rule phase-estimation.cts documents. budget_estimate is self-referential (its own digits are part of the emitted bytes), resolved by iterating to a fixed point; the sequence only ever grows, so it settles in a couple of passes and errs toward over-reporting. Tests pin estimator to emitter so this cannot silently re-diverge if output() ever changes its indentation. Both new tests fail against the pre-fix source. * test(#2738): property-test the budget-limit and reporting contract RULESET.TESTS.property-based-testing names budget-limit contracts, and this module is the literal case: #2819 turns it into a *reporting* contract, which is what properties express well. Five invariants over arbitrary small graphs and any budget >= 0: - budget_met === (budget_estimate <= budget) - budget_estimate === the tokens actually emitted - the seed set is a floor the reduction never goes below (the changeset's "seeds are a floor" claim, previously asserted only for one hand-built fixture) - total_nodes/total_edges match the returned arrays - a larger budget never yields a smaller payload Two notes on what these do and do not prove. The emitted-payload property fails against the pre-fix source; the budget_met/budget_estimate agreement property does NOT — pre-fix both derived from the same wrong number, so it is a contract guard, not a regression proof. Monotonicity is asserted over the payload (node/edge counts), not over budget_estimate: the estimate measures emitted bytes exactly, and budget_met renders as "false" (5 chars) or "true" (4), so an identical payload can measure one token larger when the budget is missed. That is the estimate being honest, not a monotonicity break. The generator seeds on `label` — seedAndExpand matches label/description, never id/name, and a fixture that gets this wrong expands to nothing and passes vacuously. * test(#2738): pin the budget boundary and label the forward-guard test Two test-coverage gaps from review. RULESET.TESTS.boundary-coverage wants limit-1 / limit / limit+1. The added tests used 1, 0, 200, 100000, 50 — all far from the decision point. The branch that matters is `estimate <= budgetTokens`, so the input that decides it is budget === estimate exactly: that is the one value where an off-by-one in the comparison flips budget_met, and nothing else in the suite would catch it. Pinned at the limit and either side of it. The "omits budget fields when no budget was requested" test passes unchanged on next — graphifyQuery never set those keys before the fix, so both assertions already held. It has value as a forward guard against the spread leaking budget fields, but it is not failing-first and should not be counted toward RULESET.TESTS.regression-must-fail-first. Said so in a comment, so a later reader does not mistake it for the regression proof. * fix(#2738): keep a non-finite budget out of the budget path Switching `!budgetTokens` to `budgetTokens == null` widened the internal contract to admit NaN, where every `estimate <= NaN` is false: the loop strips all three tiers and returns a seeds-only payload that is indistinguishable from a legitimate aggressive trim. Unreachable through the CLI — graphify-command-router rejects a non-numeric --budget with makeInvalidArgs before graphifyQuery is called — so this is hardening, not a live defect. It is still worth guarding: graphifyQuery and applyBudget are module-level entry points a future caller could reach without the router's validation, and the failure mode is silent. Number.isFinite also routes Infinity to the no-budget path, deliberately: an unbounded budget is not a budget, and parseInt cannot produce one anyway. --------- Co-authored-by: Tom Boucher <trekkie@nomorestars.com>
35 KiB
35 KiB