LLM Spend Cap

The bug this exists to prevent

Almost every spend guard is written after the fact: sum what has been spent, and stop when the total is too high. That catches the slow case and misses the expensive one.

The failure that costs real money is a single request that should never have started — a prompt that already contains 400,000 tokens of context, pointed at the most expensive model, with a large max_tokens. By the time it appears in the ledger it has already been paid for. A cumulative cap is a smoke alarm that rings once the house has burned.

The other half is worse because it is invisible: an agent in a retry loop issues requests that are each individually reasonable. Nothing looks anomalous per request. The bill is the only place the loop is visible, and it arrives weeks later.

Rule 1 — price the ceiling, not the average

estimateMaxRequestUsd() costs a request at its maximum: the input tokens already counted, plus the model's full max_tokens of output, each at that model's rate. decidePerRequest() compares that ceiling to your limit and refuses before the call is made.

Estimating the likely cost is the mistake worth avoiding. A request that usually returns 200 tokens and occasionally returns 8,000 is exactly the request that breaches a cap, and averaging prices it as safe every time until it isn't.

Rule 2 — the pricing table is passed in, never duplicated

PriceLookup is a function you supply. This package ships no model prices.

That is deliberate. A second copy of a pricing table is a copy that goes stale — provider changes a rate, one table updates, and the guard now enforces a limit computed from prices that no longer exist. It fails in the safe-looking direction, permitting spend, and nothing reports an error because the arithmetic is still perfectly valid.

approxTokensFromChars() is available for the pre-tokenizer estimate, and it is deliberately rough — the point is refusing a 400k-token request, and no reasonable error in a 4-chars-per-token approximation changes that answer.

Rule 3 — day and month are separate ceilings

decideCumulative() takes the current rolling totals and the pending amount and reports which ceiling would break. resolveOrgCaps() applies defaults where an organisation has none set.

Both windows exist because they fail differently: a day cap contains a loop that started this morning, a month cap contains steady over-use that never trips a daily limit.

What this does NOT do

No counting, no storage, no ledger, no provider calls. It decides; you persist. That is why it is testable without a database, and it is the half that is worth buying — the storage is straightforward and the policy is where the mistakes live.

It does not know your prices. See Rule 2.

Verified

19 tests covering ceiling estimation, per-request refusal at and around the boundary, unknown models, day and month cumulative breaches, and default resolution.