Every endpoint that calls a model is an open wallet

TL;DR: Any route that calls a paid model turns a loop, a bot or one curious user into a bill. My rule is now fixed: every endpoint that calls a model gets a per-user limit, a global daily cap and a kill switch that works without a deploy, and every public form in front of one gets a challenge. The counters fail closed, so a broken counter never becomes an unlimited budget. Every model call writes a durable cost row into a spend ledger inside the product. Environments are separated at the provider, each with its own project and hard cap, so the production numbers stay clean. Provider spend caps alone are not enough; some of them are not even hard.

The problem: authenticated is not the same as budgeted

In an AI-visibility product I run, the expensive work (asking AI engines questions on a schedule) lived in a background worker, and that worker had a hard daily dollar cap from early on. The dashboard was a different service. A few of its routes also called a model: generating suggestions during onboarding, writing content briefs for a customer. Those routes checked that the user was logged in, and nothing else.

With open signup, "logged in" means "anyone who filled in a form". A script that signs up and calls one of those routes in a loop spends my money at whatever rate the provider allows. Nothing is broken in the security sense. Authentication works, the data is scoped correctly, no one reads anything they should not. The route simply has no concept of how much it is allowed to cost.

The industry name for this is denial of wallet: an attack, or an accident, whose damage is measured in your cloud or API bill rather than in downtime. The OWASP Top 10 for LLM applications files it under unbounded consumption. For a solo product it is the most likely serious incident, because it needs no skill at all.

What guardrails does an endpoint that calls an LLM need?

I ended up with the same layers everywhere, ordered cheapest check first so that a blocked request costs one fast read:

1. Global kill switch. One flag in the database that every spending path honors, in every service. Flipping it needs no deploy, and a short cache keeps the read off the hot path while still reaching every instance within seconds. It sits above everything else: it stops spend even when the cap is off.
2. Per-user limit. A per-hour allowance per account and route. This protects the wallet, and it protects the other customers too, as described below.
3. Global daily cap. A dollar ceiling across all users and all paths. It bounds the blast radius of many accounts at once, which per-user limits cannot do.
4. Challenge on public forms. Signup, login, password reset, and any anonymous form that leads to a model call get a bot challenge and an IP-based rate limit.

// illustrative shape - the guard in front of a model call
const verdict = await checkQuota({ user, route });
// order: kill switch -> per-user window -> global daily spend
if (!verdict.ok) return tooMany(verdict.retryAfterSec);

const result = await model.run(input);
await recordSpend({ route, user, usage: result.usage }); // durable row

Anonymous callers get their own set of limits with their own names in configuration, separate from the logged-in ones, so tightening the public storefront can never loosen onboarding, or the other way around. Identical repeat requests are served from a response cache and never reach the guard at all, so the allowance is spent only on genuinely new inputs. The account owner bypasses every limit, which is consistent with how plan limits already work in the product.

On the paid path, the most effective guardrail is ordering. The expensive generation for a one-off report runs only after the payment is confirmed. The free part is a draft; the model call is triggered by a paying customer, so a bot filling in the form costs nothing.

The options, and why no single one is enough

Provider spend caps only. They are the last line and you should set them, but they are coarse (per project or account, not per user), slow to react, and not always hard. With one provider I used, the spend cap let the account run past its prepaid balance. A prepaid account with a hard balance is a real ceiling; a cap with overage is an alert with extra steps.

Per-user limits only. They stop one account from hammering an endpoint. They do nothing against a hundred fresh accounts, which on open signup cost an attacker a few minutes.

A global cap only. This one has a failure mode I found in my own audit. A daily cap shared by all users, with no per-user limit on one of the routes in front of it, means a single trial account can burn the shared budget and block every paying customer's scheduled work for the rest of the day. The cap converts a cost problem into an availability problem. Per-user limits exist to protect the cap from being used as a denial-of-service lever.

A challenge only. It raises the cost of automation, and it has its own silent failure: in my setup, a missing challenge secret meant verification returned success for everything, with no error and no log. The captcha was off and the page looked the same. After every secret rotation I now check that the secret is still present on the service that verifies it.

So I run all four, and the combination is cheap: a few indexed reads per request and one table for the switch.

Fail closed, except where it must not

The guard reads counters from the database. If that read fails, the guard refuses to spend and returns a retry-after. The reasoning is simple: if I cannot prove we are under the limit, we do not spend. The alternative, where a database hiccup quietly lifts every brake at once, is exactly the moment you do not want to discover.

Two deliberate exceptions keep this from turning into self-inflicted outages. A kill-switch table that does not exist yet means "not halted", because a fresh database has simply never been toggled. And the daily cap defaults to off: a mis-set cap must never silently kill legitimate work. Once a cap is configured, reads fail closed. Fail-closed behavior protects limits someone chose; missing configuration gets a safe default of its own.

The limit of a fixed dollar cap is growth. A static ceiling sized for today would cut off legitimate work during a good signup day. The plan I have written down for that is a dynamic cap derived from active subscriptions times a budget per customer, so the ceiling grows with revenue instead of with traffic.

How do you know what your AI features actually cost?

You need a spend ledger in the product, and you need environments that do not pollute it.

The ledger. Every model call writes a durable row: route, account, raw input and output tokens, and the computed cost. Raw tokens matter because providers change prices, and you want to be able to recompute history. The rate-limit window is a different table with a different job: it counts calls, it is purged after a couple of days, and it is not accounting. Mixing the two is how I found that my first cost panel summed only the background engine work, while two paid model paths on the dashboard were invisible in both the panel and the cap. The rule since then: a new endpoint that calls a model is not done until its cost lands in the ledger and counts toward the cap.

Separation at the provider. An environment is separated where the budget lives; the name of a key is only a label. Where a provider supports projects or modes, production and development get separate projects, and the development project has its own small hard cap; the answer to "how much does development cost" is then read from the provider panel per project. Where a provider has no test mode, non-production environments simply do not call it: locally, one-time codes print to the terminal instead of going through the email provider, and scans do not run without the worker. The production key never lands in a local file, both for blast radius and because development calls would muddy the production cost numbers.

With both in place, the ledger in production measures production and nothing else, which is what makes it usable for pricing decisions.

One model setting per job

The last lever is choosing which model runs where, and changing it without a deploy. The owner-only admin panel (behind a second factor) exposes the kill switch, the daily cap and the model per job at runtime: one setting for the paid deliverable, one for the free or pre-purchase helpers, one for classification. Keeping them separate means raising quality for paying customers never silently raises the cost of a free tool that anyone on the internet can trigger.

One detail matters more than it looks: every caller must resolve the model through the same function. I once had two flows that used the same kind of generation, one of which skipped the runtime override, so raising the model in the panel improved only one of them. A single resolver with a clear order (admin override, then environment, then default) removed that class of drift. Whether a cheaper model is good enough for a job is a question for an eval; I described how I run those in evals before merge.

What I would tell someone shipping an LLM feature tomorrow

• Treat "calls a model" as a security property of a route. Per-user limit, global daily cap, kill switch, before the first user sees it.
• Put a challenge and an IP limit on every public form that leads to spend, and check that the challenge is actually on after every secret change.
• Fail closed on the counters, but default a cap to off until someone sets it deliberately.
• Make the kill switch deploy-free and global. If it takes a deploy, it will be too slow on the day you need it.
• Charge before the expensive call wherever the product allows it.
• Log every call to a ledger with raw tokens, and keep that ledger separate from the rate-limit window.
• Separate environments at the provider, with a small hard cap on development, and prefer prepaid balances where a cap is not truly hard.
• Give each job its own model setting, resolved in one place.

Guardrails bound what a call may cost; they do not make its output trustworthy. That half of the problem is about schemas and validation, which I covered in tool use over parsing.