Rate limiting is a safety control, not just a cost control

An engineer at a company with a public-facing AI feature told me about a realization that came in two parts, a month apart. First, they added rate limits, because the model calls cost money and unbounded usage would blow the budget. Reasonable, and they set the limits with the bill in mind. Then, a month later, someone started hammering the feature, running a high volume of requests to probe it, extract from it, and generally use it in ways it was not meant for. The team went to respond and realized the tool they needed was the rate limiting they had already built, just tuned for a different purpose. The limits set to protect the budget were also the limits that would protect against abuse. They had built a security control and filed it under cost.

Rate limiting on an AI feature does double duty, and teams usually only see one side of it. As a cost control, it keeps usage and the bill bounded. As a security control, it constrains what an abuser can do, how fast they can probe the feature, how much they can extract, how hard they can push it. Same mechanism, different purpose, and the security purpose is the one that gets overlooked. A feature with no rate limiting is not just financially exposed. It is open to being used at a volume and speed that enables abuse the feature was never meant to allow.

Unbounded access enables more than a big bill

When an AI feature has no meaningful limits on how much any one caller can use it, several kinds of abuse open up at once. Someone can run a high volume of requests to extract as much as possible from the feature, turning your model access into their data source. Someone can probe rapidly, sending many variations to find an input that breaks the feature or gets around a guard. Someone can simply overwhelm the feature, degrading it for everyone else. All of these depend on being able to make a lot of requests quickly, which is exactly what rate limiting prevents.

The cost framing hides this. When you think of rate limits as a way to control spend, you set them at whatever level keeps the bill acceptable, which may be far higher than the level that would stop abuse. A limit generous enough to protect the budget can still be generous enough to let an abuser run thousands of probing requests. The mechanism is there, but it is tuned for the wrong threat, so it does not do the security job it could.

The deeper point is that rate limiting is one of the few controls that directly bounds an attacker’s throughput. Many defenses try to catch bad requests. Rate limiting caps how many requests anyone can make at all, which limits every kind of high-volume abuse regardless of what the individual requests look like. That makes it a foundational control, not just a billing tool.

Set limits with abuse in mind

The fix is to design rate limiting as a security control, tuned for the abuse you want to prevent, not only for the budget you want to protect.

What that involves:

Limit per caller, not just in aggregate. A global limit protects the bill. A per-user or per-key limit constrains what any single abuser can do, which is the security-relevant bound.
Tune for the threat, not only the cost. Set limits tight enough to make high-volume extraction and rapid probing impractical, which may be tighter than the budget alone would require.
Watch for abnormal patterns. A caller suddenly running far more requests than normal is a signal worth catching, because it often precedes abuse.
Combine with other controls. Rate limiting caps throughput. Pair it with the access, output, and monitoring controls that handle what individual requests can do.

The teams that get this right recognize rate limiting as protecting two things at once and set it for both. The budget matters, and the abuse matters more, and a single well-tuned limit guards against each. Treating it as only a cost control leaves the security value on the table.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week, $2,500 engagement, we look at rate limiting as a security control, not just a line-item defense. Whether limits are set per caller, whether they are tight enough to constrain abuse and extraction, and whether abnormal usage gets noticed. A feature limited only for cost is often still open to the high-volume abuse the same mechanism could prevent. We would rather tune the limits for the threat in a diagnosis than during the week someone is hammering the feature.

Rate limiting protects your budget and your feature at the same time, and only one of those usually gets designed for. Set the limits with abuse in mind, so the control you built for cost also holds against the attacker.