Secrets management for model keys, and the leak that follows a demo

A backend lead at a fintech tool had a demo to give on a Thursday. The model integration worked on his machine and nowhere else, because the key lived in his shell environment. To make the demo run on the shared staging box, he pasted the key straight into the config file, committed it, and moved on. The demo went well. Everyone forgot the key was there.

Weeks later a contractor cloned the repo, and the key was sitting in the history in plain text. Nothing bad happened that time. It could have. A model key is a spending credential. Whoever holds it can run calls against the provider on the account’s dime, and the first sign of a problem is often the bill, not an alert. The commit that saved a Thursday became a credential nobody was tracking, living in a place plenty of people could read.

I have seen the same key end up somewhere worse: in a frontend bundle. To call the model from the browser during a rushed build, someone puts the key in client-side code. Now the key ships to every visitor. Anyone who opens the network tab can read it. That is a leak on a timer.

The key you rushed is the key you have to rotate

A model key is a live credential to a paid service, and it should be handled like any other. The reason these keys leak is not carelessness in the moment. It is that “just get the demo working” and “handle this secret correctly” feel like different jobs, and under a deadline only the first one is due. The key goes somewhere convenient, the demo ships, and the convenient place is a repo, a config file, or a browser bundle. Convenient and safe are not the same place.

Two properties matter most. A key should be scoped, so that if it does leak the damage is bounded to what that key was allowed to do. And it should be rotatable, so that when it leaks you can kill it in minutes without a code change. A key you cannot rotate quickly is a key you will be afraid to revoke, which means you will leave a known-leaked credential live because turning it off breaks something. That fear is the real cost of doing it wrong.

What good teams do

Keep keys in a secret manager, injected at runtime. Not in the repo, not in a config file, not in the image.
Never put a model key in client-side code. Calls to the provider go through your backend, which holds the key.
Scope keys to the least they need, and use separate keys per environment so staging cannot spend production’s budget.
Rotate on a schedule, and rotate immediately on any suspicion. Make rotation a routine step, not an emergency.
Scan commits for secrets before they land, so a pasted key is caught at the door instead of in an audit.

How we approach it at Density Labs

Our AI Opportunity Assessment is a fixed two-week engagement at $2,500 that scopes real production work before you build. Secret handling is one of the first things we check, because it is cheap to fix before launch and expensive after. We look for where the model key actually lives: environment files, commit history, frontend bundles, the places a demo deadline tends to push it. We check whether keys are scoped, whether they differ across environments, and how fast the team could rotate one if it leaked today.

The finding is usually the same. The key that runs the feature is the key someone hard-coded to hit a deadline, and nobody moved it afterward. It works, so it is invisible, until the day it is not.

Treat every model key as a spending credential that will eventually leak. Scope it, store it in a secret manager, keep it off the client, and make rotation something you can do before lunch.