Fine-tuning is a commitment, not a setting
Fine-tuning shows up in the docs next to temperature, as if it were another knob. It behaves more like hiring. You take on the data, the retraining, and the lock-in for as long as the feature lives.
Fine-tuning is a commitment, not a setting
Fine-tuning sits in the documentation next to parameters like temperature and top-p, which makes it look like one more setting you flip on when the base model falls short. It is not that kind of thing. A fine-tuned model is a dataset you now own, a training process you now repeat, and a version you now maintain. You are not adjusting a knob. You are taking on a small ongoing project that outlives the day you started it.
The model froze while the world moved
A semiconductor engineering leader who spent years across major chip makers gave me the analogy that stuck. He compared a fine-tuned model to a chip taped out to fixed silicon. Once it is fabricated, it is fixed. If the requirements move, you do not tweak it. You spin a new one, which costs real time and money.
His team fine-tuned a model to their internal terminology and it worked well. Then two things happened. The base models kept improving, and a newer base model with a good prompt started matching their tuned one out of the box. Their fine-tuned model, meanwhile, was frozen at the moment they trained it. To catch up they would have to fine-tune again on the newer base, redo the evaluation, and redeploy. The tuning was not a setting they had turned on. It was a treadmill they had stepped onto, and staying still meant falling behind.
The dataset is the real product
A head of data at a retail company put the weight in a different place. The hard part of her fine-tuning effort was never the training run. It was the dataset. Assembling clean, labeled, representative examples took her team most of a quarter, and keeping that dataset current as the business changed became a standing job that never ended.
She made the point that fine-tuning inherits every data problem you already have and adds a few. AI generally needs something like 85% or better data accuracy to be useful, and readiness is relative to the specific use case rather than a universal state you reach once. If the examples drift from what the business actually does now, the tuned model gets confidently wrong in ways that are harder to spot than a base model’s plain uncertainty. The dataset was the product, and the product needed an owner.
Before you fine-tune, get honest answers to these:
- Who owns the dataset next year. Curation is not a one-time task. It is a role for as long as the feature runs.
- What triggers a retrain. New base models and business change both age your tuned model. Decide what makes you redo it.
- Whether a good prompt gets you close enough. Prompting and retrieval often reach the bar without freezing you to a training snapshot.
- What the tuned model locks you into. A fine-tuned model does not move to another provider. That is a real cost of leaving.
How we approach it at Density Labs
When a team reaches for fine-tuning in the AI Opportunity Assessment, we treat it as the commitment it is, not the toggle it resembles. We ask whether prompting and retrieval clear the bar first, because they usually get closer than people expect and they keep you free to ride base-model upgrades. When fine-tuning is genuinely the right call, we scope the dataset ownership and the retrain trigger up front, so the commitment is one the team chose with eyes open rather than one they discovered a year in.
Fine-tuning can be exactly right, and it is never free after the first run. Start it the way you would start a hire, not the way you would flip a switch.