When the model answered a different question than the one you asked

A data scientist at a travel company brought me a transcript she found maddening. The model had answered a question with total confidence. The answer was well-written and completely beside the point. It was addressing something she had not asked.

Her prompt asked the model to look at a booking and say whether the customer should be offered a refund or a rebooking. The answer that came back explained, at length, the difference between refunds and rebookings as policies. Clear, correct, and useless. She wanted a decision about this customer. She got a definition.

A confident wrong answer looks like a model problem

She was ready to log it as a reasoning failure. The model could not follow a simple instruction. It had gone off and answered something adjacent instead of the thing in front of it.

We read the prompt together. It asked the model to “consider the refund and rebooking options for this booking and explain them.” Read it slowly. That sentence contains two requests. One is a decision: pick refund or rebooking for this customer. The other is an explanation: describe what the options are. The model saw both and answered the one it could answer without risk, the explanation. It did the safe half of an ambiguous instruction.

The model did not misunderstand. The prompt genuinely asked more than one thing, and it chose. When a request is ambiguous, the model does not stop and ask for clarification. It resolves the ambiguity on its own, silently, and commits to whichever reading it landed on. The confidence you see is not confidence that it answered your question. It is confidence in the answer to the question it decided you meant.

Ambiguity becomes a wrong answer, not an error

This is the part teams miss. An ambiguous prompt does not fail loudly. It does not return an error or a blank. It returns a fluent, assured answer to a slightly different question, and that answer reads exactly like a correct one until you notice it is solving the wrong problem. The cost is hidden, because nothing looks broken.

Overloaded prompts are the usual cause. You want the model to do a few related things, so you list them in one sentence, joined by “and.” The model now has to guess which one is the real ask, or it tries to serve all of them and serves none well. Vague verbs make it worse. “Consider,” “look at,” “think about” do not name an output. They invite the model to produce something in the neighborhood and hope it lands.

The fix is to make the question single and explicit:

Ask one thing per prompt. If you need a decision, ask for the decision. Do not also ask for an explanation in the same breath unless you name both as separate outputs.
Name the output you want. “Return refund or rebooking” beats “consider the options.” A concrete verb removes the room to wander.
Split compound requests. When a prompt has an “and” joining two asks, break it into two, or define each as its own field in the response.
Read your prompt as an adversary. If there is a cheaper, safer question hiding inside yours, the model may answer that one. Close the gap before you ship.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week, $2,500 engagement that scopes real production work before you build, we test prompts against the exact questions they are meant to answer, and we watch for the model answering a nearby one. When it happens, we do not reach for a bigger model. We read the prompt for ambiguity, because that is almost always where the drift starts. A single, explicit question is cheaper to write than a quarter of debugging why the answers keep missing.

The model will answer something. Make sure the something is your question.