The tool the model would not call when it should have

A support lead at a software company told me about a bot that kept giving confidently wrong answers about account status. The odd part was that the bot had a tool wired up that could look up the exact, current status for any account. It just did not always use it. For some questions the model reached for the live lookup. For others it answered from what it seemed to already know, and that knowledge was stale. The tool was right there. The model chose not to call it.

Availability is not a trigger

The team’s mental model was that giving the model a tool would make it use the tool. Wire up the lookup, and questions about account status get answered by the lookup. That is not how it works. A tool being available is an option, not an instruction. On every turn the model decides whether calling the tool is worth it, and sometimes it decides the answer is already in hand and skips the call.

This is easy to miss because it works often enough to look reliable. The model calls the tool on the clear cases, you see it working, and you assume it always will. Then a question comes in phrased a little differently, or the model feels confident from context, and it answers straight from its own weights. The result reads just as fluent as a tool-backed answer. It is simply out of date, because it never checked.

The model answered the question it thought it was asked

The support lead pulled the traces and the pattern was clear. When a user asked “is my account active,” the model usually called the lookup. When a user asked something like “why can’t I log in,” the model sometimes reasoned about login problems in general and answered without ever checking the actual account. It had drifted from “look this up” to “explain this,” and explaining did not feel to the model like it needed the tool.

The tool’s description was part of the cause. It said what the tool did, tersely, but not when the model should reach for it. Left to infer the boundary, the model drew it in the wrong place. It treated the lookup as something for direct status questions and not for the indirect ones that still depended on live status.

Make when-to-call explicit, and watch for silent skips

Getting reliable tool use takes two things: telling the model clearly when to call, and checking that it did.

Write when-to-call into the tool description. Do not only describe what the tool does. Say when the model must use it, with the kinds of questions that require live data spelled out. The description is where the model decides.
Reinforce it in the system prompt. For data that must be current, instruct the model plainly that it may not answer from memory and has to call the tool first.
Detect the silent non-call. Log when the model answers a question of a type that should have triggered a lookup but did not. That gap is invisible unless you measure it, and it is where the stale answers hide.
Correct it when you catch it. When you find a class of question the model answers without checking, tighten the description and add it to your test cases so the fix sticks.

The goal is to stop treating tool use as automatic and start treating it as a behavior you specify and verify, the same as any other.

How we approach it at Density Labs

In the AI Opportunity Assessment, our fixed two-week engagement priced at $2,500, we do not assume that a wired-up tool is a used tool. We look at the traces to see when the model actually calls it and when it answers around it, we make the when-to-call explicit in the description and the prompt, and we set up detection for the silent non-calls that produce stale answers. A tool the model skips is worse than no tool, because it gives you the confidence of live data without the data.

Wiring up a tool does not mean the model will use it. Tell it when to call, and check that it did.