Why “90% accurate” is a number that means almost nothing

Someone reports that the AI feature is 90 percent accurate and the room relaxes. It sounds like a grade. It is closer to a rumor. A single accuracy figure hides the three things that decide whether the feature is safe to ship: which inputs it was measured on, who judged the outputs right, and how the cases were weighted. Change any of those and the same feature reports a different number.

The average smuggles in a distribution

A staff engineer at a networking-hardware company put it in a way that stuck. An average is a story with the interesting parts removed. Ninety percent accurate can mean the feature is a little wrong on almost everything, or nearly perfect on the common cases and badly broken on a small, important slice. Those are different products. One is shippable. One will produce a headline. The single number cannot tell them apart.

He tests components in isolation for exactly this reason, running a known input and checking the known output so a failure has nowhere to hide inside an aggregate. The same instinct applies to an AI feature. You want the score broken out by the kind of input, not blended into one comforting figure.

Accurate against what, exactly

A semiconductor engineering leader made the second point. He asked, every time a number crossed his desk, what it was measured against. A test set of easy inputs produces a high score that means nothing. A test set weighted toward the cases that actually occur produces a number you can plan with. If the evaluation set was built from the happy path, ninety percent is a description of the happy path and not of the feature.

There is a quieter trap under this. Who decided the outputs were correct. If one person eyeballed the results in an afternoon, the accuracy number carries that person’s mood and blind spots. A real figure needs a defined rule for what counts as right, applied the same way every time.

What to ask before you trust an accuracy number

Accurate on which inputs, and does the test set match the traffic you actually see?
Broken out how, by input type and by severity, so a small critical failure does not vanish into a big average.
Judged by whom, against a written definition of correct, not one reviewer’s gut.
Weighted by cost, because a wrong answer on a rare high-stakes case is not paid off by a hundred right answers on trivial ones.

The number that would actually help

A useful report is not one number. It is a short table. Accuracy on the common cases, accuracy on the rare ones, accuracy on the inputs where a mistake is expensive, and the count of cases behind each. That table takes an hour longer to produce, and it is the difference between a team that knows where its feature is weak and a team that is about to find out from a customer.

Ninety percent accurate is an answer to a question nobody asked carefully. Ask the careful version, and the single number falls apart into the handful of numbers you actually needed.