What Is an LLM Confidence Score? A Practical Guide

Ask a large language model how confident it is in an answer and it will give you a number. That number feels like it should be useful: gate expensive verification on the low-confidence answers, let the high-confidence ones through untouched, and you've built a cheap quality control system for free. The problem is that a model's self-reported confidence and its actual accuracy are often two different things wearing the same number. Here is what a confidence score actually is, why the naive version of it is unreliable, and what "calibrated" means when a vendor claims it.

What a confidence score is, and what it usually isn't

A confidence score is a number, typically between 0 and 1 or expressed as a percentage, that a model reports alongside an answer to indicate how sure it is that the answer is correct. There are two structurally different ways a model can produce that number, and the difference matters more than the number itself.

The first is a model reporting on itself: you ask a chat model "how confident are you in that answer, from 0 to 100," and it generates a number as part of its text output, the same way it generates the rest of the sentence. This number is not derived from anything mechanical inside the model's reasoning process. It is a token the model predicts is plausible to say next, shaped by patterns in its training data about how confident answers to similar-sounding questions tend to sound, not a measurement of the model's internal certainty.

The second is a score derived from the model's actual output distribution or from a purpose-built classifier trained specifically to predict correctness, which is a genuinely different kind of number because it is grounded in something measurable rather than generated as prose. This is the kind of score that a well-designed triage or verification system can actually act on.

Why an LLM's stated confidence is often unreliable

The core finding across the machine learning literature on this is that many modern neural networks tend to be overconfident relative to their actual accuracy, and the gap is not small or random, it is systematic. Guo, Pleiss, Sun, and Weinberger's widely cited paper on the calibration of modern neural networks documented this pattern across image classification models and found that increases in model capacity and training sophistication had, if anything, made the miscalibration problem worse rather than better compared to earlier, simpler models. The same underlying issue shows up in large language models asked to self-report confidence: a model that says it is 95% sure is not reliably right 95% of the time, and the gap between stated and actual confidence tends to widen for exactly the harder, more ambiguous questions where an accurate confidence signal would matter most.

This matters enormously for any system trying to use confidence as a gate. If you route "gate expensive verification only on answers below 70% confidence," and the model's 70% and 95% bands are both actually right about 80% of the time, your gate isn't sorting anything, it's closer to a coin flip with extra steps. A miscalibrated confidence score used to control spend doesn't save you money selectively, it either checks everything regardless of whether it needed checking, or it skips checks on exactly the answers most likely to be wrong.

What calibration means, and how to check it

Calibration is the specific property that makes a confidence score trustworthy: among all the answers a model rates at, say, 80% confidence, roughly 80% of them should actually be correct. Not close to 80%, not "usually high," but measurably 80% across a large enough sample of predictions at that confidence band. This is a testable claim, not a vibe. You take a large batch of predictions, bucket them by their stated confidence, and check the actual accuracy within each bucket against the stated number.

A standard visual check for this is a reliability diagram, plotting stated confidence on one axis against observed accuracy on the other; a perfectly calibrated model traces the diagonal, and a model that's overconfident bows below it. Running that check yourself, on your own traffic, rather than taking a vendor's calibration claim on faith, is the only way to know whether a confidence number is doing real work in your pipeline.

How a calibrated confidence score gets used in practice

TypeSafe AI's founder, Diogo Almeida, has framed the gap that motivated building a purpose-made decision model, Jev, TypeSafe AI's System One model, around exactly this problem: chat models generate plausible text, but a lot of production systems need a number that behaves like a real probability, not one that merely sounds confident. In llm11's routing pipeline, Jev's calibrated confidence score is what decides how much verification a given answer earns: a low-confidence routing or criticality call triggers a heavier pass through the verification ladder, while a genuinely high-confidence call doesn't. That only works, and only saves the money it's supposed to save, if the confidence number is actually calibrated rather than merely self-reported. llm11's own page on Jev is direct about the scope of that claim: the interesting property is calibration, not raw accuracy being higher than a chat model's, and the broader piece on what TypeSafe AI publishes about Jev covers which of the vendor's own figures llm11 treats as independently verified versus simply reported.

The wider lesson holds regardless of which vendor or model you're evaluating: before you build any logic that branches on a model's confidence score, whether that's routing, verification gating, or a UI that shows users a certainty percentage, check whether that number was ever validated against real outcomes. If nobody has run a calibration check on it, treat it as decoration rather than a control signal until someone has.

Frequently asked questions

What is a confidence score in the context of an LLM?

A number, usually a probability or percentage, that a model or a system built around it reports to indicate how likely an answer is to be correct. It can come from the model generating a plausible-sounding number as part of its text output, which is unreliable, or from a mechanically derived, calibrated score, which is a meaningfully different and more trustworthy kind of measurement.

Why isn't a model's self-reported confidence trustworthy?

Because when a chat model states a confidence percentage as part of its generated answer, that number is produced the same way the rest of the text is: as a plausible next token, not as a measurement of the model's actual internal certainty. Research on neural network calibration has repeatedly found that self-reported or output-derived confidence tends to run higher than actual accuracy, particularly on harder questions.

What does it mean for a confidence score to be "calibrated"?

It means that among all predictions given a particular confidence level, the actual accuracy at that level matches the stated number. If a model calls 1,000 predictions "80% confident," roughly 800 of them should be correct. That is a property you can test directly against real outcomes, not a claim you have to take on trust.

Can I check whether a model's confidence score is actually calibrated?

Yes. Collect a large batch of the model's predictions along with their stated confidence, bucket them by confidence level, and compare the actual accuracy in each bucket against the stated number, often visualised as a reliability diagram. A well-calibrated model's plotted line sits close to the diagonal; a poorly calibrated one bows away from it.

Why does calibration matter for LLM routing specifically?

Because a router that gates expensive verification checks on a confidence score only saves money and catches real errors if that score reflects real accuracy. A miscalibrated confidence number used as a gate either checks everything regardless of need or, worse, skips checks on exactly the answers most likely to be wrong, which defeats the purpose of gating in the first place.