Confidence
merchant.confidence is a model confidence score between 0 and 1, not a calibrated probability. What Noryo has measured, and how to use it without being misled.
"confidence": 0.94merchant.confidence is a model confidence score, in [0, 1]. Do not read it as "94 % chance of being correct." It is not a calibrated probability of correctness.
What Noryo has measured
On 1,799 labelled cases across 34 countries, empirical accuracy per score band is not monotonic at the top of the range:
confidence band | Measured accuracy |
|---|---|
| 0.90 to 0.95 | 98.7 % |
| 0.95 to 0.99 | 96.0 % |
| above 0.99 | 76.5 % |
A 0.99 is therefore less reliable than a 0.90. A higher score does not guarantee a more accurate result.
How to use it
- Decide on
enrichment.statusfirst. Noryo's resolution policy is what turns an identity and a score intoresolvedorpartial. The status is the verdict; the score is only one of its ingredients. - Use the score as a ranking signal within a band, for example to order a review queue, not as a probability.
- Do not build a threshold on top of ours without measuring your own. If you want to filter further, measure accuracy on your own labelled transactions, band by band, before setting a value.
Where the score appears
| Situation | confidence |
|---|---|
resolved, refresh_recalculated | present, above the policy threshold |
partial with confidence_below_threshold | present, below the threshold |
partial for another reason | present |
not_found | absent: merchant is null |
The policy threshold is not published as a contract value: it belongs to Noryo's resolution policy.
The score is frozen with the result
The score is recorded with the result. A GET read returns the score of the enrichment requested, and a replay returns that of the replayed result. Only a refresh produces a new score, with a new enrichment_id.
What not to display
Do not show the score to your users as a percentage of certainty: that would promise a calibration Noryo has measured as absent. If you need to express a level of confidence, rely on the status (resolved versus partial).
Published on 2026-10-08 21:04 UTC (version 1).