A search score is an order, not a grade

Leon

• 2026-01-21

You search for a supplier notice about a cracked bracket. The top result has a score of 0.84 and a full bar next to it. The next one has 0.79. It is hard not to read that as "84% relevant" and "almost as good". A maintenance planner asks whether 0.6 is good enough to trust. An engineer asks why yesterday's search scored higher than today's.

None of these questions can be answered from the number. A search score puts results in order. It does not say how good any one result is. We wrote a short paper about this, What a search score means, for anyone who reads ranked results in Curiosity Studio, not only for engineers. This post gives its argument and its best examples.

Think of a soup contest

The paper opens with a soup contest. Five cooks each make a soup. A judge tastes them, puts them in order and gives each one a number. From that you can say which soup came first, whether soup B beat soup C, and where the list drops off. You cannot say whether soup A is tasty, whether it is twice as good as soup B, or whether it is good enough to serve.

All five soups can be bad. Then the winner is only the least bad one. A search works the same way. It can return a first result even when nothing in your data answers the question.

Why the top bar is always full

Result lists in Studio often draw a bar for each score. The bar is as long as the score, compared with the best score in that list. So the top result always has a full bar.

The paper shows two searches side by side. In search A the best score is 0.84 and the next ones are 0.79, 0.66, 0.41 and 0.38. In search B the best is 0.29, then 0.27, 0.22, 0.12 and 0.11. Both lists start with the same full bar. Search A found strong matches. Search B did not. The bar tells you about the order inside one list, and nothing about the list compared with another one.

What goes into the number

A score in Studio is built from several parts, each with its own scale:

Search adds the parts up, with weights, into one number. That number has no fixed top. Which parts count also depends on the search. A plain keyword search mainly uses search terms and recency. A similarity view can add meaning and shared links. So a score of 0.6 from one kind of search and 0.6 from another are not the same 0.6.

Why it is not a percentage

The paper gives four reasons, each with an everyday example.

1. The scale is not fixed. The same number can mean different things in different searches. In German schools 1 is the best mark. In some other countries 1 is the worst.
2. A score only means something inside its list. The tallest child in a class is not always a tall person.
3. There is no whole. A percentage is a share of a whole, like 50% of a pizza. A score has no pizza. No number means "perfect".
4. A percentage looks more exact than it is. "87%" sounds like "87% sure". Imagine an app that says "Your day: 87% nice". Nobody knows what that measures.

Even the parts that come with a familiar range are not percentages. Text similarity is usually measured as cosine similarity, which lands between -1 and 1. Piotr Migdał makes the point in Don't use cosine similarity carelessly: values that usually fall between 0 and 1 do not represent probabilities, and "the value 0.6 tells little if we have something really similar, or not so much". If one ingredient of a score has no fixed meaning, the sum cannot have one either.

What a score can tell you

The paper sorts the questions people ask into those a score answers and those it does not:

The gap is the part that is easy to skip. In search A above, the drop from 0.66 to 0.41 is larger than any other step in the list. A big gap between two results often means a change in kind. Above it you may have the maintenance logs for that bracket. Below it, field reports that only share a few words with your question.

Three habits for reading a list

The paper closes with three habits that take a few seconds each.

• Read from the top. The order is the signal. Start at result 1 and go down.
• Look for a drop. A large gap between neighbors usually marks where the strong matches end.
• Judge by content. Open the result and read it. Only the content shows whether it fits.

The same holds when software does the reading. When an assistant in Studio answers from your part records and tickets, its answer traces back to a record you can open. The number decided what to read first. The record decides what is true.

Read the paper

The paper is ten pages, written in plain language, with the soup contest, the two searches and the tables above, plus a short glossary of terms like score, rank, signal and gap. Hand it to the people on your team who read ranked results all day and wonder what the bar means. Read it on its page, or download the PDF directly.

Read the whitepaper (PDF)

Further reading

• Don't use cosine similarity carelessly, Piotr Migdał. Why a similarity value between 0 and 1 is not a probability, with small examples where the closest match is the wrong kind of match.
• Is cosine-similarity of embeddings really about similarity?, Harald Steck, Chaitanya Ekanadham and Nathan Kallus (ACM Web Conference 2024 companion). The formal case that cosine similarity can yield arbitrary similarities, depending on how the model was trained.
• Better RAG results with reciprocal rank fusion and hybrid search, John Wang, Assembled. A team that could not find stable weights for mixing keyword and vector scores, and combined their lists by rank instead.