Your similarity threshold has no zero

Somewhere in most retrieval pipelines there is a line like if score < 0.8: skip, and somewhere in most teams there is a belief that 0.8 means something. It does not. Cosine similarity between two embeddings is a number between minus one and one in theory, and in practice, for any given model, it lives in a narrow band whose floor is nowhere near zero. Unrelated sentences do not score zero. They score whatever that model's baseline is, and the baseline is different for every model, every fine-tune and every domain. A threshold copied from a tutorial, or carried across a model upgrade, is wrong by an offset nobody measured.

Why unrelated text is not orthogonal

The reason is a property of learned representations that has been documented since the first contextual models. Ethayarajh's study of contextualised representations found that they are not isotropic in any layer of the models examined: rather than spreading evenly across the space, the vectors cluster in a narrow cone, sharing a common direction. Two representations that have nothing to do with each other still point roughly the same way, so their cosine similarity is positive, often substantially so, before any meaning is involved at all.

Sentence embedding models inherit the same shape. The models are trained so that related pairs land closer than unrelated pairs, which is the property retrieval needs, but nothing in the training objective requires unrelated pairs to land at zero. They land where the cone puts them. On one model, random pairs of chunks from a corpus might average 0.3; on another, 0.6; on a third, the same model after a domain fine-tune, somewhere else again. The relevant pairs sit above that, in a band that overlaps the random band more than anyone likes.

The stranger score

The fix is one measurement, and I do it before setting any threshold. Sample a few thousand random pairs of chunks from your own corpus, embed them with the model you actually run, and record the mean and standard deviation of their cosine similarity. That mean is the stranger score: what two chunks that have nothing to do with each other look like, under this model, on this data. Every threshold in the system is then written as "k standard deviations above the stranger score", and the rule I enforce in review is that a threshold not written that way does not get merged.

Writing thresholds relative to the baseline does three things at once. It makes them portable across model swaps, because the k survives even though the raw number does not. It makes them meaningful to read: 2.5 standard deviations above strangers says what the author intended, 0.8 says nothing. And it forces the measurement that everyone was skipping, which is the part that actually catches problems.

The measurement itself is a few lines and runs in a minute on a corpus of tens of thousands of chunks. Pick pairs at random rather than adjacent ones, because adjacent chunks share a document and are not strangers. Use the same normalisation the query path uses, because a model that expects unit vectors gives different numbers if you forget. And store the result next to the index, dated and tagged with the model id, so that the next person can see what baseline the thresholds were written against.

idx = rng.choice(len(vecs), size=(5000, 2))
idx = idx[idx[:, 0] != idx[:, 1]]
sims = (vecs[idx[:, 0]] * vecs[idx[:, 1]]).sum(axis=1)   # unit vectors
stranger_mean, stranger_sd = sims.mean(), sims.std()
threshold = stranger_mean + k * stranger_sd

One more thing the number is not. It is not a substitute for top-k. Retrieval usually asks for the k best chunks and then applies a threshold to drop the ones that are not good enough, and the two controls answer different questions: k bounds the cost of the prompt, the threshold bounds the noise in it. A pipeline with only a threshold sends the model forty chunks on a broad question and none on a narrow one. A pipeline with only top-k sends five chunks whether or not any of them is relevant, which is how an assistant ends up citing a passage about the wrong product with complete confidence. Both controls belong in the pipeline, and only one of them has a baseline problem.

What the measurement catches

Three failures, in my experience, and all three are silent without it.

The model swap. A team upgrades from one embedding model to a newer one, re-embeds the corpus, and keeps the threshold. The new model's stranger score is lower, so the old threshold now admits pairs that are two standard deviations above strangers instead of four, and retrieval quietly gets noisier. Or the score is higher and the threshold now excludes nearly everything, and the assistant starts declining questions it used to answer. Nobody changed the threshold, so nobody looks at it.

The domain shift. The same model, but the corpus changes from product documentation to support tickets. Tickets are shorter, more repetitive and full of the same identifiers, so random pairs of them look more alike; the stranger score rises and a threshold set on the documentation is now too permissive. The pgvector distance operators and the sentence-transformers similarity utilities will compute whatever you ask of them and will never tell you that the baseline moved.

The mixed index. Two corpora with different stranger scores in one index, one threshold across both. This one is common in helpdesk systems, where procedure manuals, ticket histories and product pages are embedded into the same collection because that was the simplest thing to do on the day. Every query is now evaluated against two different baselines at once, and the results from the corpus with the higher floor crowd out the other. The measurement per corpus is what shows you the floors differ, and the fix is either separate thresholds or separate indexes.

What it does not fix

The stranger score is a baseline, not a relevance model. It tells you where unrelated pairs sit; it does not tell you where the relevant pairs sit, and the two distributions overlap. It also says nothing about direction: two chunks can be far above the baseline because they share a rare identifier, a product code or a customer name, and be about entirely different things. Lexical overlap of that kind is exactly what dense models are worst at distinguishing from meaning, which is why the labelled relevant set is not optional. Setting k requires the second distribution too: a few hundred labelled relevant pairs from your own data, so that you can see how much of the relevant band a given k keeps and how much of the random band it admits. That labelled set is the same one you need for any retrieval evaluation, so it is not extra work. It is the work, and the stranger score is the first thing to compute from it.

When I was tuning retrieval for the helpdesk assistant at CRIS, retrieval depth was the parameter that got the attention, and the threshold was the parameter that got copied. In hindsight the threshold was the more dangerous of the two, precisely because it looked like a number with a meaning. It has one, but the meaning is relative, and the zero it seems to be measured from does not exist.