Two Signals

A reranker cannot find what retrieval missed

A team adds a cross-encoder to a pipeline that has been returning the wrong passages. Latency roughly doubles. The failing queries fail identically. Someone concludes the reranker is bad.

The reranker is fine. It reordered a list of ten candidates that did not contain the answer, and it did so correctly — the best of ten wrong passages is still wrong.

Query: "does the warranty cover water damage on refurbished units"

CANDIDATE SET (10, from both retrievers, fused)
1. Warranty terms — general exclusions        ~
2. Refurbished unit certification             ~
3. Water damage assessment process            ~
4. Warranty registration steps                ✗
5. Accidental damage cover, new units          ~
   … (5 more, all ~ or ✗)

AFTER RERANKING
1. Water damage assessment process            ~
2. Warranty terms — general exclusions        ~
3. Accidental damage cover, new units         ~
4. Refurbished unit certification             ~
5. Warranty registration steps                ✗

The correct passage — refurbished units carry a reduced warranty that excludes liquid ingress — sat at rank 34 in the dense list and rank 41 in the lexical list. Both retrievers were asked for ten. It was never a candidate, so reranking could not promote it, and the reordering above is a genuine improvement to a set that cannot answer the question.

Two properties of the stage

Stated as plainly as possible, because almost every misuse follows from ignoring one of them.

Reranking is a permutation. Its input is a set and its output is the same set, ordered. It cannot add a document, cannot reach the index, and cannot know that a better passage exists.

Its cost is linear in the candidate count. A cross-encoder runs a model over every query-passage pair. There is no shared work to amortise across candidates the way an index amortises work across documents, which is precisely why the cross-encoder ranks better and costs more.

Put together: the reranker’s ceiling is set by the candidate set, and the candidate set is set by a stage that costs almost nothing. That asymmetry is where the money gets misallocated.

Recall failure and ordering failure look identical from outside

Both present as “the answer wasn’t in the results”. The distinction is one lookup and it decides which component to touch.

Is the correct passage anywhere in the candidate set the reranker received?
    no  → recall failure. Reranking is irrelevant. Fix retrieval.
    yes → ordering failure. Reranking is the right tool.

Which means the reranker doubles as a diagnostic. If you log the candidate set, you can answer this per query without re-running anything, and the distribution across a batch of failures tells you where the pipeline’s weakness actually is. A team that adds reranking and sees no improvement has learned something important: their failures were recall failures, and they now have evidence rather than a hypothesis — the same procedure applied to one query, aggregated.

Getting the candidate set right first

Four cheap things, all of which precede any reranker decision.

Retrieve deeper than you rerank. The candidate set should be generous, because retrieval depth is the cheap stage and the reranker’s job is to make a large set usable. A pipeline retrieving ten and reranking ten is paying model cost to reorder a set it should have made bigger for free.

Make sure both signals contribute. If one retriever’s results are being crowded out of the merge, the candidate set is effectively single-signal and it inherits that signal’s blind spots. Count, per query, how many final candidates came from each retriever; a large imbalance is a fusion or weighting issue masquerading as a reranking one.

Deduplicate before reranking, not after. Reranking near-identical candidates spends model cost on redundancy and, since scoring is per-pair, the duplicates will occupy adjacent top positions and crowd out coverage.

Check filters didn’t empty it. A predicate applied after retrieval can reduce a candidate set to almost nothing, and the reranker will diligently order what’s left.

What reranking is uniquely good at

None of the above means the stage isn’t worth it. Where it earns its cost:

Long candidate lists that need precision at the very top. When you can only pass a few passages downstream, the difference between rank 1 and rank 8 matters enormously, and a cross-encoder reading query and passage together makes distinctions neither retriever can.

Queries where the wording matters. Negation, qualifiers, conditions — “refurbished”, “excluding”, “only if” — are the parts of a query that both retrievers under-weight and a cross-encoder attends to directly. That’s the case in the example above, and it is why the reordering was a real improvement even though the answer was absent.

Turning depth into quality. A reranker is what makes a deep candidate set safe to use. Without one, increasing depth adds noise to the top of the list; with one, depth is nearly free improvement bounded only by rerank cost. The two stages are complements, which is why treating either as a substitute for the other goes wrong.

Supplying a comparable score. Reranker scores are far more comparable across queries than raw retrieval scores are, so this is where an abstention decision belongs if you need one — see why a similarity floor doesn’t hold.

The order to work in

Establish that your failures are ordering failures before you buy a reordering stage.

Take fifty failing queries. Retrieve at an absurd depth. Record whether the correct passage appears at all, and at what rank in each list. Three outcomes and three different projects:

  • Absent from both lists at any depth — recall problem. Look at the analyser, the query, the embedding fit, or whether the passage is even indexed. A reranker will do nothing.
  • Present, but below your current depth — a configuration change. Increase depth. Then a reranker becomes valuable, because you have just added candidates that need ordering.
  • Present within your current depth, ranked below the cut — an ordering problem, and the one case where reranking is the correct first move.

The uncomfortable finding, and it is common enough to expect: most teams who reach for a reranker are in the first two categories. The stage they need costs nothing and the stage they are buying costs latency on every query forever.

rerankingdiagnosis