Two Signals

Routing a query before you retrieve it

Two queries arrive at a parts catalogue a second apart: 4471-B-REV2 and “what fitting do I need for a 22mm copper pipe”. Running both retrievers on both queries costs twice as much as necessary and, for the first one, produces a merged list in which the dense contribution is pure noise.

Routing is the alternative to fusing: decide what kind of query this is, and send it where it will be answered. It is cheaper, it is faster, and it converts a graceful degradation into a hard failure.

Query: "4471-B-REV2"

LEXICAL (BM25)                            DENSE (vector)
1. Part 4471-B rev 2 datasheet        ✓   1. Part 4470-A datasheet            ✗
2. 4471 family compatibility           ~   2. Part 4471-C datasheet           ✗
3. Revision history, 4471 series       ~   3. Revision numbering scheme        ~
4. Superseded parts list               ~   4. Part 4482-B datasheet           ✗
5. Ordering guide                      ✗   5. Catalogue structure overview     ✗

Fusing those two lists gives the dense retriever’s four wrong answers rank contributions that compete with the lexical retriever’s correct one. Routing this query to lexical alone gives a better result for half the cost.

What routing decides

Not one decision but a family of them, and it is worth being clear about which you are making:

  • Which retriever — lexical only, dense only, or both.
  • Whether to transform — rewrite, expand, decompose, or leave alone.
  • Whether to retrieve at all — an exact identifier lookup is a key fetch, not a search.
  • How deep and whether to rerank, since not every query needs the expensive stages.

The first is what “routing” usually means. The others are where most of the value is, because they save more and risk less: choosing not to expand an identifier query cannot make it worse, while choosing the wrong retriever can make it unanswerable.

Classes that are worth separating

Ordered by how reliably you can detect them.

Identifier or code. Matches a known pattern — your SKU format, an error code prefix, a version string, a file path. Detected by regex against a small set of patterns you already know, because you assigned them. High precision, and the payoff is the largest available: this is the class where dense retrieval is structurally wrong.

Quoted span. The user used quotation marks. They are asking for exact matching; give them exact matching. Trivially detected and routinely ignored.

Exact key. A complete identifier that exists in your metadata. Fetch it. Ranking a set of one is nothing but latency.

Natural-language question. Sentence-shaped, has a question word or a question mark, filler words present. Dense retrieval is the primary signal and a rewrite is likely to help.

Short keyword phrase. Two or three content words, no sentence structure. The genuinely ambiguous middle, and the class where fusing both signals is the right answer because you cannot tell which one will fire.

Constraint-bearing. Contains a date, a numeric comparison, a status. Needs predicate extraction more than it needs a retriever choice.

Negated. Contains “not”, “without”, “except”. Neither retriever handles it; the routing decision is to transform or to ask the user, not to pick a signal.

Rules before models

The temptation is to train a classifier. Resist it long enough to write down the rules, because the rules are inspectable, free, deterministic, and cover most of the distribution.

Regexes for your own identifier formats. Quote detection. Token count. Presence of a question mark or question word. A digit-density test. A check for whether every token exists in the corpus vocabulary, which separates typos from unknown terms. That set of tests takes an afternoon and will classify the classes above with precision high enough to route on.

Where a model earns its place is on the boundary between “natural-language question” and “keyword phrase”, and on intent distinctions that surface features cannot express — and both of those are the classes where the routing decision matters least, since both are served by fusing. The cases a classifier is needed for are the cases where routing is optional, and the cases where routing pays are the cases a regex handles. That is a strong argument for keeping this stage boring.

The failure mode, stated fairly

A misrouted query has no second chance. If a natural-language question is classified as an identifier and sent to lexical retrieval alone, the dense list that would have answered it does not exist. There is no partial credit, no degraded ranking — the pipeline is simply missing the signal.

Compare with fusion, whose failure is that the wrong signal’s results dilute the right one’s, costing positions rather than the answer. Fusion degrades; routing breaks.

Three ways to keep the breakage contained:

Route only on high-precision tests. A regex matching your SKU format is a fact, not a prediction. Route on facts and fuse on guesses. This single rule captures most of what good routing practice amounts to.

Prefer soft routing where you are uncertain. Instead of choosing one retriever, run both and weight them according to the classification. A wrong classification then costs ranking positions rather than the whole list.

Fall back on an empty result. If the routed path returns nothing or nothing above a sanity floor, re-run with both retrievers. Costs an extra round on the queries that were misrouted, which are exactly the queries worth spending on.

What routing buys besides quality

Two things that get undersold because they aren’t relevance improvements.

Cost and latency. Not running a retriever is the cheapest possible optimisation, and identifier queries are a large fraction of traffic in technical and catalogue corpora. Skipping reranking on queries that don’t need it saves the expensive stage where it adds nothing.

Explainability. A routed pipeline logs why it did what it did — “classified as identifier, lexical only, no expansion”. When a result set is wrong, you know which path ran. In a fused pipeline the equivalent question requires reconstructing both lists and the merge, and the diagnostic procedure has more steps. Logging the route is one field and it shortens every future investigation.

Telling whether it’s worth building

Classify a sample of your query log with the rule set described above, and count. The distribution decides it:

  • A large identifier or exact-key share — route, immediately. This is the highest-return change available in a hybrid pipeline, and it is a regex.
  • Almost everything is a natural-language question — routing has little to separate. Spend the effort on rewriting and reranking instead.
  • Broadly spread across classes — route the confident classes, fuse the rest, and treat the middle as permanently ambiguous rather than as a classifier you haven’t built yet.

Then check the routes against outcomes: for each class, what fraction of queries ended with the user finding something? A class with a poor outcome rate and a confident route is a route worth doubting, and that is the only ongoing monitoring this stage needs.

query-transformtradeoffs