Query expansion, and how it drifts

A hospital’s policy search is asked for “annual leave carry-over”. The policies say “vacation accrual” and “unused PTO rollover”. Lexical retrieval finds nothing, because there is no term overlap to score, and the obvious repair is to add the missing words to the query.

That repair works, and it introduces a failure the original query didn’t have. Both halves are worth understanding before switching it on.

Query: "annual leave carry-over"

LEXICAL (BM25)                            DENSE (vector)
1. Annual shutdown calendar           ✗   1. Vacation accrual policy         ✓
2. Leave of absence, unpaid           ✗   2. Unused PTO rollover rules       ✓
3. Carry-over of training credits     ✗   3. Leave of absence, unpaid        ~
4. Annual review schedule             ✗   4. Holiday entitlement summary     ~
5. Public holiday list                ✗   5. Sick leave carry-over           ~

Expanded to annual leave carry-over vacation PTO accrual rollover:

LEXICAL (BM25)
1. Unused PTO rollover rules          ✓
2. Vacation accrual policy            ✓
3. Sick leave carry-over              ~
4. PTO request workflow               ~
5. Accrual caps by tenure             ~

What expansion is actually fixing

Vocabulary mismatch, and only that. Term-based retrieval scores overlap between query tokens and document tokens; when the corpus and the user use different words for the same thing, the overlap is zero and no parameter change rescues it. This is the structural limit of lexical retrieval, and adding the corpus’s vocabulary to the query is the direct repair.

Note what the paired lists show: dense retrieval had no trouble with this query. Expansion is mostly a lexical intervention, and its value depends on how much you are relying on the lexical signal. In a hybrid pipeline where dense retrieval already covers paraphrase, expansion’s marginal value drops — it stops being “the fix” and becomes “the thing that makes the lexical list a useful fusion partner on paraphrase queries”, which is a smaller but not worthless job.

Where the terms come from, and how each source drifts

A hand-maintained synonym list. Your domain’s terms mapped to their variants. Precise, auditable, predictable, and someone has to maintain it. The drift is organisational: the list stops being updated, new product names never enter it, and its coverage silently decays.

A general thesaurus. Wide coverage, and the failure is immediate and severe, because words are polysemous and a thesaurus doesn’t know your domain. In a logistics corpus, expanding “container” with “receptacle” and “vessel” adds a shipping-vessel sense that competes with the intended one. A general thesaurus expands into senses your corpus doesn’t use, and lexical retrieval cannot tell which sense a document meant.

Terms mined from your own corpus. Co-occurrence statistics, or terms drawn from the query’s own top results. Domain-appropriate by construction, since every candidate term appears in your documents.

Terms generated by a language model. Fluent, broad, and it invents plausible vocabulary your corpus may not contain — harmless when absent, harmful when the invented term is common in a different context within your corpus. Same failure as the thesaurus, arriving by a different route.

Terms from the query log. What users type before and after reformulating. Genuinely the highest quality source when you have the volume, because it captures the mismatch as it actually occurs, and useless before you have traffic.

Pseudo-relevance feedback, and the reason it is risky

The self-contained version of expansion: run the query, take the top few results, harvest their distinctive terms, run the query again with those terms added.

It works when the initial results are relevant. When they aren’t, it amplifies the error — the second query is now tuned toward the vocabulary of documents that were already wrong, and it retrieves more of them, more confidently. The technique’s quality is bounded by the quality of the first retrieval, and it degrades fastest exactly when you needed it most.

Two mitigations, both partial. Weight expansion terms far below original terms, so feedback nudges rather than steers. And gate the whole procedure on the first retrieval looking healthy — if the top results have no term in common with each other, they are not a coherent basis for anything, and skipping expansion is better than compounding.

It also costs a second retrieval round, which doubles the cheapest stage of your pipeline. That is usually affordable; it is not free.

Drift, concretely

The general mechanism has a name worth using: query drift — the expanded query is no longer about the same thing.

Every added term is a claim about what the user meant, and each one shifts the query’s centre of mass. Add enough and the original terms are a minority of the query, at which point the highest-scoring document is the one matching the expansion rather than the request. The symptom is subtle: results are topically plausible, related to something the user mentioned, and not responsive. That reads as vague ranking rather than as a broken component, so it survives review.

Three practices that keep it contained:

Weight, don’t append. Most lexical engines allow per-term weights. Expansion terms at a fraction of the original terms’ weight preserve the query’s identity while still admitting documents that would otherwise score zero. Appending unweighted terms treats a guess as evidence.

Cap the count. A small number of added terms, chosen by whatever score your source provides. Long expansions drift monotonically.

Never expand identifiers or quoted spans. A SKU has no synonyms. Expanding an identifier query destroys precisely the rare-term advantage that made it answerable, replacing a query that would have worked with one that returns its neighbours. This is the single most damaging expansion mistake, because identifier queries are the ones lexical retrieval was carrying.

Expansion versus the alternatives

Expansion competes with two other ways to solve vocabulary mismatch, and it is not obviously the best of them.

Dense retrieval handles paraphrase natively. If you are running it anyway, the mismatch is already partly covered and expansion is a second, redundant, drift-prone attempt at the same job.

Learned sparse retrieval performs a form of expansion inside the model, at indexing time, with the term weights learned rather than guessed — a different trade with different costs.

Index-side synonym expansion applies the mapping when documents are indexed rather than when queries run. Faster at query time and it makes changing the synonym list a reindex, which is a real operational cost and, for a large corpus, the deciding one.

Telling whether it’s your problem

Take the queries where lexical retrieval returned nothing and dense retrieval returned something useful. Read fifty of them. If the pattern is a user word and a corpus word for the same concept, expansion targets your problem. If the pattern is conversational phrasing, a rewrite is the cheaper intervention. If it is identifiers, expansion will make things worse.

And whatever you deploy, log the expanded query alongside the original. An expansion you cannot see is an expansion you cannot debug, and drift is invisible from the outside.