Learned sparse retrieval, the third signal
A museum archive search is asked for “photographs of the dockyard before the fire”. The catalogue records say “shipyard”, they are dated, and one of them describes a blaze. Lexical retrieval misses on vocabulary; dense retrieval finds shipyard material but ranks a general history of the docks above the photograph collection.
There is a third option that is neither: a model that produces a sparse representation — a set of weighted terms, like a lexical index entry — but chooses the terms and weights itself, including terms the document never used.
Query: "photographs of the dockyard before the fire"
LEXICAL (BM25) LEARNED SPARSE
1. Dockyard closure records ~ 1. Shipyard photographic survey ✓
2. Fire safety in storage ✗ 2. Dock area, images 1890-1910 ✓
3. Photographic conservation notes ✗ 3. Dockyard closure records ~
4. Dock labour disputes ✗ 4. Waterfront fire, aftermath ~
5. Fire of 1904, press cuttings ~ 5. Fire of 1904, press cuttings ~What it is, mechanically
A lexical index stores, per document, which terms it contains and how heavily each should count. Those weights come from a formula: term frequency in the document, term rarity across the corpus, document length. The formula has two parameters and no knowledge of what the words mean.
A learned sparse model replaces the formula with a trained model. It reads the document and emits a vector over the vocabulary in which most entries are zero and the non-zero ones are its predicted importance. Two consequences follow:
- Weights reflect importance, not frequency. A term central to the document’s subject can outweigh a term repeated more often. The formula cannot express that.
- Terms absent from the text can be assigned weight. The model may put weight on shipyard for a document that only says dockyard. This is the part that fixes vocabulary mismatch, and it is expansion performed at indexing time by a model rather than at query time by a guess.
The output is still a sparse term vector, which means it can be stored and searched by an inverted index — the same data structure lexical retrieval uses. That is the whole appeal.
What it inherits from the lexical side
Exact terms still match exactly. An identifier in the document, if the model assigns it weight, is retrievable by that identifier. Whether it does assign weight is a property of the model and its vocabulary, which is the caveat below — but the representation is capable of it, where a dense embedding structurally is not.
Results are inspectable. You can look at which terms matched and what each contributed. When a query goes wrong you get an explanation, not a distance. That is a genuine operational advantage over dense retrieval, where diagnosing a miss means reasoning about geometry.
It runs on an inverted index. Familiar infrastructure, familiar operational behaviour, and filters work the way they do for term queries rather than the way they do for similarity search.
What it costs
Indexing requires a model pass over every document. Every document, once, plus every update. That is the same class of cost as embedding a corpus, and if you are already embedding, it is a second pass of the same magnitude.
Query time requires a model pass over the query. Small, but it is a model call in front of retrieval, so it is latency you cannot overlap.
The index gets larger and the queries get slower than plain lexical. Expansion means each document carries more non-zero terms than it has words, and each query carries more terms than the user typed. An inverted index’s cost scales with how many posting lists a query touches and how long they are, and learned sparse retrieval increases both. How much depends on the model’s expansion behaviour, which some models let you tune; the direction is not in doubt.
Vocabulary is fixed at training time. The model emits weights over the vocabulary it was trained with. A term outside that vocabulary is fragmented or dropped, which is the same problem dense models have with rare identifiers, arriving through the same door. Do not assume learned sparse retrieval solves identifier lookup — verify it on your own identifiers, because your SKU format was not in anybody’s training data.
Model choice is a lock-in. Switching models means reindexing the corpus. Identical in character to changing an embedding model, and equally underestimated.
Is it a replacement or a third list?
Both are defensible, and they answer different questions.
As a replacement for the lexical retriever in a hybrid pipeline: you keep two signals, one dense and one sparse, and the sparse one now handles paraphrase better than term matching did. The pipeline shape is unchanged, fusion works as before, and you have traded plain lexical’s speed and predictability for better recall on vocabulary mismatch.
As a third signal alongside both: fusion extends to any number of lists, so this is mechanically trivial. It is also the arrangement where you should be most suspicious of your own reasoning, because the marginal list is highly correlated with the lexical one. Consensus across three lists where two are near-duplicates of each other is not the consensus fusion’s design assumes, and it quietly reweights your pipeline toward the sparse side. If you add a third list, verify that fusion still behaves as intended rather than assuming more signals is more information.
As the only retriever: possible, and it gives up the thing dense retrieval does that no term-based representation does — matching text with genuinely no lexical relationship to the query, across languages or registers. Whether that class matters is a property of your queries.
Deciding
The honest summary is that learned sparse retrieval occupies a middle position and its value depends on which of the two ends is failing you:
| Lexical | Learned sparse | Dense | |
|---|---|---|---|
| Handles vocabulary mismatch | No | Partly | Yes |
| Exact term matching | Yes | If in vocabulary | No |
| Index-time model pass | No | Yes | Yes |
| Inspectable matches | Yes | Yes | No |
| Vocabulary lock-in | No | Yes | Yes |
| Infrastructure | Inverted index | Inverted index | Vector index |
Try it when your lexical list is the weak partner in fusion — when the queries fusion fails on are paraphrase queries where only the dense list contributed and its ranking was mediocre. In that situation a stronger sparse list gives fusion something to agree with, and agreement is what fusion converts into ranking.
Don’t reach for it to fix identifier queries. That is what an unmodified lexical index with a correct analyser already does, and it does it without a model, a reindex, or a vocabulary constraint.
Measure it the way you would measure any retrieval change: same queries, same known-correct passages, record where the passage lands in each candidate list. The magnitude will be specific to your corpus and your query mix, and published comparisons on public benchmarks tell you about those benchmarks.