Blog
-
How many candidates to retrieve
Retrieval depth, fusion depth, rerank depth, final depth. Where each cut-off silently loses the right document, and how to find yours.
-
Filtering before or after you retrieve
A metadata predicate applied after the search returns an empty list; applied before, it changes what the search means. Neither is free.
-
Which signal failed
A query returned nothing useful. A procedure for finding whether lexical missed, dense missed, fusion buried it, or the document was never indexed.
-
When a reranker earns its latency
A cross-encoder reads the query and passage together, which is why it ranks better and costs more. Where it belongs, what to feed it, and when to skip it.
-
Fusing two ranked lists
BM25 scores and cosine similarities aren't on the same scale. Reciprocal rank fusion, score normalisation, and what each method throws away.
-
What keyword search misses
Vocabulary mismatch: when the user's words and the document's words don't overlap, term scoring has nothing to score. Where stemming stops helping.
-
What vector search misses
Identifiers, rare terms, negation and exact phrases. Five query classes where embeddings return plausible neighbours instead of the document you asked for.
-
Rewriting the query before you search
The string a user types is rarely the best search key. What to strip, what to resolve, what to pull out as a filter, and what to leave alone.
-
Query expansion, and how it drifts
Adding terms to a query fixes vocabulary mismatch and dilutes precision at the same time. Where synonyms belong, and how expansion goes wrong.
-
Searching with a hypothetical answer
Generate a fake answer, embed that instead of the question, and search with it. Why the trick works, and the three ways it misfires.
-
One question, three searches
A compound question retrieves the average of its parts. How to split it, how to merge the result lists, and when splitting makes things worse.
-
Learned sparse retrieval, the third signal
A model that predicts term weights and adds terms the document never contained. What it inherits from lexical search, and what it costs.
-
Weighting one signal over the other
A fusion weight is a claim about your query mix, not your corpus. How to derive one, why a global weight is wrong, and when to skip it.
-
Why a similarity threshold doesn't travel
The cut-off that worked on your test queries fails on the next batch. What a cosine score is comparable to, and what to threshold instead.
-
Typos, and which retriever forgives them
A misspelling deletes a lexical match and blurs a dense one. Fuzzy matching, correction, and why the fix depends on which word was misspelled.
-
When five results say the same thing
Retrieval returned five passages and one fact. Diversity at query time: per-document caps, marginal relevance, and what each one costs you.
-
Which field the match landed in
A title match and a body match are not worth the same. Field weighting, why concatenating fields loses information, and how boosts go wrong.
-
The analyser decides what you can match
Tokenisation, stemming and stop words run before scoring. Get them wrong and lexical retrieval cannot find terms that are plainly in the text.
-
A reranker cannot find what retrieval missed
Reranking reorders a candidate set and never adds to it. Why a good reranker on a shallow pipeline is money spent on the wrong stage.
-
Using an LLM as a reranker
Asking a general model to order candidates needs no training data and brings non-determinism, position bias and per-query cost. When that trade works.
-
When the query and the document aren't in the same language
Term matching across languages scores zero. Multilingual embeddings, translating the query, and why a mixed-language corpus fails per language.
-
Routing a query before you retrieve it
Classify the query, send it to the retriever equipped for it. Cheaper than fusion, and it fails completely when the classifier is wrong.
-
Making a term mandatory
Term scoring is additive, so a document missing the decisive word can still win. How required terms work, and what they do to a hybrid pipeline.
-
Boosting recency without breaking relevance
Superseded documents outrank current ones because relevance ignores dates. Where a time signal belongs in a ranking, and how it goes too far.
-
When every document looks the same to the vector index
On a homogeneous corpus, dense scores compress and ranking becomes arbitrary. How to spot a low-contrast corpus and what still discriminates.