Boosting recency without breaking relevance

An engineer asks an internal wiki “how do we provision a staging environment”. The top result is a detailed, well-written procedure from four years ago describing infrastructure that no longer exists. The current procedure is at rank six, shorter and less thorough, and correct.

Both retrievers preferred the old document, and both were right by their own objective. Neither has any notion that one document supersedes the other, because a date is metadata and relevance is computed from text.

Query: "how do we provision a staging environment"

LEXICAL (BM25)                            DENSE (vector)
1. Staging provisioning (2022, old)  ✗   1. Staging provisioning (2022, old)  ✗
2. Environment naming conventions     ~   2. Provisioning environments, guide  ~
3. Provisioning environments, guide   ~   3. Staging setup, current            ✓
4. Staging teardown runbook           ~   4. Environment naming conventions    ~
5. Staging setup, current             ✓   5. Staging teardown runbook          ~

The old document ranks first on both sides for the same reason: it is longer, more thorough, and repeats the query’s vocabulary more. Thoroughness and staleness are correlated, because documents accrete detail over the years before being abandoned.

Recency is a different kind of signal

Relevance answers “does this text address the query”. Recency answers “is this text still true”. They are independent, and combining them is a ranking design decision rather than a retrieval one.

Which produces an immediate scale problem, and it is the same problem as fusing two retrievers’ scores. A relevance score and an age are not commensurable. Any combination requires mapping age onto a comparable scale first, and the choice of mapping is the policy — there is no neutral way to do it.

Three mappings, in increasing sophistication:

A multiplicative decay. Multiply the relevance score by a function of age that falls off smoothly. One parameter, the half-life, and it expresses “older is worth proportionally less” without a cliff.

A tiered boost. Documents within a recent window get a fixed multiplier, older ones don’t. Interpretable, explainable to stakeholders, and it creates a discontinuity at the boundary that will produce puzzling rank inversions for documents either side of it.

A hard filter. Exclude anything older than a cut-off. Not a boost at all, and it belongs to the filtering discussion, with the usual consequence: content that is old and still correct becomes unreachable.

Which date, and this is the part that goes wrong

Before tuning anything, establish what your date field means. Corpora typically carry several and they disagree:

Created. When first written. Says nothing about current accuracy.
Last modified. The one people reach for, and it is the most corruptible — a bulk migration, a reformat, a permissions change, or a link fix can touch every document and set its modification date to the same day. A recency boost on a field like that is a boost on when your infrastructure last ran a batch job.
Published or effective. Usually the one that means what you want, and usually the one that is missing on part of the corpus.
Reviewed. The most honest signal for a documentation corpus, and only present where someone built a review process.

A recency boost is only as meaningful as the date field it reads, and most date fields mean something other than what the boost assumes. Check the distribution before deploying: if a large share of your corpus shares one modification date, that field is a record of an operation, not of content freshness.

And note the null problem again — documents with a missing date need a defined treatment. Treating null as very old buries content that may be current; treating it as current advantages documents with incomplete metadata. Pick deliberately and write it down.

When recency is the wrong signal entirely

Recency is a proxy for correctness, and the proxy holds only in some corpora.

It holds for operational documentation, product specifications, prices, policies, release notes, regulations — anything that is revised and superseded.

It fails for reference material, historical records, research literature, legal precedent, and archives. In a corpus of case law or academic papers, age is nearly uninformative about value and a recency boost is an active distortion.

It inverts where the query is explicitly historical. “What was our old provisioning process” wants the stale document, and a global boost fights the user. That is a per-query decision, which means recency weighting is a candidate for query-dependent routing rather than a global setting — a query mentioning a past year or containing “previous”, “old”, “legacy”, “used to” should have its boost disabled.

A better signal than recency exists in some corpora and is worth preferring where it does: supersession. If documents record what replaced them, or carry a status field, then “current” is a fact rather than an inference from a date. A status predicate beats any decay function, because it encodes the thing you actually wanted to know.

How boosting goes too far

The characteristic failure is a search that returns the newest documents rather than the relevant ones. It arrives gradually and is easy to miss, because recent documents are often genuinely more relevant and the ranking looks defensible until someone searches for something whose best answer is old.

Three specific traps:

A half-life shorter than your corpus’s rate of change. If documents are revised yearly and your decay halves relevance in a month, you have built a chronological feed with a search box.

Boost applied before fusion, asymmetrically. Applying decay to one retriever’s scores and not the other’s distorts the merge in a way that is very hard to reason about. Apply the time signal after fusion, to the merged list, or apply it identically to both — and prefer the former, because rank-based fusion has already discarded score magnitudes and reintroducing a multiplicative factor on ranks means something different from what you intend.

Boost plus reranking, uncoordinated. A cross-encoder reorders purely on textual relevance and will undo your recency ordering completely. If both stages exist, the time signal has to be applied after reranking, or the reranker has to see the date — and giving a reranker a date to read is a way of asking a language model to weigh freshness against relevance with no stated policy, which is worse than an explicit decay function.

So: fuse, rerank, then apply the time signal, last. That ordering is the one that behaves predictably, and it means the time signal only ever adjusts a list that is already relevant.

Telling whether it’s your problem

Take a sample of queries and check the age of the top result against the age of the best result. If superseded documents are systematically winning, you have the problem. If the two are usually the same document, you don’t, and adding a boost can only cost you.

Then measure the opposite direction before you ship, because it is the failure your fix introduces: take queries whose correct answer is a known-old document — reference material, a historical policy — and verify that they still work with the boost enabled. If they don’t, your decay is too aggressive, and the fix is a longer half-life rather than a special case.

The right half-life is a property of how fast your corpus goes stale, which nobody outside your organisation can tell you. The only defensible way to set it is to look at how often your documents are actually revised and pick something on that timescale, then check both directions again.