Hybrid Search and Reranking: Finding Better Answers
Learn how combining keyword and meaning search, then reranking, fixes many retrieval failures.
Retrieval is where most RAG systems win or lose, and two upgrades fix a surprising share of its failures.
Hybrid search runs keyword search and meaning search side by side and merges the results. Remember their blind spots: meaning search shrugs at exact codes like E-207, keyword search misses “clicking noise” when the article says “ticking sound”. Run both and each catches what the other misses.
Merging is a little tricky, because the two methods produce scores on completely different scales. A popular trick ignores the scores and uses only the positions: reciprocal rank fusion gives each document points for ranking high in either list, more for first place than for fifth, and adds them up. A document that both methods rank fairly high usually beats one that only a single method loves.
Reranking then takes a closer look at the shortlist. Embedding search is fast because the question and each document are turned into vectors separately; it never actually reads them together. A reranker does: it’s a model that takes the question and one candidate as a pair, and scores how well that candidate answers that question. It notices things vectors blur over, such as a battery that died after three weeks being a warranty question, not a refund question, even if the refund page sounds similar.
Reading pairs is much slower than comparing vectors, so rerankers only see the top 20 to 100 results, never the whole collection. Fast and broad first, slow and careful second: the same pattern you’ll find in web search, recommendations and hiring.
Key takeaways
- Hybrid search runs keyword and meaning search together and merges the results.
- Reciprocal rank fusion merges by rank position, so differently scaled scores don’t matter.
- A reranker reads the question and each candidate together, so it judges relevance more precisely.
- Rerankers are slow, so they only reorder a short list: fast and broad first, careful second.
Check your understanding
Go deeper (more technical)
Reciprocal rank fusion (Cormack and colleagues, 2009) scores each document as Σ 1 / (k + rank) over the lists, with k usually 60. Rerankers are typically cross-encoders: a transformer that reads both texts in one input. Embedding models, by contrast, are bi-encoders, encoding each text separately so documents can be embedded ahead of time.
Other retrieval boosts include query rewriting, generating a hypothetical answer and searching with its embedding (HyDE), and letting an agent run several searches in a row (Track 7).
The simulation’s rankings are hand-made.