Reranking
Reorder recalled candidates by true relevance
WHAT THIS CAPABILITY MEANS
Takes a query and a short list of recalled candidates, and returns a reordered ranking with relevance scores. Its division of labour differs from embedding retrieval: retrieval uses cheap vector neighbours to pull dozens from tens of thousands, whereas reranking applies a costlier model to score those dozens carefully, so it only runs over the candidate set.
How it is done
The typical approach is a cross-encoder: query and candidate are concatenated into one sequence and a single model outputs a relevance score; seeing both at once is more accurate than two-tower encoders, but it cannot be pre-computed and must run per pair. To control latency, a common pattern truncates with a small model first and reranks with a larger one — or simply prompts a large model to score.
Representative products
4Pinecone
2019A managed vector database for similarity search
Together API
2022An inference API for open models
Replicate
2019Run community-uploaded models through an API
Hugging Face Hub
2016The open clearing house for models and datasets
Organizations involved
Typical uses
- Improving evidence quality after RAG retrieval
- Final ranking in e-commerce and content search
- Selecting the most relevant passages for a QA system
- Deduplicating and choosing among candidate answers
How it is evaluated
- nDCG@k
- Discounted gain in the top-k after reranking, rewarding better placement
- MAP
- Mean of average precision over all relevant items
- MRR
- Quality of the position of the first relevant result
Limits and hard parts
- Cross-encoders score each candidate separately, so latency climbs steeply with candidate count
- Judgements are unstable on out-of-distribution queries, and truncating long documents causes errors
- It can only reorder what was recalled; anything retrieval missed cannot be recovered
Concepts behind it
Retrieval-Augmented Generation
Rather than cramming knowledge into parameters, leave it outside and look it up on demand — an open-book exam instead of a closed-book one
Attention Mechanism
Every position can look directly at every other position and dynamically weight how much attention to pay
Model Evaluation & Cross-Validation
Accuracy is the easiest metric to fool you — get evaluation wrong and everything else follows