---
title: "Hybrid search for RAG: BM25 plus embeddings"
description: "Keyword search misses meaning; vector search misses exact terms. Build BM25 from scratch, add embeddings, and merge both rankings with reciprocal rank fusion."
url: https://articles.sythra.ai/articles/hybrid-search-bm25-embeddings-rag
slug: hybrid-search-bm25-embeddings-rag
author: "vaibhavkothari"
author_url: https://articles.sythra.ai/writers/vaibhavkothari
date_published: 2026-11-12T21:11:00.153Z
date_modified: 2026-11-12T21:11:00.153Z
topics: ["Ai", "Rag", "Python", "Search"]
reading_time_minutes: 7
publisher: "Sythra"
publisher_url: https://sythra.ai
access: free
language: en
---

# Hybrid search for RAG: BM25 plus embeddings

> Keyword search misses meaning; vector search misses exact terms. Build BM25 from scratch, add embeddings, and merge both rankings with reciprocal rank fusion.

Source: https://articles.sythra.ai/articles/hybrid-search-bm25-embeddings-rag · Author: vaibhavkothari · Published: 2026-11-12 · Reading time: 7 min · Topics: Ai, Rag, Python, Search

Most RAG systems start with vector search and work beautifully on the demo questions. Then a user pastes an error code, a product SKU or a person's surname, and the right document is nowhere in the top results. Embeddings capture *meaning*; they are poor at *exact strings*. Classic keyword search has the opposite strengths. **Hybrid search** runs both and merges the results. It is one of the cheapest retrieval upgrades available.

## Two retrievers, two blind spots

| Query | Keyword search (BM25) | Vector search (embeddings) |
|---|---|---|
| `E1042` (exact error code) | Finds it instantly | Often misses; the code means nothing semantically |
| "how do I get my money back" | Misses a document that says "refund" | Finds it; same meaning |
| `iPhone 15 Pro Max case` | Matches model names precisely | May return other phones |
| Rare names and acronyms | Strong | Weak |
| Paraphrases and typos | Weak | Strong |

Neither wins everywhere. Real queries mix both kinds, so use both.

## Part 1 — BM25 from scratch

BM25 is the standard keyword scorer behind most search engines. The idea in three parts:

- **Term frequency:** a document mentioning the query word more is more relevant, with diminishing returns
- **Inverse document frequency (IDF):** rare words matter more than common ones
- **Length normalisation:** a long document should not win just for containing more words

```python
import math
import re
from collections import Counter


def tokenize(text):
    return re.findall(r"[a-z0-9]+", text.lower())


class BM25:
    def __init__(self, docs, k1=1.5, b=0.75):
        self.k1, self.b = k1, b
        self.tokens = [tokenize(d) for d in docs]
        self.lengths = [len(t) for t in self.tokens]
        self.avg_len = sum(self.lengths) / len(self.lengths)
        self.tf = [Counter(t) for t in self.tokens]

        df = Counter()
        for t in self.tokens:
            df.update(set(t))                      # count documents, not occurrences
        n = len(docs)
        self.idf = {
            w: math.log((n - c + 0.5) / (c + 0.5) + 1) for w, c in df.items()
        }

    def scores(self, query):
        result = []
        for i, tf in enumerate(self.tf):
            score = 0.0
            for word in tokenize(query):
                if word not in tf:
                    continue
                f = tf[word]
                norm = f + self.k1 * (1 - self.b + self.b * self.lengths[i] / self.avg_len)
                score += self.idf[word] * f * (self.k1 + 1) / norm
            result.append(score)
        return result
```

`k1` controls how quickly repeated words stop helping; `b` controls how strongly length is penalised. The defaults (1.5 and 0.75) are sensible starting points.

Try it on a tiny corpus:

```python
docs = [
    "Error code E1042 means the payment gateway rejected the card.",
    "How to refund a customer when a payment fails.",
    "Our returns policy lets you send items back within 30 days.",
    "Troubleshooting login problems and password resets.",
]

bm25 = BM25(docs)
print([round(s, 2) for s in bm25.scores("E1042")])
# [1.15, 0.0, 0.0, 0.0]   exact code found at once

print(bm25.scores("reimbursement"))
# [0.0, 0.0, 0.0, 0.0]    no document contains the word
```

The exact code lands precisely on the right document. But a query using a word the corpus never uses — "reimbursement" when the docs say "refund" — scores zero everywhere. That is the vocabulary mismatch problem, and it is what embeddings solve.

(For production, use a maintained library such as `rank_bm25`, or the built-in full-text features of your search engine or database. Writing it yourself once is for understanding.)

## Part 2 — Vector search

[Embeddings](https://articles.sythra.ai/articles/embeddings-explained-mathematically) map text to vectors so that similar meanings sit close together. Retrieval is then a nearest-neighbour lookup (see [how vector databases work](https://articles.sythra.ai/articles/vector-databases-under-the-hood)).

```python
import numpy as np
from sentence_transformers import SentenceTransformer

model = SentenceTransformer("all-MiniLM-L6-v2")

doc_vectors = model.encode(docs, normalize_embeddings=True)


def dense_scores(query):
    q = model.encode([query], normalize_embeddings=True)[0]
    return doc_vectors @ q          # cosine similarity, since vectors are normalised
```

Any embedding provider works the same way: embed the documents once, embed the query at search time, take dot products. Now "reimbursement" or "get my money back" should rank the refund document first, even with zero shared words.

## Part 3 — Merge the rankings

The two scorers speak different languages: BM25 scores are unbounded (1.15, 12.7, ...) and cosine similarities sit between -1 and 1. You cannot just add them. Normalising scores works but is brittle. The robust trick is reciprocal rank fusion (RRF): ignore scores, use only **ranks**.

```
RRF score(doc) = sum over rankers of  1 / (k + rank)
```

with `k = 60` as the conventional constant (it dampens the advantage of being first).

```python
def ranks_from_scores(scores):
    order = sorted(range(len(scores)), key=lambda i: scores[i], reverse=True)
    return order           # list of doc ids, best first


def rrf(rankings, k=60):
    fused = {}
    for ranking in rankings:
        for rank, doc_id in enumerate(ranking, start=1):
            fused[doc_id] = fused.get(doc_id, 0.0) + 1.0 / (k + rank)
    return sorted(fused, key=fused.get, reverse=True)


def hybrid_search(query, top_k=3):
    bm = ranks_from_scores(bm25.scores(query))
    dense = ranks_from_scores(dense_scores(query))
    return rrf([bm, dense])[:top_k]
```

### Worked example

Suppose a document is ranked **1st** by BM25 and **3rd** by vector search:

```
1/(60+1) + 1/(60+3) = 0.01639 + 0.01587 = 0.03226
```

A document ranked 2nd by both gets `2/62 = 0.03226` — practically tied. A document ranked 1st by only one ranker and absent from the other's top results gets about `0.0164`. So RRF rewards **agreement**: documents both retrievers like rise to the top, while a document only one retriever loves still has a chance if it is ranked very high.

One practical detail: rank only the top 20–50 candidates from each retriever rather than the full list, and let documents missing from one list contribute nothing from that list.

## Part 4 — Rerank the merged list

Hybrid search improves **recall**: the right chunk is more likely to be somewhere in your candidate set. A **reranker** then improves **precision** by reading the query and each candidate together and re-ordering them. A typical pipeline:

```
query ──► BM25 top 30 ┐
                      ├─► RRF merge ─► top 20 ─► reranker ─► top 4 ─► LLM
query ──► vectors top 30 ┘
```

This is the structure described in [chunking, retrieval evaluation and reranking](https://articles.sythra.ai/articles/rag-chunking-retrieval-evaluation-reranking). Add stages only when measurement shows they help.

## Measure it, don't assume it

Hybrid usually wins, but "usually" is not a result. Build a small set of real questions, each with the document that should answer it, and compare:

```python
def recall_at_k(search_fn, cases, k=5):
    hits = 0
    for question, correct_doc in cases:
        if correct_doc in search_fn(question, top_k=k):
            hits += 1
    return hits / len(cases)

# compare: bm25 only, dense only, hybrid
```

Include exact-match queries (codes, names) and paraphrase queries in the set, since those are where the two methods diverge. If hybrid does not beat your best single method on *your* data, do not pay the complexity.

## Many databases do this for you

You rarely need to hand-roll the merge in production. Search engines and some vector databases (Elasticsearch/OpenSearch, Postgres with full-text plus pgvector, and others) offer hybrid queries with built-in fusion. The concepts above tell you what to configure and what to measure.

## Tuning tips

- **Tokenisation matters for BM25.** Keep identifiers like `E1042` and `v2.3` intact, and consider stemming for prose.
- **Chunk size affects both** retrievers. Keep chunks focused; see the chunking guide.
- **Weighted fusion** is possible (multiply one ranker's contribution by a weight) if your queries skew heavily exact-match or heavily semantic.
- **Embed the same text you index** for BM25, or retrieval results will disagree for confusing reasons.

## FAQ

### What is hybrid search?

Hybrid search combines keyword search (like BM25) with vector similarity search and merges their results, so both exact terms and semantic meaning influence the final ranking.

### What is BM25 used for?

BM25 scores how well documents match a keyword query, based on term frequency, word rarity and document length. It powers keyword search in many search engines.

### What is reciprocal rank fusion?

RRF merges several ranked lists by giving each document `1/(k + rank)` from every list and summing. It ignores raw scores, so it works even when rankers use incompatible scales.

### Is hybrid search better than vector search alone?

Often, especially for queries with exact identifiers, names or rare terms. Test both on your own questions to confirm before adding the complexity.

## Next reading on Sythra Articles

- [RAG chunking, retrieval evaluation and reranking](https://articles.sythra.ai/articles/rag-chunking-retrieval-evaluation-reranking)
- [Vector databases under the hood](https://articles.sythra.ai/articles/vector-databases-under-the-hood)
- [Build a chatbot that answers from your own PDFs](https://articles.sythra.ai/articles/rag-chatbot-pdf-python-claude)

## Glossary (terms defined in this article)
- **BM25** (advanced) — A ranking function that scores documents by how often query words appear, how rare those words are, and how long the document is
- **reciprocal rank fusion** (advanced) — Combining rankings by summing 1/(k + rank) for each document across the lists; ignores raw scores entirely