Our first reranker, and the training bug that made it worthless before it worked

Retrieval is only half of search. A retriever casts a wide net and pulls back a shortlist of candidate passages. Something still has to read that shortlist closely and decide which one actually answers the query. That second step is reranking, and today we are open-sourcing our first one.

It is called rerank-statute-en, it is public on Hugging Face, and the weights are Apache-2.0.

huggingface.co/quanfire-ai/rerank-statute-en

What it is

rerank-statute-en is a cross-encoder reranker for English central-statutory, bare-Act text. Where a retriever encodes the query and each passage separately and compares vectors, a cross-encoder reads the query and one candidate passage together and returns a single relevance score. You use it as a second stage: a bi-encoder retriever pulls the top-k candidates, and the reranker reorders them so the best answer moves to the top.

It is the companion to embed-statute-en, the statute retriever we released earlier. That model finds candidates; this one decides the order. It is a reranker, not a retriever, and it cannot stand in for one.

The number that matters

We measured the honest end-to-end quantity: retrieve first, then rerank, and see how often the single top result is correct. On 1,200 held-out queries against 1,182 unique statutory passages, scored on GPU:

That is a +0.131 gain, or +63.8%, and the paired 95% confidence interval is [+0.106, +0.157], which excludes zero. The improvement is not noise: on the same queries, adding the reranker moves the right passage into first place far more often than not.

The headroom we are not hiding

An absolute Recall@1 of 0.336 is modest, and we will say so plainly. The reason is a ceiling we do not control: the retriever only surfaces the correct passage anywhere in its top-100 about 71.8% of the time. A reranker can only reorder what it is handed, so it can never rank a passage the retriever never retrieved. That 0.718 is the wall this stage is working under. Getting from 0.205 to 0.336 against it is a real, significant first step, and there is genuine headroom left. This is a strong first reranker, not a claim that statute search is solved.

The bug worth telling you about

The first training run collapsed. The reranker scored candidates almost at random, no better than no reranker at all. The cause was in the negatives, not the model.

To train a reranker you show it correct query-passage pairs and a pile of wrong ones, and it learns to score the right pair higher. We first drew those wrong pairs with ordinary hard-negative mining, and it turned out the negatives were form-separable: they carried surface tells, things like OCR artifacts and repeated boilerplate, that let the model tell right from wrong without reading the query at all. It learned the shortcut instead of the task, and a shortcut does not transfer to real candidates at inference time.

The fix was to change the negatives, not the architecture. We switched to form-matched negatives: clean statutory bodies drawn from other records, so a wrong candidate looks exactly like a right one except that it does not answer the query. With the shortcut gone, the only way to score well is to actually compare meaning, and the model learned to do that. The lesson generalizes: a reranker is only as honest as its negatives, and those negatives have to match the candidate distribution it will see at inference. If your wrong answers are distinguishable by anything other than relevance, your reranker will learn that anything else.

Where the data came from

The reranker is trained on 858 Central Acts of the Indian Parliament, from a public dataset (Zenodo record 5088102, released under CC-BY-4.0). We attribute it plainly:

Source: "An annotated dataset of Central Acts enacted by the Indian Parliament," Zenodo record 5088102, CC-BY-4.0.

The model is non-reconstructive: it emits a relevance score and cannot reproduce the source text. We ship the weights, not the corpus. Reproduction of a bare Act is a permitted act under section 52(1)(q)(ii) of the Copyright Act, 1957. This is general information about how the model was built, not legal advice.

What it is for, and what it is not

We are precise about scope. rerank-statute-en is built and validated for English central-statutory (bare-Act) text, as a second-stage reranker over a retriever's shortlist. It is not a retriever and cannot be used as one. It is not validated for court judgments, for state legislation, rules, regulations or notifications, or for non-English text. A model that tells you where it works is worth more than one that claims to work everywhere.

Why we build in the open

This is another clean, verifiable model put out in the open, and the reason we told you about the collapsed run is the same reason we publish the confidence intervals: we would rather you trust the method than the marketing. The model card has the full benchmark, the ceiling, the intervals and the scope in one place. Pull it, put it behind your own retriever, and judge it on your own statutory text. Our training framework is open source at github.com/quanfire-ai/quanfire-multilingual-embedding (Apache-2.0). That is the discipline we bring to our products too.