Passages, reranked.

MS MARCO v1 contains 8,841,823 passages from millions of web pages. TREC DL 2019 supplied 1,000 candidates for 41 queries, plus lists of 37 and 5. This view replays JEV’s original-text scores for all 41,042 supplied pairs and compares its ranking with monoBERT on the same candidates.

Sample of 20 search queries

monoBERT order

Ranking quality · supplied candidates

MetricmonoBERTJEV

0.0000

—

0.0000

—

0.0000

—

0.0000

—

0.0000

—

0.0000

Fixed pool

Percentage change is relative to monoBERT. nDCG@10 uses grades 0–3; AP@1000 and RR@10 count grades 2–3 as relevant.

Results

Up to 10 shown · all supplied candidates reranked

NIST grades

01
PASSAGE 8588219 3 · Exact answer
User: Hydrogen is a liquid below what temperature?
—
02
PASSAGE 85882220 · Irrelevant
Answer to: Hydrogen is a liquid below what temperature?
—
03
PASSAGE 2697752 2 · Limited answer
Hydrogen's state of matter is gas at standard conditions of temperature and pressure.
—
04
PASSAGE 3905802 3 · Exact answer
Hydrogen is found naturally in the molecular H2 form.
—
05
PASSAGE 85882270 · Irrelevant
A further temperature decrease below the boiling point eventually results in mixtures of liquid and solid hyd…
—
06
PASSAGE 128989 3 · Exact answer
At STP (standard temperature and pressure) hydrogen is a gas.
—
07
PASSAGE 6080460 3 · Exact answer
Hydrogen can exist as a liquid under high pressure and an extremely low temperature of 20.28 kelvin (−252.87°…
—
08
PASSAGE 2697746 2 · Limited answer
What is neon's state of matter?
—
09
PASSAGE 128984 3 · Exact answer
Hydrogen gas has the molecular formula H 2.
—
10
PASSAGE 1770027 2 · Limited answer
1 APPEARANCE: Hydrogen fluoride is a colorless fuming liquid below 67°F (19.4°C), or a colorless gas.
—

All supplied candidates were scored.Saved experiment replay

Across 43 queries

Completed experiment on all 41,042 supplied pairs.

nDCG@100.7177 0.6835Graded relevance

MAP@10000.4488 0.4729Grades 2–3 relevant

Estimated API cost$0.761Successful responses

41,042 scored pairs · 1575.1 s reranking · retrieval time unknownExperiment report