Qdrant Builds a 10-Billion-Document Vector Search Benchmark Dataset on Vultr
Qdrant has released Qdrant-Fineweb-10B, a new public dataset designed to help developers test vector search at a scale closer to real production environments.
The dataset contains embeddings for 10 billion documents from FineWeb, together with exact results for 120,000 queries. Developers can use it to measure how accurately vector search systems retrieve results as datasets grow to billions of records.
Vultr provided the cloud infrastructure used to generate and store the embeddings behind the dataset, helping Qdrant process the full 10-billion-document corpus in approximately five days.
Why vector search needs better benchmark datasets
Vector search powers applications including retrieval-augmented generation (RAG), semantic search, recommendation systems, and AI agents. But testing vector search at a realistic scale is difficult.
Small or randomly generated datasets do not behave like real production data. As datasets grow, factors such as memory requirements, index build times, sharding, and retrieval accuracy can change significantly.
Qdrant-Fineweb-10B gives developers a public dataset designed to test these conditions at a much larger scale. It includes:
- 10 billion documents from FineWeb
- Dense and sparse vectors for every document
- 120,000 queries with exact ground-truth results
- Results to a depth of 1,000
This gives teams a practical way to measure retrieval accuracy and compare vector search configurations using the same large-scale dataset.
Processing 10 billion documents on Vultr
Creating the dataset required turning 10 billion documents into embeddings. Qdrant used Vultr cloud infrastructure to process the FineWeb corpus with Alibaba's gte-multilingual-base embedding model. Each document was converted into both dense and sparse vectors.
At this scale, GPU performance is only part of the challenge. Data needs to be prepared and delivered to the GPUs fast enough to keep them productive. The workflow used CPU workers to handle tokenization in parallel before sending the data to the GPUs. This helped keep the embedding pipeline moving efficiently and enabled processing rates of approximately 19,000 rows per second per node.
The full corpus was processed in approximately five days, producing around 500,000 files and 25 TB of data. Vultr Object Storage provided temporary storage for the resulting embeddings until Qdrant retrieved them for the next stage of the project.
The project shows the value of treating AI infrastructure as a complete pipeline. GPU compute, CPU processing, and storage must work together efficiently as AI workloads scale from millions to billions of records.
“Building Qdrant-Fineweb-10B meant processing data at a scale where infrastructure and execution really matter. Vultr gave us the compute and storage capacity we needed, along with hands-on support to help us move quickly from planning to processing 10 billion documents. The result is a dataset that gives developers a practical way to test vector retrieval against real-world data at a scale much closer to production.”
— André Zayarni, CEO and Co-Founder, Qdrant
Building useful infrastructure for the AI community
Qdrant-Fineweb-10B provides developers and researchers with a practical dataset for testing vector retrieval on 10 billion real-world documents, including dense, sparse, and filtered search with exact ground-truth results.
Building a dataset at this scale required a high-throughput pipeline for embedding generation, CPU-based preprocessing, and temporary storage of the resulting data. Vultr provided the compute and object storage infrastructure supporting this workflow. The result was approximately 25 TB of embedding data generated from 500,000 files in about five days, ready for anyone to use to build a benchmark.
“Qdrant-Fineweb-10B shows what it takes to work with AI data at real scale. Processing 10 billion documents requires GPUs, CPUs, storage, and a data pipeline to work together efficiently. We’re pleased to support Qdrant in making this dataset available to the wider AI community.”
— Kevin Cochrane, CMO, Vultr
