Embedding Seven Billion Records on Existing Hardware: A Hybrid AI Go/No-Go
How a global library cooperative validated that its existing on-premises GPU fleet could embed seven billion library records at scale — clearing the path for a hybrid architecture that keeps embedding on the ground and serves applications from the cloud.
$ cat case-study/details.yaml
The Problem.
The client operates a global library cooperative whose core asset is a knowledge base of seven billion library records. To bring modern semantic search and retrieval to that corpus, every record needed to be embedded — converted into a vector representation a downstream application could query for similarity, retrieval, and ranking.
The cost calculus drove the architecture. Doing all of this in the cloud — embedding seven billion records and serving them at query time — was prohibitively expensive at this scale. The opposite extreme, running everything on-premises, would mean degraded performance for the applications that ultimately consume the embeddings. The client wanted a hybrid: embed on the ground using existing hardware, then serve the embeddings to applications from the cloud where latency and elasticity matter most.
The client already owned NVIDIA GPUs from a previous AI inference investment — valuable hardware, but a generation that had no publicly available embedding benchmarks. Two questions had to be answered before the hybrid plan could move forward, and they had to be answered with enough confidence to defend a go/no-go decision:
1. Could the existing GPU fleet realistically embed seven billion records in an acceptable window — or did the client need to budget for a hardware refresh?
2. Among the candidate embedding models, which one struck the best balance of throughput, accuracy, and resource fit on this specific hardware?
Without grounded answers, the hybrid architecture stayed a theory, and the alternative — bulk-buying new GPUs or paying cloud rates to embed at this scale — was an expensive default.
What we built.
Quest1 partnered with deepset and NVIDIA to design and run a benchmarking exercise tailored to the client's exact hardware, workload shape, and decision criteria. The work was structured to produce defensible numbers, not directional impressions.
The engagement was structured as consulting and co-delivered with deepset, with NVIDIA's benchmarking methodology as the technical anchor. Quest1 led the model evaluation and hardware validation work; deepset contributed Haystack Enterprise Platform expertise and integration patterns for the resulting hybrid pipeline.
The Impact.
The benchmarking gave the client what they came for: quantified, defensible evidence that the existing on-premises GPU fleet could handle the seven-billion-record embedding workload, and a specific recommended embedding model matched to that hardware. The go/no-go decision had grounded numbers behind it.
The cost outcome followed directly. Avoiding a major GPU refresh — and avoiding the cloud bill for embedding at this scale — kept the budget within the range the hybrid architecture was designed to deliver. The client retained their existing infrastructure investment and validated a path forward that combined on-prem cost efficiency with cloud-tier application performance.
- Confidence that the existing on-premises GPU fleet is fit for purpose at seven-billion-record scale, with quantified headroom.
- A recommended embedding model, selected from a head-to-head comparison on the client's actual hardware rather than from public benchmarks that didn't apply.
- A hybrid architecture pattern — on-prem embedding, cloud serving — that the client can extend to other workloads.
- A repeatable benchmarking methodology that Quest1 can apply to future on-prem AI feasibility decisions for other clients with similar constraints.