We have built and open-sourced TurboQuant for .NET, a C# implementation of the vector quantization method described in the TurboQuant paper. The package is available on NuGet. It grew out of work with a client using Vespa and E5-768 embeddings, although we chose Vespa’s native bfloat16 support for that deployment.
Vector storage can become a significant memory cost in semantic search. The size depends on the number of documents, embedding dimensions and numerical representation.
A 768-dimensional float32 embedding occupies about 3 KB. One million such embeddings require roughly 3 GB for the vector values alone. At tens of millions of documents, reducing that footprint can be worth investigating.
The TurboQuant paper
In April 2025, researchers from Google Research, NYU and Google DeepMind published a paper introducing TurboQuant, later presented at ICLR 2026. TurboQuant uses randomized rotation and scalar quantization to reduce vector storage. Its analysis approaches an information-theoretic distortion bound within a constant factor without requiring calibration data.
Four bits per coordinate use one eighth of the raw storage of float32. That ratio excludes stored norms, headers, padding and shared quantizer state; it is not a guarantee of an eightfold reduction in application memory. Retrieval quality needs to be evaluated for the model and workload in use.
Implementations appeared in Python, Rust and C. We investigated what it would take to use the method directly in a .NET application.
Evaluating E5-768 embeddings in Vespa
The investigation began with a client using Vespa and E5-768 embeddings. We wanted to establish whether a C# implementation of TurboQuant would improve that deployment.
We chose Vespa’s native bfloat16 support for that case. Aggressive quantization caused measurable quality loss with the E5 embeddings, while bfloat16 halved the memory required for their values through a schema change, without additional application code.
We also considered workloads in which a .NET application manages vector storage directly.
Examples include in-process vector indexes, feature stores, semantic caches and batch deduplication over tens of millions of documents. In these architectures, the application may need to handle compression itself.
When Vespa or another vector store manages persistence and search, first assess its supported storage formats. An additional compression layer needs a measured benefit that justifies its integration and maintenance.
What we released
We built the library for .NET applications that need to manage compressed vectors themselves. It implements Algorithm 1 from the paper in C#, avoiding a separate Python integration for this part of a workload.
We released TurboQuant for .NET, an implementation of Algorithm 1 from the paper, on NuGet.
The feature list below describes version 0.3.0, released on 17 October 2026:
- Randomized rotation: random orthogonal rotation via QR decomposition by default; optional Hadamard rotation pads other dimensions to the next power of two, increasing the packed payload
- Lloyd-Max codebook computed using the unit-sphere coordinate density, proportional to (1 − x²)^((d − 3)/2) on [−1, 1]
- Bit-packing (2/3/4-bit), with reusable scratch buffers through ArrayPool and
Span<T> - Lookup-based approximate similarity for 2-, 3- and 4-bit vectors, computed directly from packed groups without intermediate index arrays; shared lookup tables are initialized lazily
- KV cache compression with asymmetric quantization (e.g. keys at 4-bit, values at 2-bit) and a residual window that keeps recent tokens in full precision
- Serialization built into PackedVector for storage and transport
- SIMD acceleration through
System.Numerics.Vector<T>, with hardware acceleration selected by the .NET runtime and a scalar fallback - .NET 8 and .NET 10 targets, MIT licensed, with no external runtime package dependencies
The repository includes tests comparing Dmse with theoretical distortion references, plus tests for cosine similarity, thread safety and end-to-end behaviour. These use synthetic vectors and dimension-specific thresholds. They provide implementation checks; retrieval quality and memory use still need to be measured on the application’s own data.
What we took from the client work
Our client’s deployment was the starting point, but it did not need this additional compression layer. Vespa’s native bfloat16 met the memory requirement through a schema change. We kept that solution for the client and released the .NET implementation separately.
The library is available for architectures in which the .NET application owns vector storage, including in-process indexes, feature stores and semantic caches. Those teams can evaluate the method without adding a Python service to their application.
The repository includes the implementation and its tests. Before adopting it, measure retrieval quality and memory use with your own embedding model and data, including any storage formats your search system already supports.