As a reviewer of the TurboQuant paper, I gave a high rating to support this paper due to its theoretical analysis and experimental results. However, I also explicitly pointed out that both RaBitQ and TurboQuant both use random projection and requested the authors of TurboQuant to compare how the design differences between TurboQuant and RaBitQ affect performance. I believe the right academic practice is to thoroughly discuss the differences between RaBitQ and TurboQuant in the paper. However, checking the camera-ready version of TurboQuant, I am surprised to find that RaBitQ is mentioned only once in the experiment part of the main paper.

Regards,

Reviewer WFrV

This is a serious issue that deserves more attention. It's disheartening to see the people doing the actual foundational work get overlooked while large, influential organizations hype up their own results. And when people voice honest concerns, they get downvoted for going against the wave. At this point it feels less like science and more like a PR competition.

In response to recent commentary regarding our paper, "TurboQuant," we provide the following technical clarifications to correct the record.

1. Core Novelty vs. Standard Techniques

TurboQuant did not derive its core method from RaBitQ. Random rotation is a standard, ubiquitous technique in quantization literature, pre-dating the online appearance of RaBitQ, e.g. in established works like https://arxiv.org/pdf/2307.13304, https://arxiv.org/pdf/2404.00456, or https://arxiv.org/pdf/2306.11987. The true novelty of TurboQuant lies in our derivation of the exact distribution followed by the coordinates of rotated vectors, which we use to achieve optimal coordinate-wise quantization.

2. Correction on RaBitQ Optimality

While the optimality of RaBitQ can be deduced from its internal proofs, the paper’s main theorem implies that the distortion error bound scales as $4^{\mathrm{\Theta }(-b)}$. Because a hidden constant factor within the exponent could scale the error exponentially, this formal statement did not explicitly guarantee the optimal bound. This led to our honest initial characterization of the method as suboptimal. However, after a careful investigation of their appendix, we found that a strict $4^{-b}$ bound can indeed be drawn. Having now verified that this optimality is supported by their deeper proofs, we are updating the TurboQuant manuscript to credit their bounds accurately.

3. Materiality of Experimental Benchmarks

Runtime benchmarks are immaterial to our findings. TurboQuant’s primary contribution is focused on compression-quality tradeoff, not a specific speedup. The merit of our work rests on maintaining high model accuracy at extreme compression levels; even if the runtime comparison with RaBitQ was omitted entirely, the scientific impact and validity of the paper would remain mostly unchanged.

4. Observations on Timing

TurboQuant has been publicly available on arXiv since April 2025, and one of its authors was in communication with RaBitQ authors even prior to that, as RaBitQ authors have acknowledged. Despite having nearly a year to raise these technical points through academic channels, these concerns were only raised after TurboQuant received widespread attention.

We are updating our arXiv version with our suggested changes implemented.

We thank the reviewer for their constructive and insightful feedback.

We'd like to start by elaborating on a feature of TurboQuant. One important feature is TurboQuant's "online" nature which stems from its use of a data-independent or data oblivious codebook that obviates the need for continuous updates as new data arrives. We achieve this by first normalizing the vectors by their l2 norm and then applying a random rotation to ensure the entries of the vectors will have a beta distribution post rotation. Then we show that there exists a universally optimal codebook for quantizing each coordinate with this beta distribution. Thus, there is no need to tune or update the codebook.

comparison with more recent methods: First we'd like to mention that TurboQuant is a data oblivious method which is a very important property for distributed computing in applications such as distributed nearest neighbor search. The main purpose of our experiments was to show that being data independent does not really lead to worse quantization error and TurboQuant is even able to outperform data dependent product quantization methods and achieve higher recall for nearest neighbor search, especially against the FAISS baseline. We also wish to clarify that residual quantization, while relevant to partitioning centroids in ANN indices, is not fundamentally a quantization technique. Our method is fully compatible with residual vectors. In our experiments, we focused exclusively on the quantization problem, employing exact nearest-neighbor searches with quantized data points rather than an ANN index, thereby isolating the effects of quantization. Our quantization technique is readily applicable to residualized vectors within any index, including FAISS or SCANN.

Furthermore, we ran new experiments to benchmark our performance against more recent quantization papers, including NestQuant, LSQ++, and OPQ. For your initial review, we present the recall metrics on the GloVe ($d=200$) dataset with bit-width=4, and we commit to incorporating the results for the OpenAI datasets as well in the final version of the paper.

We thank you for your positive and constructive feedback and appreciate you bringing this new paper (SAQ) to our attention.

Please note that while certain techniques in the referenced work, such as PCA, undeniably improve quality, they introduce data dependence. Our primary emphasis with TurboQuant remains data obliviousness, a property crucial for applications that need real time quantization like KV cache as well as seamless integration into distributed environments.

Furthermore, these techniques are readily combinable with TurboQuant. For instance, PCA and dimension segmentation can be used to find subspaces of varying importance (determined by singular values), allowing for projecting the vectors into these subspaces and then applying TurboQuant with different bitwidths—assigning higher bitwidths to subspaces with larger singular values and lower ones to those with smaller singular values.

Unfortunately, the code and implementation for this novel method are not yet publicly available. We plan to initiate contact with the authors of this paper soon and, upon receiving their code, we commit to running the requested experiments and incorporating them into the final version of our paper.

In my previous comment D1, I am not asking the authors to compare TurboQuant with SAQ, I am aware this is unfair because SAQ uses data distribution. I meant to compare the different design choices made by TurboQuant and RabitQ, i.e., (1) RabitQ searches for quantized bits while TurboQuant uses k-means generated codebook independently for each dimension; (2) RabitQ uses a re-normalization trick for unbiased inner product estimations while TurboQuant combines random rotation with another vector quantization method. The question is that whether the design choices of RabitQ or TurboQuant are better after applying random projection. I still think making these questions clear with benefit the field, and the code for RabitQ is easily accessible. Although this requires extra work, I strongly encourage the authors to add these experiments in the final version of the paper.

We'd like to first thank the reviewer for their feedback. We address your questions/concerns below:

imposing restrictions on the distribution of vector x: We wanted to clarify that in the proof of our distortion bounds we do not need independence of coordinates of x neither before nor after the random rotation and the statement in line 215 was just for providing an intuition about why the result is expected but the results proven in our Theorem 1, 2, 3 do not rely on independence of coordinates. We apologize if the statement is confusing and we will modify it to remove this confusion in the final version of the paper. In summary, our theorems prove that our quantizer achieves near optimal distortion bounds and there is no assumption made about the input vectors in order to prove these bounds.

multiplicative bias: We agree the bias is not important in applications to ranking points by their scores. However, in other use-cases such as KV cache compression and model quantizations, this bias becomes critical as it can exponentially alter attention scores. We believe that the observation of optimizing for MSE loss introducing a dot-product bias is both intriguing and informative for readers, as is the design of residual vector quantization schemes. However we stay open to the reviewers suggestions.

reason for multiplication with a random rotation matrix: Random rotation allows us to obtain optimal distortion rate for vectors while making no assumption about the vector distribution. Even if the vector has a worst case distribution with large outlier coordinates, after random rotation each coordinate will have a beta distribution which we know how to quantize optimally. The use of random rotation has been explored before (see references like Quarot, RabitQ, QJL, etc).

experiments are insufficient: We have run a new, comprehensive set of experiments for both KV cache quantization and Approximate Nearest Neighbor (ANN) search, and we are committed to integrating these results into the final version of the paper. First we present the new KV cache quantization performance on the GSM8K mathematical reasoning dataset, using the Qwen2-7B-Instruct model (both model and dataset are new additions not present in the initial submission). This analysis also incorporates a comparison against a new baseline method, KV-Rotate, with results summarized in the table below:

Next, we present new ANN experiments to benchmark our performance against more recent quantization papers, including NestQuant, LSQ++, and OPQ. For your initial review, we present the recall metrics on the GloVe ($d=200$) dataset with bit-width=4, and we commit to incorporating the results for the OpenAI datasets as well in the final version of the paper.

online property not explicitly demonstrated: TurboQuant qualifies as an online quantization algorithm by virtue of its data-independent nature, which eliminates the need to learn or tune parameters based on a specific dataset. Consequently, in a streaming data environment, we can quantize each data point immediately upon its arrival. The KV cache is a prime example where this property is essential: keys and values are generated sequentially on the fly, demanding real-time quantization before being inserted into the cache. This immediate, on-the-spot processing is precisely the online property we attribute to TurboQuant. We apologize if this was not explicitly articulated in the initial submission and assure the reviewer that the final manuscript will be modified to provide greater clarity on this point.

I would like to thank the authors for their explanations and additional experiments. However, it appears that the authors have not fully addressed my concerns on the two major contributions:

1. Concerning the mutual independence of the random projection of $x$: If the coordinates of the projected $x$ are not independent and the distribution of $x$ is not preconditioned, how can the vector quantization problem be equivalently transformed or simplified into a scalar quantization problem? Is vector quantization equivalent to scalar quantization without any condition on data distributions? This should be incorrect. Moreover, note that although the paper title claims to investigate vector quantization, it have actually focused on scalar quantization.

2. Regarding the multiplicative bias: Given that the bias is a known constant value of $2/\pi$, why not simply subtract it directly? This approach would be much simpler than the method proposed in the paper.

We thank the reviewer for probing the theoretical foundation of our quantization approach. We would like to first resolve the confusion on coordinate independence. Our response to the concerns about the multiplicative bias will follow in a separate comment.

mutual independence of the random projection of x: The answer lies in the Linearity of Expectation and the Isometry of the random rotation, which allow us to minimize MSE by minimizing coordinate-wise distortion, without requiring the joint distribution to factorize (i.e., without requiring independence).

1. Orthogonal Invariance of MSE: Let $x$ be an arbitrary fixed input vector. We apply a random orthogonal matrix $\mathrm{\Pi }$ to get $z=\mathrm{\Pi }x$. We then scalar-quantize $z$ to get $\hat{z}$. The reconstruction is $\hat{x}={\mathrm{\Pi }}^{\mathrm{\top }}\hat{z}$. Because orthogonal transformations preserve Euclidean norms, the distortion in the original space is identical to the distortion in the rotated space: $$|x-\hat{x}{|}^2=|{\mathrm{\Pi }}^{\mathrm{\top }}(z-\hat{z}){|}^2=|z-\hat{z}{|}^2=\sum _{i=1}^d(z_i-{\hat{z}}_i{)}^2$$
2. Linearity of Expectation (The Key Mechanism): We seek to minimize the expected MSE over the randomness of $\mathrm{\Pi }$. By linearity of expectation: $$\mathbb{E}[|x-\hat{x}{|}^2]=\sum _{i=1}^d\mathbb{E}[(z_i-{\hat{z}}_i{)}^2]$$ To minimize this sum, it suffices to minimize each term $\mathbb{E}[(z_i-{\hat{z}}_i{)}^2]$ individually. This minimization depends only on the marginal distribution of each coordinate $z_i$. It does not require the joint distribution of $z_1,...,z_d$ to be independent.
3. The Role of Random Rotation: While the input $x$ can have arbitrary distribution with weird dependencies, the random rotation $\mathrm{\Pi }$ ensures that for any input $x$, the coordinates of $z=\mathrm{\Pi }x$ are identically distributed (though not independent). Specifically, they follow a predictable Beta distribution (concentrated around 0) determined solely by the dimension $d$ and norm $|x|$ (see Lemma 1). Since the marginal distribution is known and fixed (Beta), we can design a single optimal scalar quantizer (Lloyd-Max) that minimizes $\mathbb{E}[(z_i-{\hat{z}}_i{)}^2]$.
4. Conclusion on "Equivalence": Therefore, the problem of minimizing the expected VQ distortion for $\mathbf{x}$ is mathematically equivalent to minimizing the SQ distortion for the Beta-distributed coordinates $z_i$. This holds strictly due to the geometry of the rotation, without assuming independence of the input $x$ or the projected vector $z$.

Regarding the Paper Title: The method is a Vector Quantization scheme because it operates on the vector as a whole (applying a $d\times d$ rotation matrix) to exploit the geometry of the high-dimensional sphere. A pure Scalar Quantizer would process coordinates in isolation; our method couples them via rotation to "democratize" the energy and standard deviations, achieving vector-level distortion guarantees that pure scalar quantization on natural data cannot.

Key Remark: The assertion in Line 215 regarding "near-independence" was primarily intended to provide an intuition on the efficiency of the quantization rate (an entropy argument); however, the mathematical argument for minimizing distortion (MSE) depends solely on the marginal moments, which is a significantly less stringent requirement. This initial intuition is, nevertheless, a strong approximation, since in high dimensions, the distribution of the rotated vector $z=\mathrm{\Pi }x$ converges toward a Normal distribution. Because the covariance between any pair of coordinates in $z$ is zero, resulting in a covariance matrix for $z$ that is diagonal, the coordinates of $z$ are approximately independent (If z had a Normal distribution with diagonal covariance, coordinates of z would be independent). This intuitively explains why applying an optimal scalar quantizer to $z$ can achieve a nearly optimal distortion rate.

We are committed to incorporating this detailed discussion into the final version of the paper to ensure maximal clarity for our readers.

Regarding the multiplicative bias, we appreciate the reviewer’s suggestion to use simple rescaling (multiplying by $\pi /2$) to correct the bias. You are absolutely correct that if the goal is solely to obtain an unbiased estimator, simple rescaling is the standard and most efficient approach.

However, we chose the residual quantization approach for a reason tied to the inner product error objectives of our work defined in Equation (2) of our paper. While rescaling by $\pi /2$ eliminates the bias, it strictly increases the variance (and consequently the inner product error) of the estimator quadratically. Therefore, fixing the bias via scaling amplifies the distortion by a factor $(\pi /2{)}^2$.

On the other hand, the residual’s norm decays at an optimal rate as shown in Theorem 1, Thus applying an unbiased quantizer on the residual provides a fine-grained correction that further reduces the distortion, rather than amplifying the noise of the coarse estimate. This discussion can be made more formal and we assure you a formal discussion on this will be added to the final version of the paper in case of acceptance.

Thank you for your further explanations. I would like to raise the score, yet there seems to be no option to revise the score. I hope the AC will take my opinions into account when formulating decisions.

We appreciate the reviewer’s insightful feedback. Our response to you're comments/concerns are as follows:

Additional KV compression experiments: same bit-widths, RotateKV benchmark, and ablating the impact of two-tier channel-wise quantization strategy:

First, we'd like to clarify the goal of the KV cache experiments in table 1 was to show TurboQuant is consistently better than models with higher bitwidths, simultaneously reducing the bitwidth and increasing the quality scores. For each competing method we let the bitwidth of that method to be more than TurboQuant and showed that even in this case TurboQuant still has higher accuracy. Please note that this demonstrates the strength of TurboQuant relative to other methods even more than if we compared them at the same number of bits.

Furthermore, we have now run new experiments on the GSM8K mathematical reasoning dataset, using the Qwen2-7B-Instruct model (both model and dataset are new additions not present in the initial submission), fixing the bit-width for all methods at 3 bits. This analysis also incorporates a comparison against the new baseline method, KV-Rotate suggested by the reviewer. The resulting performance table is presented below:

Next, we present the experimental results on the long-bench dataset from Table 1 while the two-tier channel-wise quantization is disabled and we precisely used the TurboQuant algorithm from Algorithm-1 of the paper. The results for 3-bit TurboQuant on Llama-3.1-8B-Instruct model are as follows:

While a slight degradation in quality is observed, the resulting performance remains superior to that of other baselines employing comparable bit-widths. We commit to integrating both these results and the complete set of new findings into the final version of the paper.

ANN experiments with stronger PQ methods:

Thank you for suggesting new relevant quantization methods to include in our ANN studies. We ran new experiments to benchmark our performance against more recent quantization papers, including NestQuant, LSQ++, and OPQ. For your initial review, we present the recall metrics on the GloVe ($d=200$) dataset with bit-width=4, and we commit to incorporating the results for the OpenAI datasets as well in the final version of the paper.

on the importance of quantization time:

We agree with the reviewer that, for the typical Approximate Nearest Neighbor (ANN) problem, the one-time indexing runtime is often less critical than the query or search latency. However, there are important applications requiring online and dynamic ANN—such as certain sparse attention inference servers that rely on real-time ANN to identify top-k attention scores—place a crucial premium on quantization and indexing speed. The primary use case for our online quantization approach is the Key-Value (KV) cache, where data points genuinely arrive as a stream, necessitating rapid, on-the-fly quantization where the temporal cost of quantization is highly significant. The ANN application serves mainly as a demonstration of our quantization’s quality which was able to achieve very high recall.

I would like to thank the authors for providing additional results on each aspect and enhancing the soundness of the paper. The results with KV-Rotate and two-tier channel-wise quantization disabled clarify the performance of TurboQuant against other methods for the main results. Adding better baselines for ANN and clarifying potential application for fast data-independent quantization make this part much more sound.

Given that those new results and clarifications will be integrated into the final version of the paper, I recommend acceptance, without further doubts on the soundness, and change my rating accordingly.