往下拉回到首頁
我們已經可以在 llama-server 中使用 Google 的 TurboQuant (TQ) 進行 KV Cache 了嗎?還是要等待 PR?

我們已經可以在 llama-server 中使用 Google 的 TurboQuant (TQ) 進行 KV Cache 了嗎?還是要等待 PR?

Can we already use Google's TurboQuant (TQ) for KV Cache in llama-server? Or are we waiting for a PR?

Hey everyone, Ever since the day Google announced TurboQuant, I've been following the news about its extreme compression capabilities without noticeable quality degradation. I see it mentioned constantly on this sub, but despite all the discussions, I'm honestly still a bit confused: is it actually applicable for us right now? And if so, how? I recently saw an article/post where someone applied this TQ quantization directly to the model weights. They managed to get Qwen3.5-27B running at near-