往下拉回到首頁
Google 的 TurboQuant 能直接用在 llama-server 了嗎?還是得等開發者合併代碼?

Google 的 TurboQuant 能直接用在 llama-server 了嗎?還是得等開發者合併代碼?

Can we already use Google's TurboQuant (TQ) for KV Cache in llama-server? Or are we waiting for a PR?

Hey everyone, Ever since the day Google announced TurboQuant, I've been following the news about its extreme compression capabilities without noticeable quality degradation. I see it mentioned constantly on this sub, but despite all the discussions, I'm honestly still a bit confused: is it actually applicable for us right now? And if so, how? I recently saw an article/post where someone applied this TQ quantization directly to the model weights. They managed to get Qwen3.5-27B running at near-