Pull down to go back
Speculative Decoding Silently Broken for Qwen3.6 on TurboQuant Fork — PR to Fix

Speculative Decoding Silently Broken for Qwen3.6 on TurboQuant Fork — PR to Fix

Qwen3.6 在 TurboQuant 分支上的推測解碼功能暗中失效——已提交修復 PR

If you're running Qwen3.6-35B-A3B on the TurboQuant fork and tried speculative decoding, it was quietly doing nothing. The server just falls back to normal decoding without any error message. Here's the basic idea if you're unfamiliar: you run a tiny model (like Qwen3.5-0.8B) alongside your big model. The small one guesses the next bunch of tokens really fast, then the big model checks all the guesses in one pass. Whatever the big model agrees with, it keeps — whatever it rejects, it redoes. So if your big model is fast enough, you get a significant speed boost for free.