Pull down to go back
What speed is everyone getting on Qwen3.6 27b?

What speed is everyone getting on Qwen3.6 27b?

大家在 Qwen3.6 27b 上跑出什麼速度?

I'm getting ~13 tps on Q8_0, with a context window of 128000, K Q8_0, V Q8_0 this is on 3x GPUS (1x2060super 8gb, 2x5060ti 16gb), via llamacpp unsure if this is slow or to be expected?