Pull down to go back
ggml-cpu: Optimized x86 and generic CPU q1_0 dot (follow up) by pl752 · Pull Request #21636 · ggml-org/llama.cpp

ggml-cpu: Optimized x86 and generic CPU q1_0 dot (follow up) by pl752 · Pull Request #21636 · ggml-org/llama.cpp

ggml-cpu:最佳化 x86 和通用 CPU q1_0 點積運算(後續追蹤)by pl752 · Pull Request #21636 · ggml-org/llama.cpp

Available from commit b8858 onwards. This optimized CPU version delivers faster token generation speeds. Testing on an older laptop (16GB DDR3 RAM) showed improvement from 0.3 t/s to 1.7 t/s, though the boost was limited due to lack of AVX/AVX512 support. Further testing planned on newer hardware. Note: Metal, Vulkan, and CUDA versions also support this optimization (1-bit quantization variants available). Worth checking if you haven't already.