Qwen 3.6 35B Model Has a Serious Problem: 40% of Its Brain Cells Are Dead

A developer discovered something alarming while debugging the Qwen 3.6 35B model at the neuron level: massive numbers of neurons in the MoE (Mixture of Experts) layers are completely dead, outputting zero values. In the model's gate and up-projection weights, they found 40% of neurons aren't firing at all. Interestingly, the smaller Qwen 3.5 9B model doesn't have this issue—all its neurons are active. The cause is still a mystery, and it raises questions about whether this is a training artifact, quantization problem, or something else entirely. This could affect model quality and efficiency in ways we don't fully understand yet.