Potential of Gemma4 Per-layer embeddings?

Hey there people. So let's talk about GEMMA 4 per layer embeddings. How far can they go? Are they streamlined clear-cut knowledge stored inside of those embeddings, while the model parameters are just for logic? Or is it like all other LLM phenomena where nothing can be said to be responsible for one single aspect of the entire performance? If it is a clear-cut storage of knowledge that the model uses as a lookup table, how far could it go and can more knowledge be added? Can the embeddings be m