00:00 / 00:00
Sleep timer ended
⏪ 10s
⏩ 10s
9.6K views
1 month ago
Learn how per-layer embeddings, or PLE, work in Gemma 4 E2B and E4B, Google's open source language model. Google DeepMind’s Maarten Grootendorst breaks down how…
...more
5 Comments
the red flag is that unchanged active parameters says nothing about memory bandwidth, and ple may still make inference slower on real hardware
Rate Google is moving it's going to be agent size
What strikes me most here is not the 2.3B table, but the move from dense compute to sparse retrieval. Does PLE make the model smarter, or just cheaper to run?
😮😮😮😮😮😮Wow Youtube Team and Google Team this actually Good with 😊
that's why the 4bit quantize of this model is larger than a 8bit quantize of a llama style architecture. <br>which is ridiculous design for a supposedly edge model.<br>in fact, let me tell you exactly how ridiculous it is, at 4bit gemma 12b which is a dense model, is smaller than the 4bit quant of E2B which is only 4b total. 3x more parameters, about 800mbs smaller. (using bnb linear quant)<br>my real question is, none of you geniuses had a calculator to just roughly calculate these numbers?
Up next
