Gemma 4 E2B on a 64 GB Mac: speed, quality, memory
Google, released Apr 2026. 5.1B parameters, dense. 4-bit MLX build, 3.6 GB download. License Apache-2.0.
- HumanEval+ pass@1: 127 of 164 (77.4%), rank 8 of 8
- HumanEval pass@1: 137 of 164 (83.5%)
- Generation speed: 130.5 tokens per second (5 trials, 130.3 to 130.6); prompt 7,728 tokens per second
- Peak memory: 3.5 GB · power while generating: 24 W · GPU peak 71 °C
Run it yourself
pip install mlx-lm mlx_lm.chat --model mlx-community/gemma-4-e2b-it-4bit
Problems it failed
HumanEval/10, HumanEval/26, HumanEval/32, HumanEval/37, HumanEval/39, HumanEval/44, HumanEval/54, HumanEval/59, HumanEval/64, HumanEval/65, HumanEval/79, HumanEval/83, HumanEval/84, HumanEval/89, HumanEval/93, HumanEval/107, HumanEval/115, HumanEval/118, HumanEval/122, HumanEval/130, HumanEval/132, HumanEval/140, HumanEval/145, HumanEval/152, HumanEval/156, HumanEval/160, HumanEval/163
Raw files: speed.log, evaluate.log, eval_results.json