Gemma 4 E4B on a 64 GB Mac: speed, quality, memory
Google, released Apr 2026. 8B parameters, dense. 4-bit MLX build, 5.2 GB download. License Apache-2.0.
- HumanEval+ pass@1: 137 of 164 (83.5%), rank 5 of 8
- HumanEval pass@1: 143 of 164 (87.2%)
- Generation speed: 80.5 tokens per second (5 trials, 79.4 to 81.3); prompt 3,353 tokens per second
- Peak memory: 5.1 GB · power while generating: 22 W · GPU peak 75 °C
Run it yourself
pip install mlx-lm mlx_lm.chat --model mlx-community/gemma-4-e4b-it-4bit
Problems it failed
HumanEval/26, HumanEval/32, HumanEval/36, HumanEval/47, HumanEval/59, HumanEval/65, HumanEval/76, HumanEval/89, HumanEval/92, HumanEval/93, HumanEval/95, HumanEval/103, HumanEval/113, HumanEval/115, HumanEval/118, HumanEval/122, HumanEval/130, HumanEval/132, HumanEval/145, HumanEval/160, HumanEval/163
Raw files: speed.log, evaluate.log, eval_results.json