Ian Ballantyne
@iballantyne.bsky.social
✨AI Developer Relations Engineer @ Google. I work on Gemini API, Gemma, AI Edge and more!
London, UK
https://www.linkedin.com/in/ianballantyne
London, UK
https://www.linkedin.com/in/ianballantyne
How fast is #gemma3 270M out of the box on a @raspberrypi.com 5? About 30 tok/s on CPU for the Q4_0 model using #ollama. Tried using #llamacpp, got about 32 tok/s. 🤯
Extremely promising for edge compute with no accelerator! Model is only ~250Mb on disk. Can't wait to fine-tune for my IoT projects 👨💻
Extremely promising for edge compute with no accelerator! Model is only ~250Mb on disk. Can't wait to fine-tune for my IoT projects 👨💻
August 15, 2025 at 12:39 AM
How fast is #gemma3 270M out of the box on a @raspberrypi.com 5? About 30 tok/s on CPU for the Q4_0 model using #ollama. Tried using #llamacpp, got about 32 tok/s. 🤯
Extremely promising for edge compute with no accelerator! Model is only ~250Mb on disk. Can't wait to fine-tune for my IoT projects 👨💻
Extremely promising for edge compute with no accelerator! Model is only ~250Mb on disk. Can't wait to fine-tune for my IoT projects 👨💻