# Run AI models on Raspberry Pi with LiteRT and Gemma
Canonical URL: https://zero2vibecode.com/blog/raspberry-pi-edge-ai-litert-gemma
Date: 2026-08-12
Tags: edge-ai, raspberry-pi, models, beginner
Google's LiteRT runtime and Gemma models bring powerful edge AI capabilities to Raspberry Pi, enabling local inference without cloud dependencies.
Running AI models directly on small devices like Raspberry Pi used to mean compromising on capability or performance. Google's new LiteRT runtime and Gemma model family change that equation, letting you build fully local AI applications that see, hear, and respond in real time.
## Why edge AI matters for Raspberry Pi projects
Edge AI means running artificial intelligence models directly on your device instead of sending data to the cloud. For Raspberry Pi projects, this brings three concrete advantages:
1. **Zero latency** - Responses happen immediately because no network round-trip is needed
2. **Complete privacy** - Sensitive data like camera feeds never leaves your device
3. **Offline operation** - Your robot or smart device keeps working without internet
The challenge has been fitting capable models onto the Pi's limited hardware. LiteRT solves this with an optimized runtime that squeezes maximum performance from the Pi's CPU and GPU.
## Gemma models designed for edge devices
Google's Gemma family provides several model sizes tailored to different edge use cases:
| Model | Size | Best for | Tokens/sec on Pi 5 |
|------------------|--------|-----------------------------------------------|--------------------|
| Gemma 3 270M | 270M | Simple classification, entity extraction | 15 |
| EmbeddingGemma | 300M | Semantic search, RAG applications | 12 |
| Gemma 3 1B | 1B | Multilingual text generation | 8 |
| Gemma 4 E2B | 2B | Real-time speech, translation | 9 |
| Gemma 4 E4B | 4B | Complex reasoning and planning | 5 |
The Gemma 4 E2B model hits a sweet spot for most projects, generating text at 300 words per minute - faster than human speech.
## How LiteRT makes this possible
LiteRT is a specialized runtime that handles three critical optimizations:
1. **Memory efficiency** - Peak memory usage stays under 1.5GB even for the 2B parameter model
2. **CPU acceleration** - Uses XNNPACK to maximize performance on the Pi's ARM processor
3. **GPU offloading** - Can run vision models on the VideoCore VII GPU while language models use the CPU
This dual-processor approach is key to real-time performance. In the Reachy Mini robot demo, the system runs object detection on the GPU while handling speech recognition and response generation on the CPU.
## What you can build today
With these tools, you can create projects like:
- **Voice assistants** that respond instantly without cloud dependencies
- **Smart cameras** that analyze video locally for privacy-sensitive applications
- **Educational robots** that interact naturally while staying entirely offline
- **Industrial monitors** that process sensor data at the edge
The [Gemma Translator GitHub repo](/blog/claude-code-artifacts) shows a complete example of an offline voice translation system.
## Getting started with LiteRT CLI
Google provides a command-line tool that handles model conversion and execution:
```bash
pip install litert-cli
litert lm run --from-huggingface-repo=litert-community/gemma-4-E2B-it-litert-lm
```
You'll need a Hugging Face authentication token to download models. Create a free account if you don't have one.
The CLI supports:
- Model quantization (reducing size with minimal accuracy loss)
- Performance benchmarking
- Local inference execution
## Performance tradeoffs to consider
While impressive, edge AI still involves compromises:
| Factor | Cloud AI | Edge AI (Pi 5) |
|-------------------|-------------------|--------------------|
| Model size | Hundreds of GB | Up to 4B parameters|
| Response time | Network-dependent | 9-15 tokens/sec |
| Setup complexity | Simple API calls | Local optimization |
| Data privacy | Lower | Complete |
For many projects, especially those involving cameras, microphones, or sensitive data, the privacy and latency benefits outweigh the smaller model sizes.
## Frequently asked questions
The LiteRT runtime works on Pi 4 and Pi 5, but for best performance use the Pi 5 with its faster CPU and GPU. The 8GB RAM model is recommended for the larger Gemma variants.
Yes, the smaller Gemma models (270M and 1B) are designed for task-specific fine-tuning. You'll need to do this training on more powerful hardware before deploying to the Pi.
## Where this technology is headed
Google plans to add support for AI accelerator chips like the Hailo-8, which could provide 10-100x performance improvements. This would enable even more sophisticated local AI applications while maintaining the privacy and latency benefits of edge computing.
For now, the combination of LiteRT and Gemma models gives Raspberry Pi developers unprecedented access to local AI capabilities without compromising on responsiveness or data security.
Want to try all of this hands-on? Start with the free [Vibe Coding 101](/learn/vibe-coding-101) course.
Based on Google's announcement, "Mastering Edge AI on Raspberry Pi with LiteRT and Gemma". Written for people learning to build with these tools.