Run Gemma 4 and Other LLMs Offline on Your Phone — Right Now
We've all been there: You're on a train, in a coffee shop with spotty Wi-Fi, or just trying to avoid burning through your data plan. You want to tinker with a language model, but the cloud feels miles away. Running an LLM locally on a phone used to be a pipe dream — or at least something that required compiling TensorFlow Lite yourself and praying the model fit in 4GB of RAM.
That's changing. And the Google AI Edge Gallery on GitHub is making it a lot easier to actually play with models like Gemma 4 on-device, without an internet connection.
What It Does
The Google AI Edge Gallery is a curated collection of demos, tools, and sample apps that show you how to run AI models — including large language models (LLMs) — directly on your Android or iOS device. Think of it as a developer sandbox for mobile AI. It's not a single tool; it's a gallery of working examples that leverage Google's AI Edge infrastructure (MediaPipe, TensorFlow Lite, etc.) to run models locally.
Inside, you'll find things like:
- Gemma 4 (the latest lightweight LLM from Google) running entirely offline on a phone.
- Image captioning that never leaves your device.
- Text classification and question answering using quantized models that fit in mobile memory.
- Audio processing demos for things like wake word detection.
Each demo comes with source code and instructions. You're not just reading about it — you can actually build or run these apps yourself.
Why It's Cool
A few things stand out here.
First, privacy by default. When the model never leaves your phone, your prompts and data never leave your phone. No cloud calls, no logging, no API fees. This matters for prototyping sensitive apps or just avoiding the creepiness of sending everything to a remote server.
Second, offline capability is a superpower. You can use an LLM on a plane, in a remote area, or in a building with terrible reception. That opens up use cases like on-device document summarization, personal assistants that don't need a SIM card, or even emergency communication tools.
Third, it's surprisingly performant. The demos use quantized models and optimized runtimes. You won't get GPT-4 levels of complexity, but for things like chat, summarization, or simple Q&A, it's fast enough to feel interactive. The Gemma 4 model, in particular, is designed to be small enough to run on a phone while still producing coherent responses.
And the gallery is open source. No "request access" wall. No "email us for a demo." Just a GitHub repo with code you can clone and build.
How to Try It
The easiest way is to check out the repository and follow the README for the demo you care about.
-
Clone the repo:
git clone https://github.com/google-ai-edge/gallery.git cd gallery -
Pick a demo — for example, the Gemma 4 chat app for Android lives under
examples/gemma_chat/. -
Open it in Android Studio (or Xcode for iOS) and build. The README for each demo has specific instructions, including model download links and necessary dependencies.
If you just want to see it in action without building, some demos include prebuilt APKs or links to a test app. Check the releases/ folder or the individual demo READMEs.
You'll need a reasonably modern phone with at least 4GB of RAM (8GB recommended for larger models). And yeah, download the model file once over Wi-Fi — after that, it's fully local.
Final Thoughts
This isn't a revolutionary new model. It's a practical, well-packaged way to run existing models where they're most useful: on the device in your pocket. For developers, it's a great starting point for experimenting with on-device AI without wrestling with the underlying inference infrastructure.
If you've been curious about running LLMs locally or building privacy-first mobile apps, this gallery is worth a Saturday afternoon. Clone it, pick a demo, and see what happens when your model doesn't need a cloud to think.
Follow @githubprojects for more dev-friendly project highlights.