We make Pinku, one of several apps that run AI models on the iPhone. The explanation applies to all of them.
What "local AI" means
A language model is a large file of numbers (its weights) plus code that uses them to predict the next word. With a cloud chatbot such as ChatGPT, Gemini or Claude, the weights stay in the provider's data centre and every message travels there. With local AI, you download the weights once and the iPhone's own chip runs them. After that it works in airplane mode, and your text does not leave the device.
Why memory decides what runs
- Size comes from parameters and bits. A model's weights take roughly parameters x bits per weight / 8 bytes. A 1-billion-parameter model stored at 4 bits is around half a gigabyte of weights; real files are a bit larger. In Pinku's catalogue, Llama 3.2 1B at 4 bits is 0.7 GB and Qwen 3 8B is 4.7 GB.
- Quantization is what makes phones possible. Storing each weight in 4 bits instead of 16 makes the file about four times smaller, at some cost in quality. Apple compresses its own on-device model, about 3 billion parameters, to 2 bits per weight (Apple Machine Learning Research, 2025).
- The whole model must fit in RAM while it runs, next to iOS and the app, plus working memory that grows with the length of the conversation. iOS closes an app that asks for too much, so the limit is the memory iOS lets one app use, not storage.
- Speed comes from the GPU. Apps such as Pinku and Locally AI use Apple's MLX framework to run the model on the iPhone's graphics processor; smaller models answer faster.
The rule we read in Pinku's code
We read the model code of the version on sale today (1.1). Pinku lists a model only if its
download is no more than 60% of the device's physical memory. Before loading one
on an iPhone, it asks iOS how much memory the app may still use (os_proc_available_memory)
and loads the model only if that is at least the model's size plus 10%; if not, it
says so and suggests a smaller model instead of crashing. In practice Gemma 3 1B (0.8 GB, the
default) needs about 1.4 GB of memory, Qwen 3 4B (2.3 GB) about 3.9 GB, and Qwen 3 8B (4.7 GB) about
7.9 GB, which on most iPhones rules it out and leaves it to iPads and Macs with more memory.
What runs where on an iPhone
| What | Where it runs |
|---|---|
| Apple Intelligence on-device model (writing tools, summaries) | On the iPhone, a ~3B model at 2 bits per weight (Apple) |
| Bigger Apple Intelligence requests | Apple's server model on Private Cloud Compute; Apple lists features that rely on server models, such as Image Playground, as having daily limits (Apple, 121115) |
| ChatGPT, Gemini, Claude and other cloud apps | The provider's servers; needs a connection for every message |
| On-device chat apps: Locally AI, PocketPal AI, Private LLM, Enclave, Pinku | On the iPhone, with open models you download (Llama, Qwen, Gemma and others) |
Step by step
- Check your iPhone. Apple Intelligence needs an iPhone 15 Pro or later. For open models, more memory means bigger models: a 1B model runs on most recent iPhones, an 8B model needs a device with about 8 GB.
- Free some storage. Models are 0.4 to 5 GB each. Settings, General, iPhone Storage shows how much room you have.
- Install an on-device chat app. Locally AI, PocketPal AI, Private LLM, Enclave or Pinku. Read the App Store privacy label: local inference does not stop an app from collecting other data.
- Download a small model on Wi-Fi. Start with a 1B model. It answers fast and leaves memory for the rest of the phone.
- Test it in airplane mode. Turn on airplane mode and send a message. If it answers, it is really running on the iPhone.
- Try a bigger model if it fits. A 3B or 4B model gives better answers but is slower and needs more memory. Close other heavy apps first.
- Know the limits. Local models are smaller than cloud ones: less knowledge, more mistakes, no web search. Check anything important.
Where Pinku fits
Pinku runs open models (Gemma, Llama, Qwen, SmolLM3, DeepSeek-R1 Distill and others) on iPhone and Mac with MLX, keeps chat history only on the device and has no account. It lists only the models your device can hold, and its larger models are part of Pinku Pro. It has no web search, no image input and no image generation. Other apps go further in some directions: PocketPal AI loads any GGUF model from Hugging Face, and Private LLM works with Siri and Shortcuts. To see which models fit your device, use which AI model fits my iPhone or Mac; for a trip, how to use AI offline on a plane.
Sources
- Apple Machine Learning Research, Updates to Apple's On-Device and Server Foundation Language Models, 9 June 2025, updated 17 July 2025.
- Apple, Apple Intelligence features and requirements (121115), published 22 October 2026.
- Pinku 1.1 source code (ModelMemory.swift, model list filter, models.json), read 23 October 2026.
- MLX, Apple's array framework for Apple silicon.
Questions people ask
How does AI run locally on an iPhone?
You download a model file once, and the app runs it on the iPhone's GPU. The text you type is processed on the device, so it works without internet.
How much RAM do I need to run an AI model on iPhone?
Roughly the model's size plus room for iOS and the conversation. Pinku lists a model only if it is at most 60% of the device's memory: about 1.4 GB for a 0.8 GB model, about 7.9 GB for a 4.7 GB one.
Is Apple Intelligence local?
Partly. Apple runs a model of about 3 billion parameters on the device and sends bigger requests to its server model on Private Cloud Compute.
Why are local models smaller than ChatGPT?
They have to fit in the phone's memory. Cloud models run on data-centre hardware with far more memory, so they know more and make fewer mistakes.
What is quantization?
Storing each weight in fewer bits, for example 4 instead of 16, which makes the model about four times smaller at some cost in quality.
Does local AI drain the battery?
Generating text uses the GPU hard, so long chats use noticeable battery and warm the phone. Smaller models use less.