An offline AI chatbot runs the language model on your own device instead of sending your messages to a server. You download a model file once, and after that the chat works in airplane mode, on a plane, in a tunnel, or anywhere with no signal. Nothing about the conversation touches the internet.
This is genuinely possible on ordinary phones now, and the catch is size: a model small enough to fit on a phone is not as capable as a large cloud model. For conversation, roleplay and companionship that gap matters much less than it does for research or coding.
How an offline chatbot works #
Cloud chatbots keep the model on powerful servers and send your text back and forth. An offline app instead ships or downloads the model as a file, typically in a compressed format like GGUF, and runs it with the phone’s own processor and memory.
Three things make this practical:
- Quantization. Model weights get stored at lower precision, usually four bits instead of sixteen. A model that would need 16 GB shrinks to around 4 GB with a modest quality cost.
- Small models that got good. Two- to nine-billion-parameter models now handle conversation and character work far better than models twice their size did two years ago.
- Phone hardware. Recent phones have enough RAM and fast enough memory bandwidth to generate text at readable speed.
The result is a chatbot with no per-message cost to the developer, no server to breach, and no connection required.
What you need to run one #
| Model size | Typical download | RAM needed | What it’s good for |
|---|---|---|---|
| 2B, 4-bit | 1.2-1.5 GB | ~3 GB | Conversation, companions, simple roleplay |
| 4B, 4-bit | 2.5-3 GB | ~6 GB | Sharper character work, better recall |
| 7-9B, 4-bit | 4-5.5 GB | ~8 GB | The best a phone runs well |
Two numbers matter and people mix them up. The download size is disk space you need once. The RAM requirement is what the phone needs free while the model is loaded, and it’s the one that decides whether an app runs at all. A phone can have 200 GB of storage and still be unable to load a 3 GB model.
Also budget headroom. The operating system reserves a large share of RAM, so a “6 GB” phone has nowhere near 6 GB available to an app. Well-built apps read the device’s memory and refuse to offer models it can’t actually run rather than letting you download five gigabytes and then crash. The full sizing guide is in how much RAM you need to run AI on your phone.
What you give up, honestly #
- Breadth of knowledge. A small model knows much less about the world and will invent specifics more readily. Don’t use one as a reference.
- Reasoning. Multi-step logic, math and code are where the size gap shows most.
- Context length. On-device models typically work with a few thousand tokens of context, so they hold a shorter stretch of conversation word for word. This is why saved memory matters so much on the device. See why AI chatbots forget things.
- Voice and images. Usually absent, or much weaker, since those need their own models.
- Battery and heat. Sustained generation warms the phone and drains the battery faster than ordinary app use.
What you get in exchange: it works with no signal, it costs nothing per message, and your conversations never leave the device.
Setting one up #
- Check your phone’s RAM. On Android, Settings has a memory entry, or check the model’s spec sheet. iPhones don’t display it; look up your model, since Apple doesn’t advertise the number.
- Download the model on Wi-Fi. These files are over a gigabyte. Good apps refuse to pull them over cellular without an explicit confirmation.
- Expect a slow first load. The model has to be read into memory the first time, which can take several seconds.
- Turn on airplane mode and test it. A real reply with no connection is proof the model is local.
Offline AI companions #
Companion chat is the use case that suits a phone-sized model best. The job is staying in character and remembering you, not recalling facts about the world, and small models do the first two surprisingly well.
Xin is an offline companion app of this kind. Its free model, Xin Core, is about a 1.3 GB download and runs on phones with roughly 3 GB of RAM; subscribers can switch to larger models that need around 6 GB or 8 GB, and the app greys out any model the device can’t run. After that one download every conversation is generated on the phone, with no account, no login, and no server involved. Chats, characters and the memory the companion keeps about you live in a local database, and an “Erase all data” option in the Me tab clears the lot.
The reason the offline part is more than a convenience feature here: a companion conversation is usually the most personal thing on a phone. With local inference there’s no server copy to leak, review or train on. The one honest caveat is that the free tier shows ads, which do need a connection and involve an advertising identifier, so a free offline app is not a zero-network app. Private AI chat apps covers how to check those claims properly.
How to tell a real offline app from a marketing claim #
“Private” and “secure” are unregulated words. “Offline” is testable.
- Airplane-mode test. Finish setup, disconnect, send a message. A contextual reply means local inference.
- Look for a model download. If the app never downloaded a gigabyte-scale file, there’s no model on your phone.
- Check the store privacy label. An app processing chats on a server almost always has to declare that messages are collected.
- Watch for a hybrid. Some apps run small tasks locally and route harder ones to the cloud. A reply in airplane mode proves local inference exists, not that nothing is ever sent.
Frequently asked questions #
Can ChatGPT work offline? #
No. ChatGPT runs on OpenAI’s servers, and the apps need a connection for every message. Offline chat requires an app built around a model that lives on your device.
Is an offline AI chatbot as good as ChatGPT? #
Not for knowledge, reasoning or coding, where the size difference is large and obvious. For open-ended conversation, character roleplay and companionship, a good 2B to 9B model on a phone holds up well, and it adds privacy and offline use that a cloud service can’t offer.
Does an offline AI chatbot need the internet at all? #
Only to download the model, and for ads or purchases if the app has them. The chat itself doesn’t. Once the file is on the phone you can stay in airplane mode indefinitely and keep talking.
How much storage does an offline AI app need? #
Plan on the model file plus about a gigabyte of headroom. A small model needs roughly 2 to 2.5 GB free, larger ones 4 to 7 GB. The app’s model screen usually states the exact download size before you commit.
Will running AI offline drain my battery? #
More than texting, less than a 3D game. Generating a reply is a burst of heavy computation lasting a few seconds. An hour of active chatting is noticeable on the battery; leaving the app open without sending anything costs almost nothing, because nothing runs between messages.