Running Local AI on a 16GB MacBook Air
My laptop is an M4 MacBook Air with 16GB of unified memory. For local AI that is not much. Most of the setups you see online have 64GB or more, or a desktop with a big GPU. But 16GB is what a normal person actually buys, so I wanted to know what local hosting looks like on this kind of machine, not on a workstation.
Short version: a 9B model is smart enough but too slow, and a small MoE model turned out to be the sweet spot.
The setup
I used llama.cpp on macOS. llama-server pulls a GGUF straight from Hugging Face and serves it on a local port, so trying a new model is one command.
This is the command for the 9B model I ended up preferring:
llama-server -hf ornith-ai/Ornith-1.5-9B-GGUF:Q4_K_M \
-ngl 99 -np 2 -c 49152 -fa on --temp 0.6 \
--cache-type-k q8_0 --cache-type-v q8_0 --no-mmproj \
--host 127.0.0.1 --port 8080 --jinja
Most of these flags are there because of the 16GB limit.
-ngl 99puts every layer on the GPU-cis the context size, and it is the number I kept fighting with-npis the number of parallel slots-fa onturns on flash attention--cache-type-k q8_0 --cache-type-v q8_0quantizes the KV cache, which saves memory so more context fits--no-mmprojskips the vision part, because I only need text--jinjauses the model's own chat template, which tool calling needs
Everything is a Q4_K_M quant. On 16GB there is no room for anything bigger at this model size.
The 9B models
I tried four of them. They are all the same size, so the commands are almost identical.
- Qwen 3.5 9B (
unsloth/Qwen3.5-9B-GGUF). The base model, and my baseline. - OmniCoder 9B (
Tesslate/OmniCoder-9B-GGUF). An agent fine-tune. - Ornith 1.5 9B (
ornith-ai/Ornith-1.5-9B-GGUF). Another agent fine-tune, and the one I prefer. - Qwen 3.5 9B Uncensored (
HauhauCS/Qwen3.5-9B-Uncensored-HauhauCS-Aggressive). An uncensored fine-tune. I ran this one at--temp 0.9, the rest at 0.6.
Ornith is pretty good. It calls tools accurately and it is smart. If I only cared about the quality of the answers, I would stop here.
1.5 is the newer release of Ornith. It is the same size as 1.0, so it needs the same memory, and its model card reports small gains over 1.0 across the board: 70.6 against 69.4 on SWE-bench Verified, and 46.2 against 43.1 on Terminal-Bench 2.1.
The problem is speed. A 9B model on this laptop runs at about 8 tokens per second, and it drops to around 6 when the context fills up. That is fine for one question. It is painful for anything agentic, where the model reads tool output, thinks, calls another tool, and does that ten times in a row.
The other problem is context. A 9B model already takes a big part of the 16GB, so there is not much left. I ran most of them at 32k, and pushed Ornith to 49k. That sounds like a lot until an agent starts reading files.
The tiny one that was not useful
I also tried going the other way with LFM 2.5 2.6B.
llama-server -hf LiquidAI/LFM2.5-2.6B-GGUF:BF16 \
-ngl 99 -c 131072 -np 3 -fa on \
--temp 0.1 --top-k 50 --repeat-penalty 1.1 \
--host 127.0.0.1 --port 8080
It is small enough that I could run it unquantized with a 131k context and three slots. But it was not useful for what I wanted. Having a lot of context does not help when the model cannot do much with it.
Ling 3.0 Tiny
Then I found Ling 3.0 Tiny. It is a lightweight hybrid reasoning MoE model with 7.9B total parameters, but only 1.3B of them are activated per token.
That one detail changes everything on this hardware. The model has the knowledge of a roughly 8B model, but each token only costs about as much compute as a 1.3B model. With a Q4 quant it runs at around 40 tokens per second on my Air, and it can fit a lot of context.
llama-server -hf inclusionAI/Ling-3.0-tiny-GGUF:Q4_K_M \
-ngl 99 -c 65536 -np 1 -fa on --temp 0.6 \
--cache-type-k q8_0 --cache-type-v q8_0 \
--host 127.0.0.1 --port 8080 --jinja
40 tokens per second compared to 8 is not a small upgrade. It is the difference between waiting for the model and actually using it.
It is not perfect. It calls tools semi accurately, so it gets the call right most of the time and not all of the time. It is also not that smart. I would not use it for coding related stuff, and I still reach for Ornith when I need a careful answer and can wait for it.
It runs hot
One thing nobody mentions in the benchmarks is heat. The MacBook Air has no fan, and running a local model keeps the chip busy the whole time it is generating. The laptop gets really hot.
So I use a laptop fan under it. If you plan to run local models on an Air for more than a quick question, get one. It is a cheap fix, and I would not run long sessions without it.
If you don't want the terminal
Everything above is llama-server commands, and that is not how most people want to use AI. For a normal consumer, I think the Unsloth desktop app is a really good way to run local AI.
It gives you a friendly UI, and underneath it uses llama.cpp with optimized flags. So you get roughly what I set up by hand in this post, without having to learn what a KV cache is.
The chat also has a free local search tool, which is really useful. A small model does not know much on its own, and letting it search covers a lot of that gap.
Conclusion
On a 16GB MacBook Air, the limit is not really whether a model fits. A 9B model fits. The limit is whether it is fast enough and has enough context left over to be worth using.
Here is where I landed:
- Ornith 9B when I need accuracy. Good tool calls, smart, 6 to 8 tokens per second.
- Ling 3.0 Tiny for everything else. Around 40 tokens per second, lots of context, semi accurate tool calls.
- LFM 2.5 2.6B is not worth it for my use.
- A laptop fan under the Air, whichever model is running.
- The Unsloth desktop app if you want all of this without the terminal.
I think the MoE approach is the sweet spot for local AI right now. Keep the total parameters reasonably high, activate only a small part of them per token, and a normal laptop can run it at a speed that feels usable. It is not going to replace a cloud model for coding. But it runs on the laptop I already own, offline, for free, and that is a good first step for consumer accessible AI.