The home AI rig
What it does
My home PC runs Qwen3.8-27B, an open-weights language model, on two gaming graphics cards. It answers like a hosted chatbot, but every word is computed in my house. I chat with it in a browser, point coding agents at it, and the transcription pipeline on this site uses it to title voice memos. The smaller card runs Whisper transcription at the same time, so the two jobs share it.
Why I built it
I wanted to know how far a home machine can go. Private notes and recordings stay on my own disks, and there is no bill per word. Then it became a game of tuning, and the tuning is where I learned the most.
Stack
- Ryzen 5 5600X, 32 GB RAM
- RTX 3070 16 GB (memory-modded) + RTX 3060 12 GB
- Windows with WSL, official llama.cpp
- Qwen3.8-27B, ByteShape IQ4_XS, 3.84 bits per weight
- 90k context, f16 KV cache
- Model split 4.2 to 1 across the two cards
- Whisper shares the 3060
The 3070 left the factory with 8 GB. The memory mod that doubled it to 16 GB is the work of Frazer, who is a genius, with GPU Solutions and the Graphics Card Repairs channel. Merci.
Old file versus new file
Same machine, same tests, two model files. The old one is Unsloth UD-Q4_K_XL, the new one is ByteShape IQ4_XS.
| Measure | Old | New |
|---|---|---|
| Context (tokens) | 32,768 | 90,112 |
| Lowest free VRAM on the 3060 during Whisper | 1,340 MiB | 1,021 MiB |
| Generation speed | 40.2 tok/s | 41.1 tok/s |
| Draft acceptance | 60.8% | 63.5% |
| Regression set | 20/20 | 20/20 |
| Hard set | 17/40 | 18/40 |
| Coding tasks | 9/10 in 82 min | 10/10 in 44 min |
Source: my own benchmark, 25 September 2026.
The seven fixes that mattered
- Switching the model file gave nearly three times the context at the same speed.
- The separate MTP draft file was redundant, because the draft head already sits inside the main GGUF. Removing it freed about 1 GB on the 3060.
- A q8 KV cache did not help, because what grows with context is the compute buffers, not the KV cache.
- Claude Code reserves 20k output tokens for models it does not know. Setting CLAUDE_CODE_MAX_CONTEXT_TOKENS and CLAUDE_CODE_MAX_OUTPUT_TOKENS=12000 fixed it. CLAUDE_CODE_AUTO_COMPACT_WINDOW does nothing below 100k.
- Open WebUI's built-in tools turned a simple "hi" into 6,400 prompt tokens. With them switched off, "hi" costs 56.
- On Windows, the Ollama uninstaller kills every process named llama-server.exe, including yours. Bon courage.
- Claude Code now starts in auto mode, which asks a model whether each Bash command is safe, using a prompt of about 30,000 tokens. With a local model behind every alias, Qwen runs that check itself and the coding session drops out of its cache. Turning auto mode off for local sessions, with an allowlist of my usual commands, cut a 9-step test task from about 3 minutes to about 1 (my own test, 7 October 2026). Qwen checking Qwen was never much of a guard.
The payoff
Qwen built The Demo Disc alone, in one night.
If you try this at home
Build a small benchmark with your own tasks before you change anything. Without it you are tuning on vibes. With it, every change gets a verdict, et voilà.