Somewhere between "runs on a Raspberry Pi" and "needs its own power substation" sits an entire universe of AI models, and almost nobody explains what actually changes as you climb that ladder. So let's climb it — using Qwen 3, Alibaba's open-weight model family, as our tour guide. It spans 0.6 billion to 235 billion parameters in one tidy, Apache-licensed lineup, which makes it the perfect specimen for a "what do I actually get for more parameters" field guide.
Two scope notes up front. First, this covers only Qwen 3's text-to-text models — the chat-style ones. Text-to-image, text-to-video, and text-to-speech models are a different animal entirely, with their own hardware and cost profiles. Second, a parameter is roughly one adjustable "knob" the model learned during training — more knobs generally means better reasoning, but also more RAM, more electricity, and more waiting. The whole game is finding the smallest model that's still good enough for what you're doing.
One trick worth knowing before we start: MoE, or Mixture-of-Experts. A dense model uses all its parameters for every word. An MoE model only wakes up a small subset of specialist "experts" per token — which is how a 30-billion-parameter model can run at roughly the speed of a 3-billion-parameter one. Closest thing to a free lunch in this field.
And one mechanic that explains almost every number below: speed is a memory-bandwidth problem, not a compute problem. A discrete GPU (RTX 3060, 3090, 4090, etc.) has its own dedicated VRAM running at several hundred GB/s to over 1 TB/s — as long as a model fits in it, you get GPU speed. An integrated GPU (AMD Radeon 680M/780M, Intel Arc iGPU) has no VRAM of its own — it shares regular system RAM, typically ~50–90 GB/s on dual-channel DDR5, so it's bottlenecked the same way plain CPU inference is. Apple Silicon (M1–M4) is the exception: also unified memory, no separate VRAM, but built 100–800+ GB/s wide, which is why an M-series Mac competes with discrete GPUs despite the shared-memory design. Same logic explained back in Part I of this series: AVX2 and core count affect the delay before a model starts answering, not how fast it types once it starts.
Hardware: A phone, a Raspberry Pi, a 2019 Chromebook, any AMD/Intel iGPU if the device has one. RAM: ~1–2 GB — the weights themselves are under 1 GB. Cost: Free, if you already own the device. Speed: ~5–10 tokens/sec on weak hardware like a Pi or a Celeron-class chip (same range as the 0.5B numbers from Part I), climbing to 20–40+ on a modern phone or laptop with faster memory. Good for: Autocomplete, simple classification, on-device assistants.
Don't expect deep reasoning here — think fast, literal intern, easily confused by nuance.
Real-world application: a smart-home hub that parses "turn off the living room lights" entirely on-device, with no cloud round-trip, no internet dependency, and no voice recording leaving the house.
Hardware: Any modern laptop CPU or iGPU — no discrete GPU needed. RAM: ~4 GB. Cost: Free-ish — you probably already own hardware for this. Speed: 15–30 tokens/sec on CPU alone. Good for: Drafting, light chat, basic coding help, short summaries.
This is roughly where it stops feeling like a toy and starts feeling like a genuinely useful assistant.
Real-world application: a note-taking app's built-in "clean this up" button — reformatting a messy voice-to-text note into readable prose, running locally so your notes never leave your laptop.
Hardware: Budget discrete GPU (RTX 3050-class), a decent iGPU, Apple M1+, or plain CPU. RAM: ~6–8 GB (CPU/iGPU share this pool; a discrete GPU takes the load off RAM instead). Cost: $150–350 for a discrete card, or nothing extra otherwise. Speed: 5–8 tokens/sec on CPU or an AMD/Intel iGPU (same bandwidth ceiling), 40–80+ on a cheap discrete GPU, 100+ on Apple Silicon. Good for: Coding help, structured writing, general chat.
The fun fact of this article: Qwen 3's 4B reportedly matches the previous generation's 72B on several benchmarks — a ~3 GB model going toe-to-toe with one that needed ~43 GB two generations ago. "Bigger is better" stopped being a safe assumption a while back.
Real-world application: this is, almost exactly, the model powering the grav-ai-chatbot project from Part III of this series — a personal website's AI search widget, answering visitor questions from the site's own content, running on a laptop that cost less than a nice dinner out.
Hardware: RTX 3060 (12 GB) or RTX 4060 — a discrete GPU is really the point here, since even a strong iGPU hits the RAM-bandwidth wall at this size. RAM: ~16 GB is a comfortable baseline; VRAM is the actual constraint (see the table). Cost: ~$330–450 for the GPU. Speed: ~40 tokens/sec on an RTX 3060, 100+ on an RTX 4090. Good for: All-around chat, coding help, general assistant duties — probably the model most people should actually run.
This is the tier where local AI stops being a novelty and starts being something you'd genuinely reach for over a cloud chatbot.
Real-world application: a solo developer's local coding assistant — living entirely in the terminal, answering questions about a private codebase, with zero risk of proprietary code getting logged on someone else's server.
Hardware: Still a single mainstream discrete GPU — 12 GB gets you there at reduced precision. RAM: ~16 GB; again, VRAM is what actually gates you — spill past it and speed falls off a cliff. Cost: ~$350–500. Speed: ~20–30 tokens/sec on a 12 GB card, 40–50 on faster ones. Good for: Meaningfully better reasoning and coding than the 8B, still comfortably consumer-grade.
Reportedly comparable to the previous generation's 32B — another rung on the "less hardware than you'd think" ladder.
Real-world application: a small company's internal knowledge-base assistant, answering employee questions by pulling from HR policies, onboarding docs, and internal wikis — sensitive enough that it staying in-house, rather than going to a cloud API, actually matters.
Hardware: A 24 GB discrete GPU (RTX 3090 or 4090). Out of reach for iGPUs at usable speed — this tier needs real VRAM throughput. RAM: ~16 GB; the ~20 GB of quantized weights is why this effectively needs a 24 GB card. Cost: ~$800 (used 3090) to $3,000+ (new 4090) — GPU prices spiked hard in 2026 due to ongoing shortages. Speed: ~20–45 tokens/sec on a 24 GB card, notably slower on anything smaller. Good for: Serious local reasoning, long-form writing, coding that used to need chunkier hardware.
Reportedly on par with the previous generation's 72B — same pattern, one more rung up.
Real-world application: a small dev team's self-hosted code review bot, reading full pull requests and catching real issues — the kind of task where a smaller model's shallower reasoning starts producing embarrassingly wrong suggestions.
Hardware: A 24 GB discrete GPU, or a Mac with 32 GB+ unified memory — this is where an M-series Mac genuinely shines. RAM: ~17+ GB VRAM on a discrete GPU (all 30B worth of weights still have to sit somewhere, even though only 3B activate per token); 32 GB is a real floor on a Mac, since unified memory plays both roles. Cost: ~$800–3,000+, same GPU tier as the 32B dense model. Speed: Often 100+ tokens/sec — the MoE trick at work. Good for: The best speed-to-quality ratio in the lineup.
It holds 30B worth of knowledge but only wakes up a 3B-sized sliver per question — feels almost as fast as a much smaller model while knowing considerably more.
Real-world application: a small startup's customer support chatbot handling real, simultaneous traffic — fast enough that visitors never notice they're talking to a self-hosted model instead of a big-name cloud API, at a fraction of the recurring cost.
Hardware: Multiple high-end discrete GPUs (2–4x 24–48 GB), or a Mac Studio with very generous unified memory. RAM: ~128–160 GB — at this tier, memory capacity and the price of the whole machine start meaning the same thing. Cost: ~$4,000 (Mac Studio) to $30,000+ (multi-GPU workstation, inflated further by 2026's GPU shortage) — the point where "just use the API" starts sounding reasonable. Speed: Single digits to ~20 tokens/sec, setup-dependent. Not a fast tier; a worth-the-wait tier. Good for: Genuinely frontier-level reasoning, math, and coding — Alibaba's answer to GPT-4o-class models.
Only 22B of its 235B parameters activate per token, same MoE trick scaled way up. Unless you already own serious hardware or need it private, renting cloud GPU time is almost always cheaper than buying into this tier.
Real-world application: a research lab or regulated enterprise (healthcare, legal, finance) that needs GPT-4o-class reasoning but can't send sensitive data to a third-party API under any circumstances — the hardware bill here is the price of compliance, not a luxury.
| Model | Active / Total | VRAM (quantized) | System RAM | Realistic Hardware | Speed (tokens/sec) | Cost | Sweet Spot |
|---|---|---|---|---|---|---|---|
| 0.6B | 0.6B | <1 GB | ~1–2 GB | Phone, Pi, old laptop | 5–10 (weak CPU) / 20–40+ (modern) | $0 | Autocomplete only — too small to trust for RAG or grounded research, easily loses the thread |
| 1.7B | 1.7B | ~1.2 GB | ~4 GB | Any laptop CPU | 15–30 | $0 | Snippets, boilerplate, short drafts; can do RAG over one small clean document, not much more |
| 4B | 4B | ~3 GB | ~6–8 GB | Budget GPU, iGPU, Apple M1+, or CPU | 5–8 (CPU/iGPU) / 40–80 (GPU) / 100+ (Apple) | $0–350 | Single functions and bug fixes; single-document RAG and source-grounded Q&A — this is the Part III chatbot's sweet spot |
| 8B | 8B | ~5 GB | ~16 GB (VRAM-bound) | RTX 3060/4060 | 40–100+ | $330–450 | Small-to-medium features; solid multi-document RAG backend, general-purpose writing assistant |
| 14B | 14B | ~8 GB | ~16 GB (VRAM-bound) | RTX 3060 12GB+ | 20–50 | $350–500 | Trickier refactors and debugging; more reliable multi-source research synthesis, longer-form writing |
| 32B (dense) | 32B | ~20 GB | ~16 GB (VRAM-bound) | RTX 3090/4090 | 20–45 | $800–3,000+ | Larger refactors, reliable agentic coding; trustworthy source-grounded research across many documents |
| 30B-A3B (MoE) | 3B / 30B | ~17 GB | ~16 GB (VRAM-bound) | RTX 3090/4090 or Mac 32GB+ | 100+ | $800–3,000+ | Tight iterative coding loops; fast enough for a high-traffic RAG chatbot without making visitors wait |
| 235B-A22B (MoE) | 22B / 235B | ~130 GB | ~128–160 GB (IS the model store) | Multi-GPU or Mac Studio | Single digits–20 | $4,000–30,000+ | Production-grade agentic coding; frontier-level research synthesis and long-form, citation-heavy writing |
Tokens/sec, not words/sec. Speed is a memory-bandwidth story throughout: system RAM bandwidth caps CPU/iGPU tiers, VRAM bandwidth caps discrete-GPU tiers, unified memory bandwidth caps Apple Silicon and the 235B tier. Figures are ballpark and vary with quantization, exact hardware, and inference backend. GPU prices have been unusually volatile through 2026 — treat every dollar figure as "the shape of the market when this was written," not a quote.
Just experimenting? Start at 4B or 8B — best value in the lineup, and it'll surprise you for its size.
Want a serious daily driver and have a decent discrete GPU? 8B or 14B covers almost everyone's real needs without turning your electricity bill into a personality trait.
Got a beefy GPU (or a well-specced Mac) and want to flex? 32B dense for raw quality, 30B-A3B if you'd rather it answer now.
Chasing frontier quality with serious hardware or a cloud budget? 235B-A22B — just go in knowing that at this tier, "self-hosted" and "cheap" stop being the same sentence.
And if none of that sounds affordable: this whole series started with a 6-Watt Celeron laptop running a 0.5B model and making it useful anyway. There's a lot of room between "nothing" and "a rig that costs more than a car," and most of it is more capable than you'd expect.
Alibaba has kept iterating on Qwen 3 since this lineup shipped — worth checking their official channels for the latest sizes and specs before buying hardware around any specific number here.
Everything above used Qwen 3 as the tour guide, and for good reason — it's rare to find one open-weight family that stretches so cleanly from "runs on a phone" to "runs on a small data center," which makes it the ideal specimen for a hardware field guide like this one. But it's far from the only option. Google's Gemma, DeepSeek, Meta's Llama, Mistral, and others all publish their own open-weight lineups, often with a similar spread of sizes and very similar hardware math — the memory-bandwidth rules in this article apply just as well to any of them. If Qwen 3's particular flavor of answers doesn't click with you, the good news is the shopping list above still mostly holds; just swap in a different name and go looking for the equivalent parameter count. Model families come and go — the physics of "your RAM has a speed limit" does not.
