I keep coming back to the same practical question: which model can I actually run where I already work? For a lot of us that means a phone, a Raspberry Pi on the bench, or a laptop with one decent GPU. The good news is that the open-weight options in those places keep getting better, and a couple of recent releases make the single-GPU desk feel a lot more capable than it did even a few weeks ago.
Start with what fits in your hand
If the goal is something you can teach, ship, and run with the radio off, I still begin with Gemma 4. Google DeepMind released it on April 2, 2026 under Apache 2.0, which matters if you care about putting weights in a student repo. The family covers a useful range: E2B and E4B for phones and small devices, a 26B MoE with about 3.8B active parameters, a 31B dense model, and the June 12B Unified multimodal SKU.
Google also published a clear LiteRT walkthrough for Gemma on Raspberry Pi. On a Pi 5, Gemma 4 E2B lands around 99 tokens per second in prefill, 9 tokens per second in decode, and roughly 1432 MB of peak memory. Those numbers are concrete enough that you can sit down and try to reproduce them. Once you have felt a small model answer a question with the network unplugged, the rest of the architecture gets easier to explain.
Then look at what fits on one GPU
Two Apache 2.0 releases in mid-August pushed the local-agent ceiling up for people with a 24 to 32 GB card.
Meta’s Muse Glimmer (August 10) is about a 30B dense model built for tool use, recovering from failures, and working with screenshots. At 4-bit it is aimed at fitting under about 20 GB, which is exactly the envelope a lot of hobby and lab machines already have.
Alibaba’s Qwen team followed on August 14 with Qwen3.8-27B: a 27B dense multimodal model with 262,144 tokens of native context, room to stretch toward a million with YaRN, and a thinking mode you can turn on or off. There is a much larger Max-class MoE in the same family. That one belongs on different hardware. The 27B dense model is the one that belongs in the same conversation as Glimmer.
I would treat both as early but exciting. Vendor cards move fast. Independent bake-offs are still catching up. Download them, point them at a tool-using prompt you care about, and see where they break.
Keep a small bench for the true SLMs
Under about 12 billion parameters, my shortlist has stayed pretty stable, and that stability is helpful when you are teaching. Qwen3.5’s dense models from 0.8B to 9B are strong multilingual companions to Gemma’s E2B and E4B. Phi-4-mini at about 3.8B (MIT, 128K context) is still the one I load first when VRAM is ugly and the task is math or short reasoning. SmolLM3 at 3B remains my favorite when I want students to see a full public recipe, not only a checkpoint.
You do not need a new hero model in this tier every week. You need a few you trust, with licenses you can live with, and enough documentation that a weekend project does not turn into archaeology.
Read the product name slowly
Labels can wander. Mistral Small 4 (March 16, 2026) is a 119B MoE with roughly 6B active parameters and a 256K window. In practice you are looking at multi-GPU infra, often on the order of four H100s. Llama 4 Scout and Maverick keep about 17B active parameters with 109B and 400B totals. They are open-weight under the Llama Community License, with EU and large-MAU limits that are different from Apache 2.0 or MIT. Sparse activation is clever engineering. It still does not make those models phone-class.
When someone says “small,” ask them about active parameters, total parameters, license, and the machine they actually ran it on. That short checklist saves a lot of confusion.
A simple pattern that still works
The setup I keep recommending is straightforward. Run a multimodal reasoner locally as the default. Send the hard, tool-heavy, or safety-sensitive work to a larger cloud model when you need to. Muse Glimmer and Qwen3.8-27B make that local default much stronger on a single consumer GPU. Gemma 4 E2B and E4B still carry the phone and Pi story. The cloud remains there for the long tail.
If I were starting a project this week, I would load Gemma 4 E2B or E4B with LiteRT for anything that has to live next to the camera roll. I would keep Qwen3.5, Phi-4-mini, and SmolLM3 nearby for teaching and tight memory. And I would put Muse Glimmer and Qwen3.8-27B on the shortlist for agents that get to use the desktop GPU.
Time to dig deeper
Three good starting points are Meta’s Muse Glimmer post, Google’s LiteRT and Gemma on Raspberry Pi guide, and the writeup of the Qwen3.8 open-weight release. Grab one phone-class checkpoint and one 27 to 30B agent checkpoint. Run the same prompt three ways: on the device, on your GPU, and in the cloud. Keep notes on where each path fails. That afternoon of tinkering will teach you more than another round of launch-day screenshots.