I keep watching developers reach for the newest frontier model when a smaller, specialized one would do the job better, cheaper and with more control. A recent test by a practicing dermatologist put that instinct under a very bright light, and I want to share it with you.
A 2015 CNN walks into a dermatology clinic
On October 25, 2026, Dr Magnus Lynch of Lynch Lab published a test that stopped me in my tracks. He is a Consultant Dermatologist at Guy’s Hospital, and he gave the same 200 biopsy-proven lesions from the public MILK10k dataset to three systems: GPT-6 Astra, Claude Fable 5.1, and a fine-tuned Inception v3. That CNN architecture dates to 2015. He fine-tuned it for about twenty minutes on public dermatology images, carefully holding out every photograph of the test lesions.
The scoreboard, all from Lynch’s own write-up:
- CNN first-choice diagnosis: 125 out of 200 (62%)
- GPT-6 Astra: 84 (42%)
- Claude Fable 5.1: 85 (42%)
Lynch is careful about the caveats, and so am I. Two hundred lesions is a small sample, and every case was hard enough that someone had already decided to biopsy it. The CNN is deliberately not state of the art. MILK10k has been public since 2025, so the frontier models might have seen related images in training, which would only have helped them. Even with all of that, a tiny old network named the right diagnosis more often than two of the strongest general models of late 2026.
This pattern shows up again and again
Lynch’s result is vivid, and it sits in a growing pile of careful comparisons where a fine-tuned specialist beats a prompted giant on a fixed labeling task.
Czech news trustworthiness. In a TrustNLP 2024 paper, Matyas Bohacek and Michal Bravansky compared an Electra plus XGBoost ensemble to GPT-4 and Llama-2-70B on classifying Czech news trustworthiness. The small ensemble reached a micro-F1 of 0.533. GPT-4 scored 0.531 when prompted in Czech and 0.425 in English. The ensemble needed about 0.9 GB of virtual memory. The authors estimate GPT-4 at about 3,370 GB.
Social science labels. Martin Bucher and Marco Martini fine-tuned RoBERTa, DeBERTa and XLNet models and compared them with zero-shot GPT-4 and Claude Opus across stance and emotion tasks. On tweets about the Kavanaugh nomination, DeBERTa reached accuracy of 0.94 while GPT-4 scored 0.58 and Claude Opus 0.61. On German political texts labeled for anger, XLNet reached 0.89 while GPT-4 scored 0.20.
Medical imaging. Ahmed, Sakib and Das compared task CNNs with GPT-4o and Llama3.2-vision on public X-ray, MRI and CT datasets. Accuracy for the CNNs against GPT-4o came out at 0.83 versus 0.62 on chest X-rays, 0.98 versus 0.60 on brain MRI, and 0.91 versus 0.22 on chest CT.
I read these papers as a reminder that when the output space is a fixed set of labels, a model trained on those labels has a real home-field advantage.
An honest counter-point
Frontier models can still win when the task looks more like open-ended reasoning than like picking from a short label set. In the Medprompt study, GPT-4 with careful prompting reached 90.2% on MedQA (USMLE-style questions), ahead of Med-PaLM 2 at 86.5%. That is a specialist medical exam framed as multiple choice with rich language, and a generalist model with inventive prompting took the lead. I want you to hold both ideas at once: narrow models often win on fixed labels, and frontier models shine when the problem asks for flexible reasoning.
A simple decision rule
Here is how I decide in my own projects.
If I have a fixed set of labels and a few hundred labeled examples, I try a narrow model first. Fine-tune a CNN, a RoBERTa-style encoder or a small specialist, measure it on a held-out test set, and only reach for a frontier API if the narrow model falls short.
If the task is open-ended, keeps changing shape, or needs long chains of reasoning across tools and documents, I start with a frontier model and keep a specialist nearby for the parts that look like classification.
That rule has saved me money, latency and a lot of quiet evenings spent staring at API bills.
Keep practicing the craft
The skill of building a narrow model is still one of the most valuable things a developer can practice. You learn how to choose an architecture, how to split data without leaking test cases, how to read a confusion matrix, and how to ship something that runs on a laptop or a phone. Lynch’s CNN is small enough to run in a phone browser with WebGPU. That kind of craft does not go out of date when the next frontier model ships.
If you want a place to start, or a place to sharpen those skills, I wrote a hands-on book on building with PyTorch. You can find it here: AI and Machine Learning for Coders with PyTorch. Work through a few chapters, fine-tune something small on a problem you care about, and keep that muscle strong. The frontier models will keep getting better. The people who know when to use a specialist, and how to build one, will keep getting better too.
Read the sources
- Lynch Lab: A 10-year-old CNN outperforms frontier LLMs for skin cancer diagnosis
- Bohacek and Bravansky (TrustNLP 2024): When XGBoost Outperforms GPT-4 on Text Classification
- Bucher and Martini: Fine-Tuned ‘Small’ LLMs (Still) Significantly Outperform Zero-Shot Generative AI Models in Text Classification
- Ahmed, Sakib and Das: Can Large Language Models Challenge CNNs in Medical Image Analysis?
- Nori et al. (Medprompt): Can Generalist Foundation Models Outcompete Special-Purpose Tuning?