Small language models: what a 9-billion-parameter model can and cannot do
Independent scores for Alibaba's new sub-10B Qwen3.5 models show real gains and a clear weakness: small models still make things up.
A small language model (SLM) is a model compact enough to run on a laptop, a phone or a single modest server rather than a rack of accelerators. The attraction for businesses is practical: data stays on your hardware, there is no per-token bill, and latency is predictable. The question is always what you give up.
This week gives a useful data point. Artificial Analysis published an independent evaluation of Alibaba's four new Qwen3.5 small models, all dense, open under Apache 2.0, with a 262K-token context window and native image input [1].
Intelligence Index score, Qwen3.5 small models
Reasoning variants, Artificial Analysis evaluation
View data
| Intelligence Index | |
|---|---|
| Qwen3.5 0.8B | 9 |
| Qwen3.5 2B | 16 |
| Qwen3.5 4B | 27 |
| Qwen3.5 9B | 32 |
| Qwen3 4B 2507 (prior gen) | 18 |
What improved
Artificial Analysis reports the 4B model scoring 27 on its Intelligence Index, against 18 for the previous-generation Qwen3 4B 2507, and the 9B scoring 32 [1]. On a multimodal reasoning test (MMMU-Pro), the 9B reaches 69.2% [1]. Estimated 4-bit memory needs are roughly 6GB for the 9B, 3GB for the 4B and under 2GB for the two smallest [1], which puts them within reach of ordinary laptops.
What did not
Three caveats matter more than the headline score.
Hallucination. On the AA-Omniscience test, the 9B hallucinates 82% of the time and the 4B 80% [1]. Artificial Analysis notes that the improvement over Qwen3 on this test is driven mainly by lower hallucination rates rather than higher accuracy [1]. Without grounding, these models should not be a source of facts.
Verbosity. All four models used more than 230 million output tokens to run the Intelligence Index [1]. Reasoning-style small models can be slow in practice.
Deployment gaps. At publication there were no first-party quantized releases for the small models and no serverless APIs, first- or third-party [1].
Where small models fit
The profile favours narrow, supervised jobs: classifying tickets, extracting fields from documents you retrieve for them, routing requests, or drafting text a person reviews. It disfavours open-ended question answering from memory. Pairing a small model with retrieval from your own documents addresses the hallucination weakness directly.
For most businesses, the practical step is to pick one well-bounded task, run a small model against a labelled sample, and compare its error rate and cost with a larger hosted model before deciding where data should be processed.
Sources
- Qwen3.5 small models: independent analysis Artificial Analysis, 2026-03-05. artificialanalysis.ai/articles/qwen3-5-small-models
.jpg){kind=link}