The default assumption in AI is 'bigger is better': use the largest model available for the best quality. This assumption is wrong for most production use cases. A 7B parameter model running at 50 tokens/second is more useful than a 70B model running at 5 tokens/second — for tasks that do not require the larger model's capabilities. The key is task-specific model selection: match the model size to the task complexity.
What Small Models Do Well
Small models (1B-7B parameters) excel at: classification (sentiment analysis, topic categorisation, intent detection), extraction (named entity recognition, key phrase extraction, structured data extraction), simple summarisation (condensing short documents), translation (between common language pairs), and formatting (converting between data formats). For these tasks, small models achieve 90-95% of large model quality at 10-20% of the cost and 5-10x the speed.
What Large Models Do Well
Large models (70B+ parameters) excel at: complex reasoning (multi-step logic, mathematical proofs), nuanced generation (creative writing, persuasive content), code generation (complex codebases, architectural decisions), and multi-domain knowledge (questions that span multiple fields). For these tasks, large models achieve significantly better quality than small models — the gap is 20-40%, not 5-10%.
Pro Tip
The quality gap between small and large models is task-dependent: 5-10% for simple tasks, 20-40% for complex tasks. Match the model to the task.
The Selection Framework
Use this framework: (1) Start with a small model for all tasks. (2) Measure quality on your evaluation dataset. (3) If quality is above the threshold, keep the small model. (4) If quality is below the threshold, try a medium model (13B-30B). (5) If quality is still below the threshold, use a large model (70B+). Most production tasks (60-70%) can be handled by small or medium models. Only 30-40% of tasks require large models. The cost savings from this approach are dramatic.
Conclusion
Bigger models are not always better. Small models handle 60-70% of production tasks at 10-20% of the cost. Use the selection framework: start small, upgrade only when quality requires it. The cost savings are dramatic.
Key Takeaways
- Small models (1B-7B) excel at: classification, extraction, simple summarisation, translation, formatting
- Large models (70B+) excel at: complex reasoning, nuanced generation, code generation, multi-domain
- Quality gap: 5-10% for simple tasks, 20-40% for complex tasks
- Start with small models for all tasks; upgrade only when quality requires it
- 60-70% of production tasks can be handled by small or medium models