SmolLM3: smol, multilingual, long-context reasoner

Source: HuggingFace Date: 2025-07-08 URL: https://huggingface.co/blog/smollm3

Summary

Model release from HF: SmolLM3-3B, a multilingual long-context (128K) reasoner trained on 11.2T tokens on 384 H100s for 24 days. Outperforms Llama-3.2-3B and Qwen2.5-3B; competitive with 4B models. Dual-mode: /think and /no_think system prompts. Reasoning enabled: AIME 2025 36.7% (vs 9.3% base), LiveCodeBench 30.0% (vs 15.2%). Supports 6 European languages, XML + Python tool calling. Full training recipe, datasets, and W&B logs released.

Implications

Thread: open-weights ecosystem health / transformers library trajectory. SmolLM3 is the most fully open model release in this batch: not just weights but the complete engineering blueprint — exact data mixtures, ablation results, synthetic data generation, all training configs. At 3B parameters with 128K context and dual-mode reasoning, it’s positioned as the practical small model for on-device or low-cost agentic deployment. The /think flag as a system prompt token (vs separate model) is elegant: single model, configurable reasoning depth. Watch whether the training recipe gets reproduced by the community to validate the data mixture claims.