Falcon-Edge: A series of powerful, universal, fine-tunable 1.58bit language models.

Source: HuggingFace Date: 2025-05-15 URL: https://huggingface.co/blog/tiiuae/falcon-edge

Summary

Model release + library: TII UAE releases Falcon-Edge 1B and 3B in native BitNet (ternary weights: {-1, 0, 1}), pre-quantized, and bfloat16 formats, with a companion onebitllms Python package for fine-tuning and quantization. A single training run (~1.5T tokens) produces both quantized and non-quantized variants via weight scale injection. Benchmarks show Falcon-Edge competitive with same-size models on HF Leaderboard v2; minimal degradation converting between BitNet and bfloat16.

Implications

Open-weights ecosystem health. Universal fine-tunable BitNet models (both base and instruction-tuned, all formats from one training run) resolve a key objection to 1-bit quantization: that you can’t fine-tune the quantized model. The onebitllms TRL integration makes this accessible to the standard HF fine-tuning workflow. If the accuracy cost of ternary weights remains small at 1B-3B scale, edge deployment use cases become substantially more viable.

Model release cadence — efficiency thread. TII UAE combining model release with a purpose-built fine-tuning package signals that the 1-bit/ternary quantization approach is mature enough for production fine-tuning workflows, not just inference-only deployment. Watch for follow-up releases as Falcon-Edge fine-tunes appear in the community.