Investing in Performance: Fine-tune small models with LLM insights - a CFM case study

Source: HuggingFace Date: 2024-12-03 URL: https://huggingface.co/blog/cfm-case-study

Summary

Integration tutorial + case study: Capital Fund Management (CFM) demonstrates LLM-assisted labeling followed by compact model fine-tuning for financial NER (company name extraction from 900k+ news headlines). Pipeline: Llama 3.1-70B via HF Inference Endpoints labels data at ~$70 total; Argilla handles human review (2,714 samples curated from 900k); GLiNER fine-tuned from 87.0% → 93.4% F1 at $0.50/hour (80x cheaper than Llama 70B). SpanMarker similarly jumps from 47.0% → 90.1%.

Implications

Open-weights ecosystem health. The 80x cost reduction at comparable accuracy (93.4% F1 fine-tuned GLiNER vs. 95.0% Llama 70B zero-shot) is a concrete ROI argument for the “distill then fine-tune small” pattern. This case study makes the pattern tangible for financial services teams evaluating open-weights vs. API spend.

HF as open-source ML hub. The full stack used here — Inference Endpoints (labeling), Argilla (annotation), HF model fine-tuning, deployment — is entirely within the HF ecosystem. CFM publishing this case study is HF’s enterprise validation story: real financial firm, real cost savings, end-to-end on HF tooling.