Profiling in PyTorch (Part 1): A Beginner’s Guide to torch.profiler

Source: HuggingFace Date: 2026-05-29 URL: https://huggingface.co/blog/torch-profiler

Summary

HuggingFace’s introductory guide to torch.profiler walks through CPU/CUDA profiling of a matmul+bias workload, producing both a statistical table and a Perfetto-compatible Chrome trace. Key findings: small matrices are overhead-bound (dispatcher cost dominates), large matrices are compute-bound (GPU is the bottleneck), and torch.compile fuses ops at the dispatcher level but increases CPU overhead enough that it doesn’t pay off on small isolated kernels.

Implications

• Feeds the model layer/open-weight ecosystem thread: practical profiling tooling lowers the barrier for teams fine-tuning or running inference on open-weight models on constrained hardware.
• The overhead-vs-compute-bound framing is directly applicable to fleet-ops hardening—understanding where dispatch overhead dominates versus GPU saturation informs batching strategy and hardware provisioning decisions.