Tool Use, Unified

Source: HuggingFace Date: 2024-08-12 URL: https://huggingface.co/blog/unified-tool-use

Summary

Library update: Transformers adds unified tool-use support via the chat templating system. Python functions passed directly to apply_chat_template are auto-converted to JSON schemas from type hints and docstrings. Tool calls standardized as tool_calls field in assistant messages; responses use role: "tool". Supports Mistral, Llama 3.1, Command-R+, Hermes-2-Pro, and others. Caveat: output parsing remains model-specific (JSON/regex) — input is unified, output isn’t yet.

Implications

Thread: transformers library trajectory / agentic patterns. Unified tool-use input is an important step: it means the same Python function definition works across model families without per-model prompt engineering. The remaining gap — parsing model-specific output formats — is the harder problem, but having a standard input representation is the prerequisite for standardizing output handling. The auto-schema-from-docstring approach is the right ergonomic choice: it keeps tool definitions as regular Python and removes the JSON-schema boilerplate that previously made tool use tedious to maintain across model updates.