Picture This: Open Source AI for Image Description

Source: fly.io Date: 2025-02-12 URL: https://fly.io/blog/llm-image-description/

Summary

Engineering narrative by Nolan Darilek documenting an open-source image description service for blind users built with Ollama (LLaVA model), PocketBase (SQLite-based backend), and a Python client. The system supports follow-up questions via event hooks and runs on Fly.io GPU machines with autostop for cost control. The author is blind and describes the capability as transformative: LLM-generated image descriptions are now comparable to human-written alt text, and the stack is easy enough to self-host.

Implications

Edge deployment economics / local-first architecture. This signal is notable for two reasons: the accessibility use case demonstrates LLM capability in a domain (visual accessibility) that had previously required human labor at significant cost; and the stack (Ollama + PocketBase + Fly GPU autostop) is a real-world proof that low-overhead self-hosted inference is viable for niche, high-value applications. PocketBase as a Firebase alternative (SQLite-backed, single-binary) is the local-first pattern in action. The autostop GPU feature — pay nothing when no requests arrive — is the economic model that makes this sustainable at hobbyist scale. Watch whether this stack pattern (lightweight model + SQLite backend + autostop GPU) becomes a common template.