Observing LLM Applications with OpenTelemetry

Ever since OpenAI launched ChatGPT in November 2022, AI usage has exploded worldwide, with LLM integration rapidly becoming a competitive requirement rather than an experimental feature. Integrating large language models into applications presents unique observability challenges due to their non-deterministic nature, where identical inputs can produce vastly different outputs across runs. This article explores why observing LLM-based applications is critical, explains what OpenTelemetry is, and demonstrates practical integration through a working demo using FastAPI and the OpenAI Agents SDK.