llm-d is an open-source, Kubernetes-native framework for serving large language models at scale. It doesn’t run models itself. It sits in front of inference engines such as vLLM, another open-source, high-throughput serving engine, and decides how…
Self-Hosting LLMs on Kubernetes: Securing llm-d and vLLM with Kong AI Gateway (Part 1)
calendar_today
November 5, 2026
domain
kong