Know a great AI tool or AI news source? Submit it to AiByts
language-model5 min read

Buoyant Releases Kinference: An Open Source Inference Control Plane for Kubernetes and vLLM

Buoyant has released Kinference, an Apache 2.0 inference control plane for vLLM on Kubernetes. It adds declarative routing policies, KV-cache-aware load balancing and per-tenant metrics, and cut median TTFT 95% in Buoyant’s own benchmark.

By Michal SutterMarkTechPost
Buoyant Releases Kinference: An Open Source Inference Control Plane for Kubernetes and vLLM

Read the full story

Originally published by MarkTechPost on Nov 7, 2026.

Read on MarkTechPost(opens in a new tab)

Comments

Sign in to join the discussion.

No comments yet — be the first to share your thoughts.