Serving Kimi K3 on vLLM was hard. Here is what we measured.
No released vLLM build served Kimi K3 when we started. We resolved several distinct failures on 8x B300 and measured the result, its operating crossover, and its honest limits.
RunInfra Blog10 min read
