RunInfra is now inference for agents
RunInfra now focuses on hosted inference for agents, with pay-as-you-go Model APIs and setup guides for Claude Code, Codex and other coding agents.
RunInfra Blog1 min read
RunInfra
7 stories
With software alone, one B200 beats the LPU and gets close to Cerebras
Stock SGLang served gpt-oss-120b at 411 tokens per second on one rented B200. Four settings took the same card and the same weights to 1,366, and single prompts to 2,215. No kernel written, nothing recompiled.
RunInfra Blog17 min readLossless Inference
How to make LLM serving faster without touching the model. Exact kernels, speculative decoding, lossless compression, KV reuse and scheduling, with the math and how to verify it with logit parity.
RunInfra Blog8 min readThe fastest way to serve DeepSeek V4 Flash
Historical measurements published August 2, 2026 for DeepSeek V4 Flash 0731 on four B200 GPUs. As of September 21, 2026, no current measured optimization package is published for it.
RunInfra Blog8 min read$0.09 and $290.12: What Actually Moves Your Inference Bill
Nine cents and $290.12 are both prices for one million output tokens. The gap is driven by accelerator, provider, concurrency, utilization, and reasoning token volume.
RunInfra Blog10 min readServing Kimi K3 on vLLM was hard. Here is what we measured.
No released vLLM build served Kimi K3 when we started. We resolved several distinct failures on 8x B300 and measured the result, its operating crossover, and its honest limits.
RunInfra Blog10 min readvLLM vs SGLang vs TensorRT-LLM: a serving benchmark
Self-reported Llama-3.1-8B results for three LLM inference engines. Published unique-prompt throughput and latency rows cover H100 bf16. FP8 and L40S appear only in derived cost results. The harness and raw data are not publicly available.
RunInfra Blog9 min read
Showing 7 of 7 stories
You're all caught up.
AiByts Weekly Brief
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts