Benchmarking LLM Inference at Scale with AIPerf
You’re deploying a model on a system. It starts up, prompts are getting responses. Now the hard question: Is this fast? Your instincts might lead you to send...
NVIDIA Developer Blog8 min read
Agentic AI / Generative AI
29 stories
How to Use AI Agents to Prepare 3D Scenes for Simulation
Agentic AI workflows can be used to prepare and validate digital twins for physical AI systems. Agents can inspect 3D scenes, author simulation-relevant data in...
NVIDIA Developer Blog14 min readTensorRT Edge-LLM Completes the MLPerf Edge Agentic Benchmark 6.4x Faster on Jetson AGX Thor
AI agents are moving from cloud data centers to vehicles, robots, and other edge devices. Unlike a chatbot that answers a single prompt, an agent works through...
NVIDIA Developer Blog6 min readTranslating CUDA Tile Operations from Python to Rust Using Agentic AI
cuTile Rust (cutile-rs) is a tile-based system for safe, idiomatic GPU kernel authoring in the Rust programming language. Extending the Rust ownership model to...
NVIDIA Developer Blog17 min readDense vs. MoE Models: Active Parameters, Throughput, and When to Choose Each
How can a 30B-parameter model activate only 3B parameters per token, and still use the capacity of the larger model? Nemotron 3.5 Lightning illustrates the...
NVIDIA Developer Blog7 min readHow NVIDIA NVLink 6 Delivers Multi-Layer Resiliency for AI Factories
For operators of large-scale AI factories, maximizing continuous output is essential for productivity. In massive-scale AI training, every GPU in the cluster...
NVIDIA Developer Blog9 min readHow NVIDIA Groq 3 LPX Deterministic Execution Drives Power-Efficient High-Interactivity Inference on NVIDIA Vera Rubin
Power is a defining constraint for AI factories. As AI workloads demand a full compute platform to serve them, each component of that platform must maximize...
NVIDIA Developer Blog7 min readAccelerating Dropless MoE Training in JAX with NVIDIA Transformer Engine
Mixture of experts (MoE) has become one of the defining architectural trends in large-scale AI model training. DeepSeek, Qwen, and Mixtral are examples of MoE...
NVIDIA Developer Blog11 min readHow Full-Stack NIM Optimizations Deliver 2.5x More Users on Nemotron 3 Ultra
Deploying a large language model is only the first step toward production-ready serving. Production teams also need to serve as many concurrent users as...
NVIDIA Developer Blog6 min readFrom Wafer-Out to First Token: Codifying Supply Chain Expertise with Nemotron and Palantir Foundry
NVIDIA has one of the largest and most complex supply chains in the world, and its performance is measured from wafer-out to first token. The interval is in two...
NVIDIA Developer Blog10 min readWhen to Use Encode-Prefill-Decode Disaggregation to Accelerate Multimodal Model Serving
Encode-prefill-decode (EPD) disaggregation is an inference optimization technique for multimodal models that separates the vision encoder stage from the prefill...
NVIDIA Developer Blog10 min read
Showing 29 of 29 stories
You're all caught up.
AiByts Weekly Brief
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts