v0.7.0
LLM inference in C/C++. Contribute to ggml-org/llama.cpp development by creating an account on GitHub.
llama.cpp11 min read
PyTorch Blog
Stay updated with the latest developments in Artificial Intelligence
Agents on live hardware: reason where you must, engineer everywhere else
CoreWeave Blog10 min read
b11669
llama.cpp1 min read
Extending Inductor for the IBM Spyre Accelerator
TL;DR The PyTorch-based software stack for IBM’s Spyre accelerator recently moved its torch.compile backend from sendnn, a separate compilation stack built entirely from scratch, to backend=inductor, an open-source backend built...
PyTorch Blog16 min read
Session-Aware Agentic Inference with NVIDIA Dynamo
TL;DR Agentic workloads change the traffic an inference server sees. Unlike single-turn chat, an agent session can involve a large initial prefill, repeated model calls, and subagents running in parallel,... The post Session-Aware Agentic Inference with NVIDIA Dynamo appeared first on PyTorch.
PyTorch Blog16 min read
Building Spyre as a Native PyTorch Device
TL;DR Spyre becomes a native PyTorch device by connecting PyTorch’s existing device, allocator, stream, and compiler abstractions through torch-spyre to the Spyre runtime and firmware. PrivateUse1 gives Spyre a real... The post Building Spyre as a Native PyTorch Device appeared first on PyTorch.
PyTorch Blog22 min read
Modernizing Table Batched Embeddings with FBTriton
This post explores the FBTriton kernel design for Table Batched Embedding (TBE) forward and backward passes. These core operators handle embedding lookups across thousands of sharded GPUs within recommendation systems....
PyTorch Blog10 min read
Evolution of the PyTorch Media Processing Landscape
TL;DR If you need to decode or encode media, whether it’s images, video, or audio, use TorchCodec. If you need to transform media, use TorchVision for images and video, and...
PyTorch Blog5 min read
PyTorch Hardware Enablement: Updates from the Accelerator Integration Working Group
TL;DR The Accelerator Integration Working Group plays a vital role in standardizing how new hardware architectures connect to the open source AI ecosystem. As compute platforms diversify across cloud, edge,...
PyTorch Blog13 min read
Building a High-Performance and Portable vLLM Linear Backend with Helion
TL;DR We integrated Helion into vLLM’s linear backend to explore how an autotuned, high-level kernel DSL can improve LLM inference performance while reducing kernel implementation complexity. A single Helion general...
PyTorch Blog13 min read
New Pathway to PyTorch Certified Associate (PTCA) Certification
We’re excited to bring you the new PyTorch Certified Associate (PTCA) Certification Pathway, combining focused learning modules and the PyTorch Certified Associate (PTCA) certification exam together in one structured experience...
PyTorch Blog2 min read
Optimizing Jagged Flash Attention with TLX: The Road Toward SOTA FA4 on Blackwell
TL;DR In this blog post, we present our work on Jagged Flash Attention (JFA) — the attention kernel behind Meta’s Generative Ads Model (GEM) — on NVIDIA Blackwell (B200), built...
PyTorch Blog16 min read
A Ray-Focused Guide to PyTorch Conference North America
TL;DR In only a few weeks, PyTorch Conference North America 2026 will begin in San Jose, bringing together the open source AI community to share ideas and collaborate. If you’ve...
PyTorch Blog7 min read
From Upstream Changes to Downstream Confidence: Inside Torch Spyre’s Integration with PyTorch CRCR
TL;DR PyTorch’s Cross-Repository CI Relay (CRCR) gives out-of-tree accelerators a clean, scalable way to plug into upstream CI – and it leaves each backend free to decide which of PyTorch’s...
PyTorch Blog15 min read
Accelerate Your AI Journey with new Introduction Track at PyTorch Conference NA 2026 and PyTorch Associate Training
As deep learning models move rapidly from research prototypes into core enterprise infrastructure, the demand for practical, end-to-end PyTorch expertise has never been higher. Building robust neural networks requires more...
PyTorch Blog5 min readFrom Research Project to Open Source Ecosystem: Bring Your Academic PyTorch Project to PyTorchCon NA
Some of the most interesting work being built with PyTorch starts in universities, research labs, student groups, and academic institutions. A new model architecture. A library created to support a...
PyTorch Blog6 min read
Hardware-Agnostic Models in vLLM
TL;DR To achieve state-of-the-art performance at the frontier, vLLM is changing its internal implementation in ways that make it incompatible with fullgraph torch.compile. This may have consequences for users who...
PyTorch Blog9 min read
How Shopify built a continual learning loop with PyTorch and vLLM
TL;DR: This case study explores how Shopify compresses production failures into model weights every day, beats frontier-model quality, and cuts serving costs 96% by building a continual learning loop with...
PyTorch Blog10 min read
TinyTorch: Don’t Just Import PyTorch. Build It.
A framework you write yourself, tensors through transformers TL;DR Every mature systems project eventually needs a teaching version. TinyTorch is a free, open-source curriculum where you build a working ML...
PyTorch Blog12 min read
PyTorch Day Japan 2026 Comes to Tokyo on December 10
PyTorch Day Japan 2026 will bring the open source AI community together in Tokyo on December 10 for a full day of technical talks and interactive discussions designed to foster...
PyTorch Blog2 min read
Open Research, Tooling & Optimization at PyTorch Conference North America 2026
TL;DR Taking place October 20 to 21 in San Jose, California, PyTorch Conference North America 2026 highlights open research, tooling, and performance optimization across compiler architecture, cross-hardware kernel domain-specific languages,...
PyTorch Blog20 min read
Showing 18 of 21 stories
Load more storiesAiByts Weekly Brief
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news from PyTorch Blog
Join 15,000+ AI enthusiasts