28 stories
LFM2.5-24B-A2B is a 24B-parameter reasoning model with just 2B active parameters, built for fast, private agents on-device. It thinks before it answers, calls tools, and works through multi-step tasks inside popular agent harnesses.
Learn how to reduce LLM inference costs and latency using quantization, vLLM, SGLang, and speculative decoding without upgrading your hardware.
Discover how to boost your LLM inference performance and customize responses using SGLang, an innovative framework for structured LLM workflows.
As you can see in the AINews X recap section below, everyone on earth has cloned the Jev API, but only one company can ever create the category. TypeSafe announced their “Series AI” and Sequoia “leaked” that they crossed 100M ARR in their first week.
We are expanding the Clef decision model family with Clef-omni, natively processing audio, video, images, and text in a single pipeline. We’ve also lowered Clef-flash pricing and boosted Clef inference speeds by up to 2.0x.
Hermes "response truncated" or "remained truncated after 4 continuation attempts"? 5 causes and fixes for v0.21.5, plus the Pi agent truncation error.
AIE NYC leadership tickets will sell out tomorrow, while for SF folks, AIE CODE applications are still open for the top agentic engineers in the world.
TL;DR Agentic workloads change the traffic an inference server sees. Unlike single-turn chat, an agent session can involve a large initial prefill, repeated model calls, and subagents running in parallel,... The post Session-Aware Agentic Inference with NVIDIA Dynamo appeared first on PyTorch.
IQuest-Q1 is a 320B MoE coding model with 15B active params and 512K context. Here's what it takes to run it with SGLang or vLLM.
A Context Language Model is an existing model that rewrites its own context like a file. It manages working context inside a task better than summarisation, with less compute, and throws that context away when the task ends.
Try Zapier: https://bit.ly/4yVvOtV Zapier helps you build custom automation and we're looking at how Zapier CLI can help me build news scanner app that keeps me up to date with AI.
Runpod's serverless platform keeps evolving, especially for LLM workloads. Learn what's new in 2025 and how to get the most from scalable deployments.
Learn how to scale Mixture-of-Experts (MoE) reinforcement learning on Amazon EKS using Elastic Fabric Adapter (EFA) and DeepEP. This post presents an architecture that combines Amazon EKS, EFA, and Amazon S3 and increased aggregate reinforcement learning rollout throughput by 40% for large-scale…
Faster inference: decode speedups up to 3.13x on device and 2.66x on an H100, with end-to-end gains up to 2.62x and 2.27x.
An AI coding agent’s patch can pass tests yet fail when the server loads a real model and handles requests. Evaluating changes to inference-serving software...
When comparing Chinese AI vs ChatGPT, your decision comes down to cost versus security. If you need low API pricing, open weights, and fast coding assistance, Chinese AI models like DeepSeek lead the way. However, if your team...
PyTorch Conference China 2026 brought the PyTorch community together in Shanghai on September 8–9 alongside KubeCon + CloudNativeCon and OpenInfra Summit, following sponsor-hosted co-located events on September 7. Technical discussions...
TL;DR: Granite 4.2 is our first family of dense, decoder-only reasoning LLMs, released in three sizes: 3B, 8B, and 30B. These models are post-trained from Granite-4.1 base models.
Showing 18 of 28 stories
Load more storiesGet the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts
