Even (very) noisy LLM evaluators are useful for improving AI agents
Noisy evaluators have limited value for production decisions that hinge on judging a single output (e.g. guardrails).
TensorZero Blog14 min read
TensorZero
12 stories
Stop comparing price per million tokens: the hidden LLM API costs
Most engineers know they need to evaluate models on their specific task because performance varies. But so does cost: the same input can cost several times more on one provider than another, even when their list prices look similar.
TensorZero Blog3 min readWe're building an automated AI engineer, and it works
We’re building an automated AI engineer that dramatically improves performance of LLM agents on every single benchmark we’ve tried.
TensorZero Blog11 min readBandits in your LLM Gateway: Improve LLM Applications Faster with Adaptive Experimentation (A/B Testing)
Developing LLM applications involves numerous design decisions: which model to use, how to structure prompts, what temperature and sampling parameters to set, etc.
TensorZero Blog19 min readIs OpenAI's Reinforcement Fine-Tuning (RFT) Worth It?
We benchmarked OpenAI’s supervised fine-tuning (SFT) and reinforcement fine-tuning (RFT) workflows on three tasks:Data Extraction: RFT improved performance with limited data, though SFT on a larger dataset achieved better results at a still much lower cost.Agentic Coding: RFT improved performance…
TensorZero Blog54 min readTensorZero Raises $7.3M Seed Round to Build an Open-Source Stack for Industrial-Grade LLM Applications
We’re excited to announce a $7.3M seed round to support the development of 11.5KTensorZero, an open-source stack for industrial-grade LLM applications.
TensorZero Blog4 min readDistillation with Programmatic Data Curation: Smarter LLMs, 5-30x Cheaper Inference
Every LLM application builder faces a fundamental tradeoff: large models deliver superior performance at higher costs, while small models are affordable but often underperform. This choice becomes critical when scaling to production workloads where inference costs can quickly explode.
TensorZero Blog38 min readAutomatically Evaluating AI Coding Assistants with Each Git Commit
AI coding assistants are performing an ever-increasing share of software engineering work. Generic LLM coding benchmarks fail to capture the effectiveness of a coding assistant for specific use cases and workflows.
TensorZero Blog11 min readReverse Engineering Cursor's LLM Client
We wired 11.5KTensorZero between Cursor and the LLMs to see every token fly by… and bend those API calls to our own will.
TensorZero Blog10 min readFrom NER to Agents: Does Automated Prompt Engineering Scale to Complex Tasks?
Crafting good system prompts is one of the trickiest parts of working with language models — especially when you’re dealing with complex, multi-step tasks. Doing it manually doesn’t scale, and the results are often hit-or-miss.
TensorZero Blog125 min readCase Study: Automating Code Changelogs at a Large Bank with LLMs
One of the largest European banks used 11.5KTensorZero to build an LLM-powered changelog generator for GitLab merge requests. They optimized model quality, deployed fully on-premise, and iterated quickly — without needing a team of ML engineers.
TensorZero Blog3 min readThink of LLM Applications as POMDPs — Not Agents
LLM engineering is a young field. Practitioners suffer from a lack of clarity about their problem space, resulting in tools and applications that are hard to reason about, manage, and iterate on — slop.
TensorZero Blog13 min read
Showing 12 of 12 stories
You're all caught up.
AiByts Weekly Brief
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts