Your LLM Judge is Partly Grading by Position — and a Bigger Model Doesn’t Fix It
I checked LLM-as-judge for order bias with no labels.Continue reading on Towards AI »
Towards AI1 min read
LLMOps
8 stories
Jev vs. LLM-as-a-Judge for AI Evals
A model that answers yes/no questions instead of writing text came out 3.5× cheaper and 3.8× faster than our gpt-4o-mini judge on 1,000 real production turns, and agreed with it on 897 of them.
Opik Blog1 min readCapturing a 400-Turn Claude Code Session in a Single Image
We ran a small model over thousands of Claude Code session and discovered how people really use coding agents. Comet Cost Intelligence is a tool for optimizing engineering token spend on Claude Code and Codex.
Opik Blog1 min readPrompt Management for AI Agents: The Best Tools Compared (2026)
Despite how often prompts change once an agent is live, most teams don’t have a real prompt management process. Somewhere on a shared drive is prompt_final_v2.txt, and somewhere else is a version someone pasted into a Slack DM that is two edits newer.
Opik Blog1 min readUX Without a UI: Notes from Researching Terminal UX for Opik MCP
The way developers interact with products is changing, and opening your app’s dashboard is no longer the obvious entry point. They want to use their apps inside the terminal. Why? Because they’re already there. We’re seeing more users instrumenting MCP and it makes sense.
Opik Blog1 min read
What is AI Observability? A Complete Guide to Debugging and Monitoring Modern AI Systems at Scale
Your new AI product is live. Infra dashboards are all green. Latency is low, error rates are flat, and CPU/GPU utilization looks healthy. Yet your Slack channels are full of screenshots from users asking, “Why did it do this?” The same user goal and high-level prompt can still produce wildly…
Opik Blog14 min read
LLM Model Selection: How to Pick the Right Model for Every Agentic Task
Someone on your team defaulted to the latest and greatest model available, which is also the most expensive model. Maybe it was an agent calling the flagship model for every tool call or a coding agent set to its most capable model out of the box. It worked, so nobody revisited it.
Opik Blog8 min read
Stop Paying for the Same LLM Call Twice | Semantic Caching with BetterDB
BetterDB: https://betterdb.com/b/F4KZe LLM API calls are expensive. Semantic caching is how you stop paying for the same response twice. In this video, I'll show you how to implement semantic caching using BetterDB — step by step, with a live demo — so your AI app serves cached responses for…
Applied with AI
Showing 8 of 8 stories
You're all caught up.
AiByts Weekly Brief
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts