59 stories
I tested Jev and five LLM judges across ten Arize Phoenix evaluators to measure how often their answers changed alongside accuracy, cost, and latency. The post What Jev’s probabilities reveal that repeated LLM judgments miss appeared first on Arize AI.
An agent passes every offline eval in CI. You ship it. Production breaks within a week, because a class of prompts your eval dataset never...
Last Updated on September 25, 2026 Most engineers approaching their first production LLM system bring a mental model that works well for software: test your inputs and outputs with assertions, check your code into git, roll back if something breaks, and rely on system prompts or configuration files…
Model routing for coding agents in September 2026: a frontier model plans, a cheap one executes. The plan and price numbers developers measured.
We replaced the 58 tools PXI used to drive the Phoenix UI with two tools and a JavaScript sandbox that runs in your browser tab. Here's why, how it works, and what it cost. The post How we built UI Code Mode into Arize Phoenix appeared first on Arize AI.
Evaluate agent responses with Jev and Arize AX. Build a FastAPI remote evaluator, return labels and scores, and choose a threshold for your data. The post How to build a remote evaluator in Arize AX with Jev appeared first on Arize AI.
We benchmarked Jev against Claude Opus 5 and GPT-5.6 Terra on accuracy, cost, and latency. Learn how threshold tuning changes hallucination detection. The post Jev vs. LLM-as-a-Judge: Accuracy and cost benchmarks appeared first on Arize AI.
The following article originally appeared on Arize’s blog and is being reposted here with the author’s permission. This week the AI community was in uproar about Jev from TypeSafe, not just a new model but a new kind of model: one that classifies, scores, and routes but can’t write a sentence.
Compare Jev and GPT-5.4 nano for real-time LLM guardrails, with demo results on latency, cost, and checks on agent inputs, replies, and tool calls. The post Real-time LLM guardrails with Jev: comparing latency and cost appeared first on Arize AI.
Daytona cofounder Ivan Burazin wants agents that can finish the job within the authority they have been given. His interview offers a starting point for examining how agents use your software. The post What changes when AI agents use your software appeared first on Arize AI.
Sessions are now a first-class unit of work in Arize AX. Annotate a whole conversation, queue it for review, and ask Alyx to filter for it — plus Agent-as-a-Judge on every plan, vision judges, faster datasets, and everything else that shipped between August 6 and September 18, 2026.
Still guessing if your prompts work? Learn how to build a systematic prompt testing framework that catches regressions before they reach production.
Free AI Evals Course : https://github.com/aishwaryanr/awesome-generative-ai-guide/blob/main/free_courses/ai_evals_for_everyone/README.md 🚀Get 3 months of Arize AX Pro Free: https://app.arize.com/auth/join?utm_source=youtube&utm_medium=video&utm_campaign=ai-evals-video&utm_content=aishwarya-nr…
An open-source (MIT) spec-and-verify harness that sits around your coding agent — a strong idea with broad host support, still early and unverified by us at runtime.
by Tuana Celik ( LI, Twitter) This is part one of the Advanced Use Cases series: 1️⃣ Extract Metadata from Queries to Improve Retrieval cookbook & full article 2️⃣ Query Expansion cookbook & full article 3️⃣ Query Decomposition cookbook & the full article 4️⃣ Automated Metadata Enrichment In this…
A researched map of how teams watch AI agents calling REST and general APIs in 2026: the application-layer trace platforms, the API-gateway layer where identity actually lives, and what genuine per-agent attribution requires.
A researched map of how teams actually watch MCP traffic today: the protocol's own shift to OpenTelemetry, the observability platforms that added MCP tracing, and the gap between debugging a server and running one.
A buyer's comparison of seven LLM evaluation tools: what each one meters, whether it scores curated datasets or live production traffic, what its license actually permits, and how much of your setup survives a switch.
Showing 54 of 59 stories
Load more storiesGet the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts
