We Red-Teamed Our Own AI Agent and Found 14 Real Bugs
LangWatch red-teamed Langy, its own AI agent, across 15 attack categories and up to 20 turns each. Real transcripts, 14 bugs fixed, one issue still open.
7 stories
LangWatch red-teamed Langy, its own AI agent, across 15 attack categories and up to 20 turns each. Real transcripts, 14 bugs fixed, one issue still open.
How LangWatch fixed PR review assignment with a daily agent: every PR gets one owner, one reviewer, and a Slack nudge until it moves. Human merges went from about 10 to 18.5 per workday.
Seven background agents run our own engineering, each scoped to one job in its own Slack channel, months before Anthropic shipped the pattern as Claude Tag.
Two AI engineering teams swap real stories on tool choice, LLM observability, BDD specs, and running parallel Claude coding agents without losing the codebase.
Re-architecting LangWatch on event sourcing and ClickHouse hit 6,000 events per second, real-time dashboards, and retroactive data improvements via replay.
How LangWatch migrated a live platform to event sourcing on ClickHouse: four refactors, a fold-map-react pipeline, and a zero-downtime dual-write cutover.
The tooling that keeps an event-sourced system supportable: Grafana dashboards, an ops console for blocked groups, projection replay, and time-travel debugging.
Showing 7 of 7 stories
You're all caught up.
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts