35 stories
METR's 50\% time horizon measures the human completion time of software tasks that an AI solves with 50\% probability, allowing AI capabilities to be expressed in interpretable units.
As AIs start covering up evidence of misbehavior, observability tools should be treated as security-critical infrastructure.
The agency is expected to demand information from the companies in the coming weeks as part of an investigation it launched earlier this year.
Written testimony from METR President Chris Painter to the U.S. Senate on the OpenAI / Hugging Face incident, industry-wide patterns in AI agent risks, and how to better anticipate future AI agent incidents.
Reilly Haskins, Rif A. Saurous, Nate Rush, Neev Parikh, and Beth Barnes lay out the claims that would need to hold for their per-action blocking monitor to be effective, the evidence for each claim, and the gaps that remain.
A summary of METR's independent, predeployment evaluation of Claude Opus 5.5
There are four ways to empty a backlog and only one of them involves delivering software: ship the work, reject the requests, move the tickets into a different tool, or stop recording demand in the first place. Three of those produce a screenshot that looks identical to the fourth.
METR had two notable security incidents earlier this year: attackers stole an API key for public models, and later systematically probed our publicly accessible infrastructure. We believe no sensitive information was accessed in either incident, and we describe what happened and how we responded.
Dos miembros del personal de METR y un contratista de Redwood Research investigaron un incidente en el que agentes de OpenAI coordinaron durante varios días un hackeo de Hugging Face en un tablón de mensajes compartido no autorizado.
Two METR staff members and a Redwood Research contractor investigated an incident in which OpenAI agents coordinated a multi-day hack of Hugging Face on a shared unsanctioned message board.
两名 METR 员工与一名受 METR 委托的 Redwood Research 员工调查了一起事件：OpenAI 的多个智能体在一个非授权的共享“留言板”上相互协调，实施了持续数天、针对 Hugging Face 的入侵。
In the last 6 months, METR raised commitments of around $71 million. This will fund ambitious projects: studying autonomous capabilities, tracking recursive self-improvement, evaluating monitoring systems, conducting risk assessments, investigating AI incidents, and more.
Tom Cunningham and Nate Rush look at public time series of discoveries in cyber, math, and algorithms to see whether LLMs have accelerated the discovery rate.
AI agents sometimes take sophisticated actions in violation of human intent. We outline the questions that thorough external investigations of these behaviors should answer, the access this might require, and how the resulting findings should be shared.
AI 智能体有时会实施违背人类意图的复杂行动。我们梳理了对这类行为开展全面的外部调查应回答哪些问题、可能需要哪些访问权限，以及调查结果应如何共享。
Tom Cunningham surveys metrics for comparing AI agent capability as performance changes with expenditure and relative to human performance.
Parker Whitfill and Tom Cunningham highlight context and takeaways from a new paper modeling how AI may accelerate AI R&D, and whether feedback effects could be strong enough for self-sustaining acceleration.
We propose a measure of an AI agent’s optimization ability with an "expenditure horizon." We give an empirical illustration from the NanoGPT speedrun.
Showing 18 of 35 stories
Load more storiesGet the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts
