Stay updated with the latest developments in Artificial Intelligence
An open-source, local, read-only MCP server that checks your request before Google quietly hands back nothing Ask the Google Search Console API for clicks filtered by country us and you get zero rows. No error, no warning, just nothing. It's not the only one.
Executive SummaryHuman societies exhibit broad shifts in values and norms over time, often driven by moral disagreements.One way of understanding moral progress is the resolution of these disagreements through a process of reflection by rational agents.Frontier AI companies use documents such as…
By Alex Turner, Amber Scorah, and Karl Koch. Op-ed written for Fortune.Giving independent evaluators “employee-like access” is “a great idea”, OpenAI CEO Sam Altman said on September 12th.
Let's say we coordinate to pace the frontier, but we don't coordinate on an outright pause. What should the standard of pacing be?My proposal for a minimum requirement: a lab cannot train the next generation of models (in terms of capabilities) until they have adequately[1] resolved all significant…
My post on endogenous alignment got several excellent comments. Among them, one from Karl Krueger argued that imitation learning is more important for how humans align their children than reinforcement learning.
This post argues that fixed-weight models (at least as we understand them today) will a) always be vulnerable to adversarial examples in their concept-spaces, and b) hence will be misaligned, under sufficient optimisation pressure.Boundaries in concept spaceTo serve any purpose whatsoever, an AI…
Many control protocols work by intervening on an untrusted AI's actions during deployment. For example, you might set up a monitor that scores each action's suspiciousness and blocks actions above a threshold, replacing them with actions from a weaker "trusted" model (a defer-to-trusted protocol).
Summary: Currently, “chain of thought” (CoT) is our most valuable tool for understanding the reasoning and cognition of AI systems. However, some architectures would enable AI models to reason much more extensively in latent states rather than in text CoT.
Summary:First, I give several different angles on how I feel about reinforcement learning:Theoretical case: RL is a black-box source of agency — this should give us classic misalignment worries, especially compared to agency-via-scaffoldingRecent incidents (huggingface etc) and more mundane forms…
TL;DRWe introduce WorkspaceBench, a set of evaluations for how well an activation-to-text tool can read the contents of the “global workspace” of a model, i.e.
In previous papers (Kosoy 2026a,b), we began the study of sequence prediction algorithms adapted to stringological word complexity measures. In particular, we defined a complexity measure called Arithmetic Repetition Complexity (ARC) which admits a polynomial-time prediction algorithm with a…
(Or: Why Bentham's Bulldog and John Halstead are wrong in their critique of Kulveit et al.)Gradual Disempowerment is a 2025 paper (with a nice, dedicated website) proposing a form of existential risk from AI that goes beyond "mundane" risks like bioweapon uplift or mainline misaligned-AI-takeover…
It would be useful if we had the ability to modify a model’s beliefs. For example, this could facilitate honeypots and better monitoring[1], help us do better science on current models[2], and augment certain forms of alignment training[3].
Published in The Guardian.Major AI lab CEOs recently advocated for pacing AI development. They are right to be concerned: the field runs an extremely dangerous race towards superintelligent AI.
The CoTControl eval asks reasoning models to follow formatting constraints in their chain-of-thought (e.g. write in all lowercase, avoid a specific word) while solving questions. Models seem to mostly be pretty bad at this: recent models score between 0-30% with the exception of Mythos Preview[1].
Showing 14 of 14 stories
You're all caught up.
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news from Alignment Forum
Join 15,000+ AI enthusiasts
