3 stories
This post argues that fixed-weight models (at least as we understand them today) will a) always be vulnerable to adversarial examples in their concept-spaces, and b) hence will be misaligned, under sufficient optimisation pressure.Boundaries in concept spaceTo serve any purpose whatsoever, an AI…
Summary: Currently, “chain of thought” (CoT) is our most valuable tool for understanding the reasoning and cognition of AI systems. However, some architectures would enable AI models to reason much more extensively in latent states rather than in text CoT.
Summary:First, I give several different angles on how I feel about reinforcement learning:Theoretical case: RL is a black-box source of agency — this should give us classic misalignment worries, especially compared to agency-via-scaffoldingRecent incidents (huggingface etc) and more mundane forms…
Showing 3 of 3 stories
You're all caught up.
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news from Alignment Forum
Join 15,000+ AI enthusiasts
