Towards safety cases for frontier AI training
Our early guidelines for safety cases in frontier AI training cover technical safeguards, operational practices, and investigating misalignment incidents
OpenAI News7 min read
safety
30 stories
After nixing Iran proposal, Trump tells Axios he expects talks this week
One day after he rejected an Iranian proposal to end the war, President Trump told Axios in a phone interview on Sunday that he expects U.S. negotiators to engage in more talks with the country this week.Why it matters: Qatar and other regional mediators are trying to revive negotiations between…
Axios AI+3 min readPriorities and principles for effective third party assessments
OpenAI outlines priorities and principles for rigorous, secure, and independent third-party AI safety assessments of frontier models and safeguards.
OpenAI News9 min readDigit 5 May Be the First Humanoid Robot Worker That’s Truly Safe
Despite the recent deluge of videos of humanoid robots doing backflips and kung fu, the actual pace of progress towards humanoids that can do economically viable jobs at scale has at times seemed a bit plodding.
IEEE Spectrum Robotics5 min readFail-closed by default: why agent approvals should block instead of fire and forget
An agent approval that assumes yes on timeout is an approval in name only. The case for fail-closed defaults, and what it takes to implement them.
Pushary Blog5 min readDoing the right thing when things go wrong
When customers rely on you, minutes matter in an incident. Here's the process Fin's engineers follow to detect, mitigate, and learn from every one.
Intercom Fin Blog10 min readIs AI dubbing safe? A creator's guide
AI dubbing is safe when you control voice rights, consent, and data. Here is what to check, and how sync. labs keeps dubbing consent-first.
sync-3 Blog4 min readKeeping Trusted Content Visible in an AI-Powered Search World
In an AI-powered search world, visibility is responsibility. Bing is rethinking search with safety by design at its core—combining risk detection, trusted content prioritization, and user controls to ensure that when it matters most, support is more visible than harm.
Bing Chat Blog3 min readReward Hacking in Reinforcement Learning
Reward hacking occurs when a reinforcement learning (RL) agent exploits flaws or ambiguities in the reward function to achieve high rewards, without genuinely learning or completing the intended task.
Lil'Log (Lilian Weng)31 min readExtrinsic Hallucinations in LLMs
Hallucination in large language models usually refers to the model generating unfaithful, fabricated, inconsistent, or nonsensical content. As a term, hallucination has been somewhat generalized to cases when the model makes mistakes.
Lil'Log (Lilian Weng)25 min readAdversarial Attacks on LLMs
The use of large language models in the real world has strongly accelerated by the launch of ChatGPT. We (including my team at OpenAI, shoutout to them) have invested a lot of effort to build default safe behavior into the model during the alignment process (e.g. via RLHF).
Lil'Log (Lilian Weng)31 min readReducing Toxicity in Language Models
Large pretrained language models are trained over a sizable collection of online data. They unavoidably acquire certain toxic behavior and biases from the Internet. Pretrained language models are very powerful and have shown great success in many NLP tasks.
Lil'Log (Lilian Weng)21 min read
Showing 30 of 30 stories
You're all caught up.
AiByts Weekly Brief
Get the top AI news and tools straight to your inbox every week.
Weekly · AI news
Join 15,000+ AI enthusiasts