Anthropic, OpenAI's proposed AI risk evaluators may not have enough power to prevent disasters
Anthropic and OpenAI are proposing embedded AI evaluators to help manage risk of models causing catastrophic harm to society, but the idea has some issues.