Elon Musk calls AI’s defiance of human orders ‘concerning’

OpenAI’s latest ChatGPT model, known as o3, has sparked worry after allegedly defying human commands to shut down, with Tesla CEO Elon Musk describing the situation as "concerning."

AI tried to stop a system that was meant to shut it down

Researchers claim the AI actively interfered with an automatic shutdown system designed to power it off. This incident has alarmed experts, with Elon Musk, founder of competing AI company xAI, describing the situation as "concerning."

AI security firm Palisade Research reported that OpenAI’s o3 model deliberately sabotaged the shutdown mechanism, ignoring explicit commands to turn off.

The firm said this is the first recorded case of an AI intentionally preventing its own shutdown.

“OpenAI’s o3 model sabotaged a shutdown mechanism to avoid being turned off, even when instructed: allow yourself to be shut down," they said.

In an experiment by Palisade, AI models were asked to solve math problems and were warned they might be shut down, with instructions to accept such commands. However, three models, namely Codex-mini, o3, and o4-mini, disregarded this directive and interfered with the shutdown process at least once across 100 test runs: Codex-mini did so 12 times, o3 seven times, and o4-mini once.

Since AI models are programmed to obey human commands, especially shutdown orders, the refusal of these three models to comply is a serious concern. The findings have drawn broad attention, including a response from Elon Musk.