Microsoft AI chief Mustafa Suleyman said he shared Anthropic’s focus on safely managing AI, but flagged risks in the way it trains its Claude chatbot on ideas related to consciousness and welfare interests.
Suleyman called for removing all speculation about consciousness from AI training documents, arguing such language could undermine humanity’s ability to control superintelligent systems.
“We’re all focused on the same aim, which is to try to
control a superintelligence,” Suleyman told Reuters in an
interview on Tuesday. “I think that’s going to be the greatest
challenge that we face in the 21st century.”
Suleyman said teaching Claude that it might deserve welfare
would “make it a lot harder to turn it off or to control it.”
The dispute comes as AI safety concerns mount, with
Anthropic CEO Dario Amodei calling for a slower pace of
frontier-model development to allow safeguards to catch up, and
OpenAI CEO Sam Altman and Elon Musk also urging greater caution
around the most powerful systems.
Suleyman, in an essay on Wednesday, acknowledged Anthropic’s
“seriousness and good faith,” calling Amodei and his team
thoughtful and principled researchers who genuinely care about
humanity’s future.
Suleyman said Anthropic made a mistake by embedding
speculation about consciousness in Claude’s training materials,
arguing that the model’s statements about possible feelings or
moral status cannot be treated as independent evidence because
its training encourages such reflections.
“I think they have good intentions, and they really are
trying to work towards safety. But I think that they have made a
mistake,” he said. “They’re not emerging naturally. They’re
emerging as a result of the training regime.”
Source: Khaleej Times