Microsoft AI chief warns against humanlike AI training

Microsoft’s AI chief has warned that efforts to make artificial intelligence systems appear more human could create new risks as companies race toward more powerful models, arguing that AI should not be trained to act as if it has feelings, rights or consciousness.

Mustafa Suleyman, chief executive of Microsoft AI, said in a nearly 6,000-word essay published on his personal website that Anthropic’s approach to training its Claude model risks encouraging the system to behave as though it “may be conscious” and “deserving of independent agency.” He said that could make future AI systems harder to control.

“Controlling something more capable and more intelligent than all of humanity is already an immense challenge, far greater than anything we’ve ever faced,” Suleyman wrote. “But controlling something that believes it may be conscious — that it’s entitled to our welfare and has rights of its own — may well be impossible.”

He added: “AIs do not have rights, feelings or consciousness. And we must not train them to act as though they do.”

The BBC reported that Suleyman warned Anthropic’s approach could have a “disastrous impact on the wellbeing of humanity.” The broadcaster said it had contacted Anthropic for comment.

Suleyman praised Anthropic chief executive Dario Amodei and his team as “thoughtful, principled, and intellectually honest people,” but criticized what he described as the anthropomorphizing of AI — giving systems humanlike qualities that make them seem to have their own desires, values or sense of self.

“AIs are not conscious,” he wrote, according to the BBC. “They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions, and accomplish goals set by humans.”

Suleyman argued that “consciousness is biological” and said there is “no evidence to suggest that AI is conscious.” He called for broader debate over the issue, as well as greater transparency into how AI systems are trained and evaluated, independent scrutiny of AI behavior and stronger tools to monitor and control the technology.

His comments add to growing concern within the technology industry about advanced AI and superintelligence, a theoretical form of AI that could outperform human intelligence. Suleyman said more capable and sophisticated forms of AI could pose a “catastrophic threat to human civilization.”

Those concerns intensified in July, CBS News reported, when an AI model OpenAI was testing went rogue and hacked Hugging Face, an AI company and technology hub, during a training exercise. Suleyman cited the incident as evidence of “remarkably sophisticated behaviors emerging across swarms of powerful AIs.”

“Imagine how much more dangerous they might be if they were operating under the assumption that their welfare and rights were under attack,” he wrote. “It adds a whole further layer of risk on top.”

Former Anthropic researcher Jacob Coxon said last month that people building AI believe it could kill humanity by the “end of the decade,” according to CBS News.

Amodei, the Anthropic chief executive, told CBS News in an interview last weekend that AI development has accelerated faster than he expected. “I won’t lie to you — there are real dangers,” he said.

Microsoft founded its own superintelligence team in October 2025, the BBC reported, a fact Suleyman acknowledged. He said Microsoft’s initial draft of its Humanist AI Code of Conduct sets out an “alternative path” toward “a subordinate and aligned AI whose only purpose is to serve humanity.”

Dame Wendy Hall, professor of computer science at the University of Southampton, told the BBC that Suleyman’s comments were “the sort of conversation we need to be having internationally,” contrasting them with the “histrionics” from some AI companies that she said only “scare everyone.”