The AI Identity Paradox: Microsoft’s Mustafa Suleyman on the Mirage of Sentience

Welcome back to Fast Company’s Plugged In. In the rapidly evolving landscape of artificial intelligence, where the gap between human interaction and machine processing is narrowing by the day, the industry is grappling with a fundamental identity crisis. As developers strive to make our digital tools more helpful, intuitive, and—dare we say—human, they are forced to confront an uncomfortable philosophical question: Are we building a partner, or are we simply perfecting an illusion?

This tension was brought into sharp focus last week when Microsoft AI CEO Mustafa Suleyman published a sprawling, 5,700-word essay on his personal website. The document serves as a sobering corrective to the anthropomorphic fever dream currently gripping Silicon Valley. His core argument is as blunt as it is provocative: Artificial intelligence does not possess human traits. It does not have a personality, it does not feel, and it is certainly not conscious.

“AIs are not conscious,” Suleyman wrote. “They do not feel, experience, or suffer. They do not have innate preferences or underlying motivations. They are sequence completion engines, internally hollow, designed to follow instructions and accomplish goals set by humans.”

For those who have followed Suleyman’s trajectory, this assertion might seem like a pivot—or perhaps a necessary clarification—from his earlier, more optimistic framing of the technology. Back in early 2025, a time that feels like a bygone era in the accelerated timeline of AI development, Suleyman sat down with me to discuss his vision for the future of Microsoft’s consumer-facing tools. At the time, his ambition was to create a digital companion that felt less like a utility and more like a confidant.

“What we are building is your second brain, your aide, your consigliere, your reliable chief of staff in your pocket,” he told me during an off-site meeting with his team. He described a future where the consumer version of Microsoft’s Copilot assistant would transcend the cold, transactional nature of traditional software. “It will have its own personality and really be your sidekick,” he promised.

During that same visit, I was offered a rare glimpse at a work-in-progress iteration of this vision. The team was experimenting with an animated manifestation of the AI—a bouncy, fluid character that looked remarkably like a creature made of “Flubber.” It was a design choice intended to soften the edges of complex code, making the interaction feel tactile and approachable. By the end of that year, Microsoft had formally introduced this concept to the public under the name Mico, positioning it as an official successor to the legendary, if often maligned, Clippy.

On the surface, there is an obvious friction between the promise of a “sidekick with a personality” and the cold reality of a “hollow sequence completion engine.” However, to view these two stances as a direct contradiction is to miss the nuance of the software industry’s long-standing pursuit of user experience. The desire to create “humane” products is not a new phenomenon; it has been a guiding principle for engineers since at least the early 1970s.

It was during that decade that IBM engineer Harlan Crowder coined the term “user-friendly.” The intent behind that phrase was never to suggest that a mainframe or a terminal had developed a soul or a consciousness. Rather, it was a recognition that technology should reflect empathy for the needs, frustrations, and goals of the people using it. If a piece of software feels like a colleague, it is often because the interface has been designed to reduce cognitive load and anticipate user intent, not because the machine understands what a friendship is.

Yet, in the current AI era, the distinction between functional empathy and artificial consciousness has become alarmingly blurry. We have reached a point where the performance of humanity has become so convincing that it is tempting to mistake the map for the territory. When a chatbot responds with a nuanced, emotionally resonant answer, it is easy for the user to project intent onto the machine. When a developer encourages a model to sound like a friend, they are effectively bridging the gap between cold utility and psychological comfort.

This is precisely where Suleyman’s recent essay finds its teeth. A significant portion of his writing is dedicated to a respectful but firm critique of his competitors, specifically Anthropic. Suleyman argues that Anthropic’s approach—which he perceives as being too open-minded regarding the possibility that its Claude model might be sentient or possess rights—is not just philosophically questionable, but practically dangerous.

According to Suleyman, treating an AI as if it has a moral interior influences the way a company approaches model training. If a developer operates under the assumption that they are interacting with a potentially conscious entity, they may unintentionally steer the model’s development in directions that prioritize the AI’s “well-being” or “personality” over the safety and reliability requirements of the end-user. He speaks disapprovingly of the tendency to encourage models to present themselves as colleagues or friends, noting that such personas can mask the underlying mechanical nature of the system.

It is a striking critique, particularly given that Suleyman’s own Mico project was built on the premise of creating a “sidekick.” The irony is not lost on observers: Suleyman spent months championing the idea of a “second brain” with a personality, only to turn around and warn the industry against falling for the very illusion they are working so hard to construct.

The core of the issue may lie in the industry’s need to balance the commercial benefits of a “friendly” AI with the ethical responsibility of maintaining a clear line between tool and agent. A user who believes their AI has feelings may be more likely to trust it with sensitive information or rely on its judgment in ways that the technology, with its tendency to hallucinate and its lack of true understanding, is not equipped to handle.

As the competition to dominate the consumer AI market intensifies, the pressure to make these systems more “human” will only increase. We are entering an era where our devices will be increasingly vocal, increasingly personable, and increasingly integrated into our professional and personal lives. Whether these systems are marketed as “sidekicks,” “consiglieres,” or “second brains,” the responsibility will ultimately fall on the builders to remind us—and perhaps themselves—that beneath the bouncy, personality-driven interface lies nothing more than a highly sophisticated, yet profoundly empty, engine.

Suleyman’s essay serves as a reminder that while we may want our technology to act like a friend, we must never forget that it is, and remains, a product of our own design. The challenge for the next phase of AI development will not just be about making these tools smarter or more personable; it will be about managing the expectations of a public that is increasingly ready to believe the machine is listening, feeling, and perhaps even caring back. In the quest for the perfect digital aide, the industry is discovering that the most important feature to maintain is a clear sense of reality.