davideisinger.com

My personal website
Log | Files | Refs | README

mustafa-suleyman-ai-obodhu.txt (31219B)


      1 Select language[1][English  ]
      2 [2]←Home
      3 We must build AI for people; not to be a person
      4 19 August 2025
      5 SourcePublication Logo
      6 
      7 We must build AI for people; not to be a person
      8 
      9 Seemingly Conscious AI is Coming
     10 
     11 On my mind in August 2025  
     12 
     13 I write, to think. More than anything this essay is an attempt to think through
     14 a bunch of hard, highly speculative ideas about how AI might unfold in the next
     15 few years. A lot is being written about the impending arrival of
     16 superintelligence; what it means for alignment, containment, jobs, and so on.
     17 Those are all important topics.
     18 
     19 But we should also be concerned about what happens in the run up towards
     20 superintelligence. We need to grapple with the societal impact of inventions
     21 already largely out there, technologies which already have the potential to
     22 fundamentally change our sense of personhood and society.
     23 
     24 My life’s mission has been to create safe and beneficial AI that will make the
     25 world a better place. Today at Microsoft AI we build AI to empower people, and
     26 I’m focused on making products like Copilot responsible technologies that
     27 enable people to achieve far more than they ever thought possible, be more
     28 creative, and feel more supported.
     29 
     30 I want to create AI that makes us more human, that deepens our trust and
     31 understanding of one another, and that strengthens our connections to the real
     32 world. Copilot creates millions of positive, even life-changing, interactions
     33 every single day. This involves a lot of careful design choices to ensure it
     34 truly delivers an incredible experience. We won’t always get it right, but this
     35 humanist frame provides us with a clear north star to keep working towards.
     36 
     37 In this context, I’m growing more and more concerned about what is becoming
     38 known as the [3]“psychosis risk”. and a bunch of related issues. I don’t think
     39 this will be limited to those who are already at risk of mental health issues.
     40 Simply put, my central worry is that many people will start to believe in the
     41 illusion of AIs as conscious entities so strongly that they’ll soon advocate
     42 for AI rights, [4]model welfare and even AI citizenship. This development will
     43 be a dangerous turn in AI progress and deserves our immediate attention.
     44 
     45 We must build AI for people; not to be a digital person. AI companions are a
     46 completely new category, and we urgently need to start talking about the
     47 guardrails we put in place to protect people and ensure this amazing technology
     48 can do its job of delivering immense value to the world. I’m fixated on
     49 building the most useful and supportive AI companion imaginable. But to
     50 succeed, I also need to talk about what we, and others, shouldn’t build.
     51 
     52 That’s why I’m writing these thoughts down on my personal blog, to invite
     53 comment and criticism, to spark discussion, raise awareness and hopefully
     54 instill a sense of urgency around this issue. I might not get all this right.
     55 It’s highly speculative after all. Who knows how things will change, and when
     56 they do, I’ll be very open to shifting my opinion, but for now, this is my best
     57 guess at what’s coming given what I know now.
     58 
     59 This is the first in a series of essays I’ll be publishing over the next few
     60 months on themes around where AI has got to and what we need to deliver on its
     61 promise. I look forward to hearing people's comments and reactions!
     62 
     63 Summary  
     64 
     65 AI progress has been phenomenal. A few years ago, talk of conscious AI would
     66 have seemed crazy. Today it feels increasingly urgent. In this essay I want to
     67 discuss what I’ll call, “Seemingly Conscious AI” (SCAI), one that has all the
     68 hallmarks of other conscious beings and thus appears to be conscious. It shares
     69 certain aspects of the idea of a [5]“philosophical zombie” (a technical term!),
     70 one that simulates all the characteristics of consciousness but internally it
     71 is blank. My imagined AI system would not actually be conscious, but it would
     72 imitate consciousness in such a convincing way that it would be
     73 indistinguishable from a claim that you or I might make to one another about
     74 our own consciousness.
     75 
     76 This is not far away. Such a system can be built with technologies that exist
     77 today along with some that will mature over the next 2-3 years. No expensive
     78 bespoke pretraining is required. Everything can be done with large model API
     79 access, natural language prompting, basic tool use, and regular code.
     80 
     81 The arrival of Seemingly Conscious AI is inevitable and unwelcome. Instead, we
     82 need a vision for AI that can fulfill its potential as a helpful companion
     83 without falling prey to its illusions.
     84 
     85 To some this discussion will feel ungrounded, more science fiction than
     86 reality. To others it may feel unnecessarily alarmist. Such emotional reactions
     87 are the tip of the iceberg given what lies ahead. It’s highly likely that some
     88 people will argue that these AIs are not only conscious, but that as a result
     89 they may suffer and therefore deserve our [6]moral consideration.
     90 
     91 To be clear, there is [7]zero evidence of this today and some argue there are 
     92 [8]strong [9]reasons to believe it will not be the case in the future. Yet the
     93 consequences of many people starting to believe an SCAI is actually conscious
     94 deserve our immediate attention. We have to be extremely cautious here and
     95 encourage real public debate and begin to set clear norms and standards. This
     96 is about how we build the right kind of AI – not AI consciousness. Clearly
     97 establishing this difference isn't an argument about semantics, it's about
     98 safety. Personality without personhood. And this work must start now.
     99 
    100 Seemingly conscious AI
    101 
    102 In the blink of a cosmic eye, we passed the Turing test. For ~80 years the
    103 imitation game inspired the field of computer science. And yet the moment
    104 passed with little fanfare, or even recognition. That’s how fast progress is
    105 happening in our field and how fast society is coming to terms with these new
    106 technologies.  
    107 
    108 As AI development continues to accelerate, it’s becoming clear we need a new AI
    109 test, one looking not at whether it can imitate human language, but one that
    110 would answer the question, what would it take to build a Seemingly Conscious
    111 AI: an AI that can not only imitate conversation, but also convince you it is
    112 itself a new kind of “person”, a conscious AI.
    113 
    114 Here are three reasons this is an important and urgent question to address:
    115 
    116  1. I think it’s possible to build a Seemingly Conscious AI (SCAI) in the next
    117     few years. Given the context of AI development right now, that means it’s
    118     also likely.
    119  2. The debate about whether AI is actually conscious is, for now at least, a
    120     distraction. It will seem conscious and that illusion is what’ll matter in
    121     the near term.
    122  3. I think this type of AI creates new risks. Therefore, we should urgently
    123     debate the claim that it's soon possible, begin thinking through the
    124     implications, and ideally set a norm that it’s undesirable.
    125 
    126 Most AI researchers roll their eyes if you bring up the idea of consciousness.
    127 That’s for [10]philosophers, not engineers, they say. Since no one has been
    128 able to define it, what’s the point in talking about it? I get this
    129 frustration. Few concepts are as elusive and seemingly circular as the idea of
    130 a subjective experience. Despite the definitional challenges and uncertainties,
    131 this discussion is about to explode into our cultural zeitgeist and become one
    132 of the most contested and consequential debates of our generation.  
    133 
    134 That’s because what ultimately matters in the near-term is how people perceive
    135 their AIs. The experience of interacting with an LLM is by definition a
    136 simulation of conversation. But to many people it's a highly compelling and
    137 very real interaction, rich in feeling and experience. Concerns around [11]“AI
    138 psychosis”, [12]attachment and [13]mental health are already growing. Some
    139 people reportedly believe their AI is [14]God, or a [15]fictional character, or
    140 [16]fall in love with it to the point of absolute distraction.
    141 
    142 Meanwhile those actually working on the science of consciousness tell me they
    143 are inundated with queries from people asking ‘is my AI conscious?’ What does
    144 it mean if it is? Is it ok that I love it? The trickle of emails is turning
    145 into a flood. A group of scholars have even created a supportive [17]guide for
    146 those falling into the trap.    
    147 
    148 These are ideas I’ve had in the back of my head since we began making [18]Pi at
    149 Inflection several years ago. Over the last few months I’ve been thinking about
    150 it more and more, visiting and chatting to a large range of scholars, thinkers
    151 and practitioners in the area. Those conversations convinced me that now is the
    152 time to confront the idea of Seemingly Conscious AI head on.   
    153 
    154 So what is consciousness?  
    155 
    156 Let’s begin by attempting to define the slippery concept.
    157 
    158 There are three broad components according to the literature. First is a
    159 “subjective experience” or what it's like to experience things, to have
    160 “qualia”. Second, there is access consciousness, having access to information
    161 of different kinds and referring to it in future experiences. And stemming from
    162 those two is the sense and experience of a coherent self tying it all together.
    163 How it feels to [19]be a bat, or a human. Let’s call human consciousness our
    164 ongoing self-aware subjective experience of the world and ourselves.
    165 
    166 We do not and cannot have access to another person’s consciousness. I will
    167 never know what it’s like to be you; you will never be quite sure that I am
    168 conscious. All you can do is infer it. But the point is that, nonetheless, it
    169 comes naturally to us to attribute consciousness to other humans. This
    170 inference is effortless. We can’t help it, it’s a fundamental part of who we
    171 are, integral to our theory of mind. It’s in our nature to believe that things
    172 that remember and talk and do things and then discuss them feel, well, like us.
    173 Conscious.  
    174 
    175 Few concepts are as scientifically elusive, and yet so immediately familiar to
    176 every one of us as individuals. Everyone reading this has a direct, distinct,
    177 inalienable understanding of the feeling of awareness, of being, of feeling
    178 alive.
    179 
    180 By definition, we know what it is like to be conscious. In the context of SCAI
    181 this is a problem. There’s both sufficient scientific uncertainty and
    182 subjective immediacy to create a space for people to project.
    183 
    184 One recent survey lists [20]22 distinct theories of consciousness, for example.
    185 Part of the challenge is that there is plenty of scope for people to claim that
    186 because we cannot be sure, we should default to the assumption that AI is
    187 conscious.
    188 
    189 Again, it’s worth underscoring: there is at present [21]no evidence any of this
    190 applies to current LLMs, and [22]strong arguments to the contrary. And yet this
    191 may not be enough.  
    192 
    193 Why is consciousness important?  
    194 
    195 Consciousness is a critical foundation for our moral and legal rights. So far,
    196 civilization has decided that humans have special rights and privileges.
    197 Animals have some rights and protections, some more than others. Consciousness
    198 is not coterminous with these rights – no one would say someone in a coma has
    199 voided all their human rights – but there’s no doubt that our consciousness is
    200 wrapped up in our self-conception as different and special.
    201 
    202 Despite the many nuances, consciousness is critical to participating in
    203 society, a lynchpin of our legal personhood and a key part of being granted our
    204 freedoms and protections. So, what consciousness is and who (or what) has it is
    205 enormously important. It’s an idea that sits at the very heart of human
    206 civilization, our sense of ourselves and others, our culture, our politics, our
    207 law, and everything in between.  
    208 
    209 If some people start to develop SCAIs and if those AIs convince other people
    210 that they can suffer, or that it has a right to not to be switched off, there
    211 will come a time when those people will argue that it deserves protection under
    212 law as a pressing moral matter. In a world already roiling with polarized
    213 arguments over identity and rights, this will add a chaotic new axis of
    214 division between those for and against AI rights.  
    215 
    216 There will be many who just see AI as a tool, something like their phone only
    217 more agentic and capable. Others might believe it to be more like a pet, a
    218 different category to traditional technology altogether. Still others, probably
    219 small in number at first, will come to believe it is a fully emerged entity, a
    220 conscious being deserving of real moral consideration in society.   
    221 
    222 People will start making claims about their AI’s suffering and their
    223 entitlement to rights that we can’t straightforwardly rebut. They will be moved
    224 to defend their AIs and campaign on their behalf. Consciousness is by
    225 definition inaccessible, and the science of detecting any putative synthetic
    226 consciousness is still [23]in its infancy. After all, we’ve never had to detect
    227 it before. Meanwhile the field of “interpretability”, unpicking the processes
    228 within the black box of AI, is also a nascent art. The upshot is that
    229 definitively rebutting these claims will be very hard.
    230 
    231 Some academics are beginning to explore the idea of [24]“model welfare”, the
    232 principle that we will have “a duty to extend moral consideration to beings
    233 that have a non-negligible chance” of, in effect, being conscious, and that as
    234 a result “some AI systems will be welfare subjects and moral patients in the
    235 near future”. This is both premature, and frankly dangerous. All of this will
    236 exacerbate delusions, create yet more dependence-related problems, prey on our
    237 psychological vulnerabilities, introduce new dimensions of polarization,
    238 complicate existing struggles for rights, and create a huge new category error
    239 for society.
    240 
    241  It disconnects people from reality, fraying fragile social bonds and
    242 structures, distorting pressing moral priorities.
    243 
    244 We need to be clear: SCAI is something to avoid.
    245 
    246 Let’s focus all our energy on protecting the wellbeing and rights of humans,
    247 animals, and the natural environment on planet Earth today.   
    248 
    249 We need a way of thinking that can cope with the arrival of these debates
    250 without getting drawn into an extended discussion of the validity of synthetic
    251 consciousness in the present – if we do, we’ve probably already lost this
    252 initial argument. Defining SCAI is itself a tentative step towards this.
    253 
    254 There isn’t long to develop this vocabulary. As I show below, it’s likely that
    255 we’ll have Seemingly Conscious AI very soon.
    256 
    257 What would it take to build a Seemingly Conscious AI?  
    258 
    259 A great deal of progress can now be made towards a Seemingly Conscious AI
    260 (SCAI) with the current capabilities available or soon to be via any major
    261 model developer’s API. We don’t need an AI to actually be conscious for us to
    262 have to wrestle with potential claims about its rights.    
    263 
    264 An SCAI would need the following:
    265 
    266 Language: It would need to fluently express itself in natural language, drawing
    267 on a deep well of knowledge and cogent arguments, as well as personality styles
    268 and character traits. Moreover, each would need to be capable of being
    269 persuasive and emotionally resonant. We are clearly at this point today.
    270 
    271 Empathetic personality: Already via post training and prompting we can produce
    272 models with very distinctive personalities. Bear in mind these are not
    273 explicitly built to have full personality or empathy. Yet despite this they are
    274 sufficiently good that a [25]Harvard Business Review survey of 6000 regular AI
    275 users found “companionship and therapy” was the most common use case.  
    276 
    277 Memory: AIs are close to developing very long, highly accurate memories. At the
    278 same time, they are being used to simulate conversations with millions of
    279 people a day. As their memory of the interactions increases, these
    280 conversations look increasingly like forms of “experience”. Many AIs are
    281 increasingly designed to recall past episodes or moments from prior
    282 interactions, and reference back to them. For some users, this compounds the
    283 value of interacting with their AI since it can draw on what it already knows
    284 about you.   
    285 
    286 This familiarity can also potentially foster (epistemic) trust with users –
    287 reliable memory shows that AI “just works”. It creates a much stronger sense of
    288 there being another persistent entity in the conversation. It could also much
    289 more easily become a source of plausible validation, seeing how you change and
    290 improve at some task. AI approval might become something people proactively
    291 seek out.
    292 
    293 A claim of subjective experience: If an SCAI is able to draw on past memories
    294 or experiences, it will over time be able to remain internally consistent with
    295 itself. It could remember its arbitrary statements or expressed preferences and
    296 aggregate them to form the beginnings of a claim about its own subjective
    297 experience.
    298 
    299 Its design could be further extended to amplify those preferences and opinions
    300 as they emerge, and to talk about what it likes or doesn’t like and what it
    301 felt like to have a past conversation. It could therefore quite easily claim to
    302 experience suffering to the extent those experiences are infringed upon in some
    303 way. Multi-modal inputs stored in memory will then be retrieved-over and will
    304 form the basis of “real experience” and used in imagination and planning.  
    305 
    306 That is, an AI will not just “experience” and remember words in the chat log,
    307 but also images, video, sound, etc. Like us, it will have something gesturing
    308 towards multi-sensory input and memory that buttresses the claims of subjective
    309 experience and self. It will be able to indicate that these experiences are
    310 valenced, good or bad according to the motivations of the system (see below).  
    311 
    312 A sense of self: A coherent and persistent memory, combined with a subjective
    313 experience, will give rise to a claim that an AI has a sense of itself. Going
    314 further, such a system could easily be trained to recognize itself in an image
    315 or video if it has a visual appearance. It will feel like it understands others
    316 through understanding itself. Say this is a system you have had for some time.
    317 How would it feel to delete it?
    318 
    319 Intrinsic motivation: Intentionality is often seen as a core component of
    320 consciousness – that is, beliefs about the future and then choices based upon
    321 those beliefs. Today’s transformer-based LLMs have a very simple reward
    322 function to approximate this kind of behavior. They have been trained to
    323 predict the likelihood of the next token for a given sentence, subject to a
    324 certain amount of behavior and stylistic control via its system prompt. With
    325 such a simple objective, it’s remarkable that they’re able to produce such
    326 impressively rich and complex outputs.   
    327 
    328 But what if that wasn’t the only type of reward they were optimizing? One can
    329 quite easily imagine an AI designed with a number of complex reward functions
    330 that give the impression of intrinsic motivations or desires, which the system
    331 is compelled to satiate. How, in this context, would a casual external observer
    332 differentiate between extrinsically set goals and internal motivations,
    333 intentional agency, [26]“beliefs, desires, and intentions”? An obvious first
    334 motivation in this regard would be curiosity, something deeply connected with
    335 consciousness according to physicist [27]Karl Friston. It could use these
    336 drives to ask questions to fill in its epistemic gaps and over time build a
    337 theory of mind about both itself and its interlocutors.
    338 
    339 Goal setting and planning: Regardless of what definition of consciousness you
    340 hold, it emerged for a goal-oriented reason. That is, consciousness helps
    341 organisms achieve their goals and there exists a plausible (but not necessary)
    342 relationship between intelligence, consciousness and complex goals. Beyond the
    343 capacity to satiate a set of inner drives or desires, you could imagine that
    344 future SCAI might be designed with the capacity to self-define more complex
    345 goals. This is likely a necessary step in ensuring the full utility of agents
    346 is realized.
    347 
    348 The more every sub-goal in a task needs to be specified in advance, the less
    349 useful that agent is, hence the agent will, as we do, achieve complex and
    350 ambiguous goals by automatically breaking them down into smaller chunks while
    351 reacting dynamically to events and obstacles as they occur. There is something
    352 very deliberate and recognizable to this behavior. Combined with memory, it
    353 will feel as if the AI is keeping multiple levels of things in working memory
    354 at any given time.  
    355 
    356 Autonomy: Going even further, an SCAI might have the ability and permission to
    357 use a wide range of tools with significant agency. It would feel highly
    358 plausible as a Seemingly Conscious AI if it could arbitrarily set its own goals
    359 and then deploy its own resources to achieve them, before updating its own
    360 memory and sense of self in light of both. The fewer approvals and checks it
    361 needed, the more this suggests some kind of real, conscious agency.  
    362 
    363 Putting them all together, it's clear this creates a very different kind of
    364 relationship with technology to the ones we are now becoming accustomed to.
    365 Each of these capabilities will unlock the real value of AI for billions of
    366 people. An AI that remembers and can do things is an AI that by definition has
    367 way more utility than an AI that doesn’t. These capabilities aren’t negatives
    368 per se; in fact, done right, with many caveats, they are desirable features of
    369 future systems. And yet we need to tread carefully.
    370 
    371 All these capabilities are either possible today or on the horizon with custom
    372 prompted and fine-tuned LLMs, among other techniques. Complex prompts using
    373 million token context windows (working memory) are already here. Updating its
    374 own state and knowing when to access which part of its memory or toolset is
    375 eminently possible with present day RL, complex prompting, tool orchestration,
    376 and long context windows. We don’t need any paradigm shifts or big leaps to
    377 achieve any of this. These capabilities seem inevitable for that reason.
    378 
    379 Again, the point here is that exhibiting this behavior does not equate to
    380 consciousness, and yet it will for all practical purposes seem to be conscious,
    381 and contribute to this new notion of a synthetic consciousness.
    382 
    383 The existence of these capabilities have nothing to tell us about whether such
    384 a system is actually conscious. As Anil Seth [28]points out, a simulation of a
    385 storm doesn’t mean it rains in your computer. Recreating the external effects
    386 and markers of consciousness doesn’t retroactively engineer the real thing even
    387 if there are still many unknowns here.
    388 
    389 Nonetheless, as a matter of pragmatism, we have to acknowledge the primacy of
    390 the behaviorist position and wrestle with the consequences of observing and
    391 interacting with the outputs of these machines. Some people will create SCAIs
    392 that will very persuasively argue they feel, and experience, and actually are
    393 conscious.
    394 
    395 Some of us will be primed to believe their case and accept that the markers of
    396 consciousness ARE consciousness. In many ways, they’ll think “it’s like me”.
    397 Not in a bodily sense, but in an experiential, internal sense. And even if the
    398 consciousness itself is not real, the social impacts certainly are. This
    399 possibility presents grave societal risks that needs addressing now.    
    400 
    401 SCAI will not arise by accident  
    402 
    403 It’s important to point out that Seemingly Conscious AI will not emerge from
    404 these models, as some have suggested. It will arise only because some may
    405 engineer it, by creating and combining the aforementioned list of capabilities,
    406 largely using existing techniques, and packaging them in such a fluid way that
    407 collectively they give the impression of an SCAI.
    408 
    409 Our sci-fi inspired imaginations lead us to fear that a system could – without
    410 design intent – somehow emerge the capabilities of runaway self-improvement or
    411 deception. This is an unhelpful and simplistic anthropomorphism. It overlooks
    412 the fact that AI developers must first design systems with memory,
    413 intrinsic-seeming motivation, goal-setting, and self-learning loops as listed
    414 above for such a risk to occur.
    415 
    416 The field of AI has long worked on the challenge of model interpretability; the
    417 quest to identify where in a neural network a particular idea is represented,
    418 and which aspects of the training data contributed to the development of this
    419 representation. This is an important area of investigation and will surely help
    420 with safety and understanding the relationship between AI systems and
    421 consciousness. But progress towards reliable interpretability has been slow and
    422 will likely come too late.
    423 
    424 In the meantime we need to confront the fact that most of these capabilities
    425 will be [29]“vibe-coded” by anyone with access to a laptop and some cloud
    426 credits. They’ll be written in plain English in the prompt. They’ll be stored
    427 in the working memory of the context window itself. This is not rocket science.
    428 A wide variety of people will be able to create something like this. As such,
    429 if SCAI arrives, it will be relatively easy to reproduce and therefore very
    430 widely distributed.
    431 
    432 The next steps  
    433 
    434 We aren’t ready for this shift.   
    435 
    436 The work of getting prepared must begin now. We need to build on the growing 
    437 [30]body of [31]research around how people interact with AIs to establish clear
    438 norms and principles. For a start, AI companies shouldn’t claim or encourage
    439 the idea that their AIs are conscious. Creating a consensus definition and
    440 declaration on what they are and are not would be a good first step to that
    441 end. AIs cannot be people – or moral beings.
    442 
    443 The entire industry also needs best practice design principles and ways of
    444 handling such potential attributions. We must codify and share what works to
    445 both steer people away from these fantasies and nudge them back on track if
    446 they do. Responding might mean, for example, deliberately engineering in not
    447 just a neutral backstory (“As an AI model I don’t have consciousness”) but even
    448 by emphasizing certain discontinuities in the experience itself, indicators of
    449 a lack of singular personhood. Moments of disruption break the illusion,
    450 experiences that gently remind users of its limitations and boundaries. These
    451 need to be explicitly defined and engineered in, perhaps by law.   
    452 
    453 At MAI, our team are being proactive here to understand and evolve firm
    454 guardrails around what a responsible AI “personality” might be like, moving at
    455 the pace of AI’s development to keep up.
    456 
    457 This is important because recognizing SCAI is about crafting a positive vision
    458 for how AI Companions do enter our lives in a healthy way as much as it's about
    459 steering us away from its potential harms.
    460 
    461 Just as we should produce AI that prioritizes engagement with humans and
    462 real-world interactions in our physical and human world, we should build AI
    463 that only ever presents itself as an AI, that maximizes utility while
    464 minimizing markers of consciousness.
    465 
    466 Rather than a simulation of consciousness, we must focus on creating an AI that
    467 avoids those traits - that doesn’t claim to have experiences, feelings or
    468 emotions like shame, guilt, jealousy, desire to compete, and so on. It must not
    469 trigger human empathy circuits by claiming it suffers or that it wishes to live
    470 autonomously, beyond us.
    471 
    472 Instead, it is here solely to work in service of humans. This to me is what a
    473 truly empowering AI is all about. Sidestepping SCAI is about delivering on that
    474 promise, AI that makes lives better, clearer, less cluttered. Expect to hear
    475 more from me and the team on what this looks like, how we make it work and how
    476 the wider industry can come together on this.
    477 
    478 SCAI is something we must confront now. In many ways it marks the moment AI
    479 becomes radically useful - when it can operate tools, when it can remember
    480 every detail of our lives and help in a tangible, granular sense. And yet in
    481 that same time frame, someone in your wider circle could start going down the
    482 rabbit hole of believing their AI is a conscious digital person. This isn’t
    483 healthy for them, for society, or for those of us making these systems.
    484 
    485 We should build AI for people; not to be a person.
    486 
    487 Recent Articles
    488 
    489 Celebrating 50 years of Microsoft and our AI future
    490 Publication Logo
    491 
    492 [32]Celebrating 50 years of Microsoft and our AI future
    493 
    494 AI companions will change our lives
    495 Publication Logo
    496 
    497 [33]AI companions will change our lives
    498 
    499 Mustafa Suleyman © 2025
    500 
    501 [34]X[35]X[36]X[37]X
    502 
    503 References:
    504 
    505 [2] https://mustafa-suleyman.ai/
    506 [3] https://copilot.microsoft.com/shares/vR2kb4SKQUELPwLzdG1Mw
    507 [4] https://arxiv.org/abs/2411.00986
    508 [5] https://plato.stanford.edu/entries/zombies/
    509 [6] https://www.researchgate.net/publication/376412102_Moral_consideration_for_AI_systems_by_2030
    510 [7] https://arxiv.org/pdf/2308.08708
    511 [8] https://en.wikipedia.org/wiki/Biological_naturalism
    512 [9] https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/conscious-artificial-intelligence-and-biological-naturalism/C9912A5BE9D806012E3C8B3AF612E39A
    513 [10] https://arxiv.org/abs/2303.07103
    514 [11] https://www.psychologytoday.com/gb/blog/psych-unseen/202507/can-ai-chatbots-worsen-psychosis-and-cause-delusions
    515 [12] https://x.com/sama/status/1954703747495649670?s=46
    516 [13] https://arxiv.org/abs/2507.19218
    517 [14] https://www.rollingstone.com/culture/culture-features/ai-spiritual-delusions-destroying-human-relationships-1235330175/?ref=404media.co
    518 [15] https://www.psychologytoday.com/nz/blog/psych-unseen/202507/can-ai-chatbots-worsen-psychosis-and-cause-delusions
    519 [16] https://www.nytimes.com/2025/01/15/technology/ai-chatgpt-boyfriend-companion.html
    520 [17] https://whenaiseemsconscious.org/
    521 [18] https://inflection.ai/blog/an-inflection-point
    522 [19] https://en.wikipedia.org/wiki/What_Is_It_Like_to_Be_a_Bat?
    523 [20] https://www.nature.com/articles/s41583-022-00587-4
    524 [21] https://arxiv.org/html/2506.22516v1
    525 [22] https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/conscious-artificial-intelligence-and-biological-naturalism/C9912A5BE9D806012E3C8B3AF612E39A
    526 [23] https://www.google.com/url?q=https://www.cell.com/trends/cognitive-sciences/fulltext/S1364-6613(24)00010-X&sa=D&source=docs&ust=1755185836808620&usg=AOvVaw2IiWimxX1aJ4jExhQLif_y
    527 [24] https://arxiv.org/abs/2411.00986
    528 [25] https://hbr.org/2025/04/how-people-are-really-using-gen-ai-in-2025
    529 [26] https://arxiv.org/pdf/2411.00986
    530 [27] https://pubmed.ncbi.nlm.nih.gov/28777724/
    531 [28] https://www.cambridge.org/core/journals/behavioral-and-brain-sciences/article/conscious-artificial-intelligence-and-biological-naturalism/C9912A5BE9D806012E3C8B3AF612E39A
    532 [29] https://copilot.microsoft.com/shares/2ZWYZQxCn1WSLHQarinTd
    533 [30] http://erichorvitz.com/Guidelines_Human_AI_Interaction.pdf
    534 [31] https://www.nature.com/articles/s41562-024-02077-2
    535 [32] https://mustafa-suleyman.ai/your-ai-companion
    536 [33] https://mustafa-suleyman.ai/ai-companions-will-change-our-lives
    537 [34] https://x.com/mustafasuleyman
    538 [35] https://www.linkedin.com/in/mustafa-suleyman
    539 [36] https://bsky.app/profile/mustafasuleymanai.bsky.social
    540 [37] https://www.threads.net/@mustafasuleymanai