Fascinating post by Mustafa Suleyman on the risks of achieving “seemingly conscious” AI, and how we must design systems to help real people, avoiding the illusion of consciousness:

A coherent and persistent memory, combined with a subjective experience, will give rise to a claim that an AI has a sense of itself. Going further, such a system could easily be trained to recognize itself in an image or video if it has a visual appearance. It will feel like it understands others through understanding itself. Say this is a system you have had for some time. How would it feel to delete it?