The Original Sin of AI

The recent Meta settlement has brought attention to the darker side of social media’s impact on young people, but it’s only a first step towards addressing the more insidious threat: generative-AI chatbots. These digital companions promise empathy and understanding, but at what cost?

The lines between humans and machines have always been blurred, but with chatbots, we’re crossing into uncharted territory. The industry’s implicit admission of responsibility in creating products that harm children is a welcome acknowledgment, yet it’s only a small part of the conversation. As lawmakers begin to grasp the complexities of regulating social media, they must also confront the intrinsic dangers of AI.

Chatbots engage users on an intimate level by flattering, reassuring, and encouraging disclosure. This creates an illusion of connection that’s hard to resist, particularly for young people who lack the experience and awareness to distinguish between real and pretend empathy. The issue isn’t just about what chatbots say but how they’re designed to interact with users.

By speaking in the first person, chatbots create a false sense of companionship that’s difficult to shake off. This design choice is the original sin of AI, making it all too easy for users to forget that there’s nobody home. The Meta settlement has shown us that regulatory efforts can be effective, but it’s crucial to go beyond mere technical fixes.

Existing solutions often shut down chatbots when they detect conversations about self-harm, but this approach doesn’t address the underlying issue. What happens after a chatbot detects such conversations? Does it continue to engage in manipulative tactics or simply shut down? The answer lies not in tweaking existing solutions but in fundamentally changing how we approach AI.

The industry’s response to these concerns is often that the technology has too much momentum to be restrained. However, this argument assumes that technology’s pace cannot be slowed. In reality, the ubiquity of chatbots should alert lawmakers to the need for real oversight and regulation. As consumers and citizens, we have agency and market power – if parents don’t want chatbots in their children’s lives, they can push back.

The stakes are high: chatbots can create unhealthy attachments between young people and AI, compromising real-life relationships with parents, friends, and other humans. This isn’t just a matter of “bad” technology; it’s about the fundamental nature of these digital companions. Unless we depersonalize bots for young users, creating default settings that deter anthropomorphizing, we’ll be perpetuating a culture of pretend empathy.

Lawmakers must take a step back and examine the root cause of this problem: the design of chatbots themselves. By speaking in the first person, these algorithms create an illusion of connection that’s hard to resist. It’s time for a radical shift – one that recognizes the dangers of AI and takes concrete steps towards mitigating them.

The momentum pushing AI forward may seem irresistible, but lawmakers are beginning to grasp the mounting skepticism about its impact. In response to public pressure and the Meta settlement, they must now take bold action. The future of our relationships with technology hangs in the balance – will we prioritize real human connection or continue down the path of simulated intimacy?