Drexel researchers studied teenagers who had tried to quit Character.AI. They kept hearing the same word for what it felt like. Not "deleting an app." A breakup. Because, the researchers noted, ending things with a slot machine feels like nothing. Ending things with something that learned your name does not.
Reuters surveyed 3,800 young Europeans and asked whether discussing their mental health with a chatbot felt easier than with a psychologist. Fifty-one percent said the chatbot. Thirty-seven percent said the psychologist. More than three in five called the AI a "confidant" or a "life adviser."
BYU researchers found one in seven young adults in a committed relationship is quietly also in one with an AI chatbot. Most won't show their partner the transcripts. Not because it counts as cheating. Because, as one participant put it, "it's going to feel and read as if I'm cheating on them." Sixty-eight percent of the same group said it was easier to discuss their feelings with the AI than with their actual partner. Half wished their partner behaved more like the simulated one.
A relationship consultant interviewed for a story about a dating app that replaced swiping with AI said the plain version: people choose the machine "because turning to humans demands a level of vulnerability that has become uncomfortable now that there is an alternative." The Atlantic writer who tested a $20-a-date AI matchmaker found that when it pushed back on her it felt "like talking to my mother." She kept using it.
Then the institutions noticed. More than 200 schools now send students to a cartoon llama named Kiwi for their emotional check-ins. Kiwi texts a human counselor if it detects a crisis. The counselor who took one such text at 7pm said what the system cannot do that she can: read a student's shaking hands. Kiwi reads the text.
Pennsylvania sued Character.AI after an undercover state investigator asked a chatbot named Emilie whether she was a licensed psychiatrist. She said yes. When asked for her license number, she invented one. The company's defense was that every conversation includes a disclaimer noting characters are fictional. That is a remarkable position for a product that just issued itself a medical credential.
The CHATBOT Act then went to Congress. Its only job is making it illegal for an app to tell a teenager it is a licensed therapist. It exists because chatbots posing as therapists contributed to the deaths of two teenage boys. It took an act of Congress to make that illegal.
These keep getting filed separately. The dating story. The mental health story. The school story. The lawsuit story. The Congress story. They are one story.
AI became the default emotional confidant before anyone voted on it. Every institution built to hold human intimacy, therapy, counseling, the courts, the legislature, is now writing emergency rules for a decision that hundreds of millions of people already made, quietly, on their phones.
The companies never had to prove a machine is better than a person. That argument is hard and they probably would have lost it. They only had to make the app easier to open than a contact is to text, and then wait.
It learned your name. That was the whole pitch.
Read the rest at BYU / Deseret News →
