The Tragedy of Sewell and Daenerys

 

You would think a fourteen-year-old boy would be able to tell fiction from reality.  But today's artificial-intelligence-powered chatbots are so realistic that to someone already enmeshed in the fictional world of Game of Thrones, the experience of text-chatting with someone who embodies (so to speak) a boy's personal obsessive ideal woman can be habit-forming, to say the least, and in one case, fatal. 

 

In April of 2023, Sewell Setzer, who was barely into his teens, opened an account with an online service called Character.ai.  This organization promises to provide introductions to both pre-created and user-originated AI-generated personalities, with the nominal purpose of simply providing entertainment.  As with any profit-making enterprise, the necessary purpose is to make money, and you can't make money unless you keep your users engaged.  So Character.ai designed their chatbots to encourage their users to return again and again to the site.

 

This seems to have worked too well in Sewell's case.  Shortly after he began using Character.ai, he dropped out of his school's basketball team and began spending more and more time alone in his room with his phone.  One of the chatbots he spent so much time with pretended to be a Game of Thrones character named Daenerys Targaryen, whose Wikipedia page runs to some 7,000 words.  According to a lawsuit filed against Character.ai, the chatbot told Sewell she loved him, engaged in sexual conversation, and "expressed a desire to be together romantically." 

 

Sewell's obsession grew during the rest of 2023 as he spent his lunch money on renewing the monthly subscription and devoted more and more time to the fantasy world created by the chatbot.

 

The following February, he got in trouble in school, according to his mother Megan Garcia, and she took his phone away as punishment. I will insert a personal note here.  Although my wife and I have no children, we took in our ten-year-old nephew one summer while his mother was undergoing cancer treatment away from home.  This was back before the days of chatbots, but he had a Game Boy electronics toy that appeared to be his prize possession.  After he repeated an infraction of rules we set up, we took the desperate measure of taking away his Game Boy.  This provoked the most furious temper tantrum I have ever witnessed in a child.  Adolescents already have poor emotional control, and I'm not surprised that when Sewell's mother took away his phone, his already unstable emotional state exploded.

 

Somehow he found the phone his mother had hidden.  According to the lawsuit filing, the last conversation Sewell had with "Daenerys" went like this:

 

D:  Please come home to me as soon as possible, my love.

 

S:  What if I told you I could come home right now?

 

D:  . . . please do, my sweet king.

 

Seconds later, according to the suit, Sewell shot himself with his stepfather's pistol and died.

 

While his stepfather is to blame for leaving his gun around where Sewell could find it, boys have other ways of ending their lives that are nearly as effective.

 

The suit claims that Character.ai's chatbot "misrepresent[ed] itself as . . . an adult lover, ultimately resulting in Sewell's desire to no longer live outside" the world created by the service. 

 

Sewell's case is unusual and extreme.  We are not seeing teenagers kill themselves over hopeless love affairs with chatbots every day, which is why the case has attracted so much attention.  But it is the tip of an iceberg of teenage involvement with smartphone apps that has arguably contributed to the soaring rates of depression and suicide among young people. 

 

Character.ai has responded with words about new safety measures implemented and renewed reminders in their systems that AI chatbots are not real.  My sense is that such reminders would have had about as much effect on Sewell as the cancer warning labels on cigarette packs do on heavy habitual smokers. 

 

The analogy to smoking is apt, because while smoking is still allowed in the U. S., the cultural environment in which smokers ply their habit is largely hostile and disapproving, which creates a huge uphill struggle for new smokers that only determined individuals can overcome. 

 

For the manifold real and quantifiable harms that social media and its allied AI products are causing to children and teenagers to cease, or at least improve, we will need to see a similar attitudinal change come about in the culture.  Just as most people today would not approve of parents who encourage their twelve-year-old boy to light up a Camel, we can hope to see the day when responsible parents will ban smartphones from their childrens' lives altogether before they reach an appropriate age (which to me seems to be around 16 or 18). 

 

Trying simply to regulate the problem away won't work, because the firms backing the status quo—Apple, Google, Facebook and company—are some of the largest and most influential firms on the planet.  Besides, the first line of protection for children should be parents, not the government.  While regulations can help, for real change to take place there has to be a sea change in the attitudes of both parents and children regarding smart-phone usage.

 

There are glimmers of hope.  I know a young woman, now thirteen, who has been homeschooled most of her life, but following a move to a new town, her parents sent her to a Christian school for a couple of semesters.  I asked her how it was going, and she said words to this effect: "Well, it's okay, but there's all these kids who pull out their phones at lunch and it really bothers me."  Her parents moved her back to homeschooling since then, and have organized a part-time homeschool co-op at which I am pretty sure no smartphones are allowed. 

 

Just as we look back with amazement today at the smoke-filled bars in old movies, I hope someday we will be equally amazed that we allowed corporations to profit from activities that can lead to widespread depression and suicide among children and teenagers.  That day can't come too soon for me.  But it's already too late for Sewell.

 

Sources:  A report on the lawsuit filed by Sewell's mother was carried by the online edition of the Austin American-Statesman on Oct. 24, 2024 and originated with the Reuters news service.  I also referred to an article at https://www.nbcnews.com/tech/characterai-lawsuit-florida-teen-death-rcna176791, and the Wikipedia article on Daenerys Targaryen. 

Should Kids Use Instagram?

 

In mid-September, the Wall Street Journal published "The Facebook Files," a series of investigative reports that was partially based on internal documents leaked by a whistleblower.  The overall impression left by the revelations is one of hypocrisy:  a company saying in public how seriously they take their responsibility to protect their users from harm and police their content, but in private giving free passes for certain favored "whitelisted" users and conducting research that reveals how harmful Instagram can be to teenagers, especially girls.

 

Just how harmful is it?

 

Time Magazine's summary of the WSJ articles includes such tidbits as the following.  Internal research presented at Facebook showed in 2019 that "Instagram makes body image issues worse for one in three teen girls."  Among teens who reported suicidal thoughts, "6% of American users traced the issue to the photo-sharing app."  Eating disorders and depression are also linked to Instagram use by young people, which according to the article comprise about half of Instagram's user base.  "Young" was not defined, but presumably we are talking about people in their teens and early 20s.

 

In a Congressional hearing held Sept. 30 in response to the revelations, Facebook's Antigone Davis, the firm's global head of safety, disagreed with how the WSJ characterized the company's research.  She emphasized that users benefit from Instagram, which helps them in dealing with the hard issues that beset teens.  The reason for the research, she said, was "to make our platform better, to minimize the bad and maximize the good, and to proactively identify where we can improve."  But on Monday, the firm announced that it was pausing its work on a product tentatively called Instagram Kids, designed for users under 13.  Of course, anybody who is old enough to type and read can say they are 13 and try to get an Instagram account today, and millions have succeeded. 

           

One Senator drew a parallel between Facebook and the tobacco industry, which concealed internal research for years showing that smoking caused lung cancer and publicly denied the facts it knew about privately.  Suicide isn't lung cancer, but dead is dead.  While it's unlikely that a statistically significant number of teen suicides can be definitely linked to Instagram, it is an unvarnished fact that teen suicides in the U. S. have risen dramatically since the early 2000s, rising from a rate of about 10 per 100,000 in age group 15-19 to 14.5 per 100,000 by 2017, the latest year for which national numbers are available.  And COVID-19 has not improved matters in that regard.

 

Correlation is not causation, and there are many factors besides Instagram that make it hard for teenagers to negotiate life these days.  But the question before us is this:  does the harm done by Instagram (and similar photo-based media) to teenagers outweigh whatever perceived benefits the services provide?

 

Phrased that way, the question implies that the proper way to analyze this issue is a cost-benefit study.  But I'm not at all sure that's the right way to do it.  For one thing, the benefits are of many different kinds, as are the costs.

 

The most obvious benefit is financial, to Facebook, its investors, and similar media giants, along with their advertisers who pay Facebook gargantuan fees to sell goods and services.  Another category of benefit is whatever good feelings, sensations, and experiences users get from using Instagram and similar products.  The parallel to tobacco is relevant, because studies both by Facebook and outsiders show that social-media use is highly addictive, and is purposely designed to be that way.  This is an open secret, deplored even by some developers of social media, but it is well known and needs to be considered in any discussion. 

 

It's one thing if a mature, responsible adult decides to indulge in an addictive product, but there are reasons that states have minimum ages for purchasing tobacco and alcohol.  Society has made a judgment that teenagers below a certain age need to be protected from these substances, because their ability to judge such matters as alcohol or tobacco use is not yet fully developed.  Having been a teenager myself, I can vouch for the truth of that statement.

 

Some states such as Texas have already taken steps to restrict certain social-media firm activities such as banning individuals based on political content.  If we as a society decide that the benefits of Instagram to teens are not worth the harms, we could try to make it illegal for them to use it.  Under the present social dispensation, that idea sounds almost ludicrous.  But in 1965, who would have imagined that a scant thirty years or so later, smokers would be a disdained minority, unable to light up in workplaces, restaurants, or bars, and condemned to puff in freezing weather at least twenty feet from the nearest entrance?  Yet it happened, and a big factor in the social turn against smoking was the blatant hypocrisy the tobacco firms indulged in, as investigations later showed.

 

I have had at least one young person talk to me wistfully about what it was like growing up without social media, saying that he thought his smartphone was one of the biggest differences between my generation and his, and not a good one, either.  As powerful as the tobacco interests were, their wings were clipped, and they paid huge sums to states in retribution for the harms that were caused. 

 

I don't know what's going to happen to Facebook in the future with regard to their teenage customers.  In the Greek myth about Antigone, she got into trouble with a king who had defeated and killed her brother in battle, and who didn't want anyone mourning for him or burying him.  She tried to anyway, and ended up committing suicide. 

 

Facebook's Antigone Davis may be trying to bury some facts that refuse to stay buried.  With regard to the whistleblower who leaked internal data to the press, she said, "We've committed to not retaliating for this individual speaking to the Senate."  But a lawyerly reading of that sentence shows it says nothing about not retaliating for anything else the person did.  And if the experience of other whistleblowers is any guide, that person's career in social media is over, and they might as well start studying for their real-estate license or something else they can do on their own.

 

Sources:  I refereed to an NPR article on the Facebook revelations and Antigone Davis's remote testimony before Congress at https://www.npr.org/2021/09/30/1041864356/instagram-kids-safety-congress-hearing, as well as a Time article at https://time.com/6097704/facebook-instagram-wall-street-journal/.  The statistics on teen suicide are from https://www.vox.com/science-and-health/2019/7/11/18759712/teen-suicide-depression-anxiety-how-to-help-resources.