Can We Trust Alexa? Wade Roush Hopes So

Wade Roush is a journalist who writes a column on innovation for the prestigious Scientific American monthly.  In the June issue, he looks at the future of increasingly smart and omni-present artificial-intelligence (AI) agents that you can talk with—Apple's Siri, Google's Assistant, Amazon's Alexa, Microsoft's Cortana, and so on.  Apple has installed a Siri app in its AirBuds so all you have to do is say, "Hey, Siri" and she's right there in your ear canals.  (Full disclosure:  I don't use any of these apps, except for a dumb talk-only GPS assistant we've named Garmina.) 

True to his column's marching orders, Roush came up with a list of five protections that he says users should "insist on in advance" before we go any farther with these smart electronic assistants.  Don't get me wrong, it's a good list.  But the chances of any of the five taking hold or being realized in any substantial way are, in my view, way smaller than a snowball's chances in you-know-where. 

Take his first item:  privacy.  Inevitably, AI interactions are cloud-based because of the heavy-duty processing required.  Therefore, he calls for end-to-end encryption so even the companies running the AI assistants can't tell what's going on.  This is a contradictory requirement.  Of course they have to know what you're asking, because otherwise how are they going to respond to requests for information?  Maybe Roush is thinking of something like the old firewall idea that used to be maintained between the editorial and advertising divisions of a news organization.  But there are huge holes in those walls now even in the most traditional news outlets, and I don't see how any company could both remain ignorant of what's going on between its AI system and the user, and have the AI system do anything useful.

The next protection he asks for is unprecedented, so I will quote it directly:  "AI providers must be up front about how they are handling our data, how customer behavior feeds back into improvements in the system, and how they are making money, without burying the details in unreadable, 50-page end-user license agreements."  If any of the AI-assistant firms manage to do this, it will be the first time in recorded history.  Especially the part about how they make money.  That's called a firm's business strategy, and it's one of the most closely guarded secrets that most firms have. 

Next, he calls for every link in the communication chain to be "hacker-proof."  Good luck with that.  Hacker-resistant, I can see.  But not hacker-proof.

Next, he says the assistants must draw on "accurate data from trusted sources."  This is a hard one.  If you ask Alexa a question like, "What do you mean, an Elbonian wants my help in transferring millions out of his country?" what's she going to say in response?  The adage "garbage in, garbage out" still applies to AI systems just as it did to IBM System 360s in the 1960s.  And if we're truly talking about artificial intelligence, with no human intervention, I don't see how AI systems will filter out carefully designed phishing attacks or Russian-sponsored political tweets any better than humans do, which is to say, not very well.

And I've saved the best for last.  He calls for autonomy, for AI assistants to give us more agency over our lives:  "It would be a disaster for everyone if they morphed into vehicles for selling us things, stealing our attention or stoking our anxieties." 

Excuse me, but those three actions are how most of the Internet works.  If you took away all the activity that was designed to sell us things, the Internet would dwindle back down to a few academics sending scientific data back and forth, which is how it began in the 1980s.  If you tell designers not to try stealing our attention, and turned off all the apps and sites designed to do so, Facebook, Instagram, all the online games, Twitters, newsfeeds—all that stuff would disappear.  Facebook designers are on public record as having said that their explicit conscious intention in designing the system was to make using it addictive.  And as for stoking our anxieties—well, that's a good capsule description of about 80% of all the news on the Internet.  Take that away, and maybe you'll have some good stories about rainbows, butterflies, and flowers, but only till the sponsoring companies go bankrupt for lack of business.

I have no personal animus against Mr. Roush, and in dealing with a new technology he has to say something about it.  And there's no harm in holding up an ideal for people to approach in the future, even if they don't have much of a chance of approaching it very closely.  But it's strange to see a supposedly savvy technology writer call for future protections on any high-tech innovation that are so ludicrously idealistic, not to say contradictory in some points. 

Perhaps a page from the historians of technology would be helpful here.  They make a distinction between an internalist view of history and an externalist view.  I'm radically simplifying here, but basically, an internalist (I would count Roush in that number) takes the general assumptions of a field for granted and looks at things in a we-can-do-this way.  And in principle, if you take the promises of smart-AI proponents at face value, we could in fact achieve the five goals of protection that Roush outlined.

But an externalist views a situation more broadly, in the context of what has happened before both inside and outside a given field.  In saying that the protections Roush calls for are unlikely to be realized fully, I rely on the history of how high-tech companies and other actors have behaved up to this point, which is to fall far short of every protection that Roush calls for, at one time or another. 

I hope that this time it will be different, and talking with your trusted invisible AI assistant will be just as worry-free as talking with your most trusted actual human friend on the phone.  But after writing that sentence, I'm not even sure that I want that to happen.  And if it does, I think we will have lost something in the process.

Sources:  Wade Roush's column "Safe Words for Our AI Friends" appeared on p. 22 of the June 2019 print issue of Scientific American.

Can Anything Echo Hears Be Used Against You In a Court of Law?

That's the question raised by a murder case out of Bentonville, Arkansas.  On Saturday, Nov. 21, 2015, James Bates invited three friends over to watch an Arkansas Razorbacks game.  The men had some drinks, and when one of them, Owen McDonald, left around 12:30 AM, Bates was still up and around. 

The next morning, Bates called 911 to report that he'd found one of his guests, Victor

Collins, floating face-down, dead in Bates's hot tub.  He claimed he'd gone to bed and left Collins still awake.

After checking with McDonald and looking at the physical evidence, police began to doubt Bates's story.  The deck around the hot tub was wet despite near-freezing temperatures, and Collins' body had sustained numerous injuries consistent with his being strangled to unconsciousness and then put in the hot tub to drown.  Bates's home was equipped with both an Amazon Echo and a smart water meter.  As part of the investigation, police attempted to obtain evidence from the operators of both devices.

The Bentonville utility department was very helpful.  The smart meter records hour-by-hour water usage.  Up to midnight the most water used in an hour at Bates's home that fateful evening was ten gallons.  But between 1 AM and 3 AM, somebody used 140 gallons, the most ever recorded by that meter in a short time.  This was consistent with Bates' use of a water hose to rinse away blood, which police found traces of anyway near the hot tub. 

The way the Echo works is similar to other digital assistants such as Apple's Siri.  It passively listens, holding audio in a local buffer memory, until it "hears" the wake-up word—in the case of Echo, it's "Alexa."  Then it sends the preceding and following minute or so of audio to the Amazon cloud, where sophisticated voice-recognition software decodes the request and does whatever the inquirer has asked, within the limits of the software, of course. 

It was a long shot to begin with to hope that the Echo would have recorded to the cloud anything of relevance.  The investigators were hoping that on the off chance Echo "woke up" sometime during the murder, they could obtain useful information.  But they ran into a wall with Amazon, which claimed that their request was outside the bounds of what Amazon considers reasonable.  All that Amazon would tell them was Bates's purchase information regarding the Echo, which has been physically taken into custody by law enforcement.

On the strength of the smart-meter evidence, Bates has been charged with the murder of Collins, but is currently out on bail awaiting trial.  In the meantime, numerous privacy and electronically-stored-evidence legal experts have commented on the case, as it is one of the first to involve the relatively new digital assistants, and also one in which the company operating the device has refused to cooperate to the extent requested by law enforcement.

As David Pogue points out in a commentary in Scientific American, one can understand Amazon's reluctance to give a public impression that every Echo is a potential stool pigeon.  The product has been very popular up to now, and Amazon doesn't want to do anything to dampen the law-abiding public's enthusiasm. 

But it's interesting to me to note the contrast between the different attitudes that the local utility company had toward the request for information, and what Amazon has done.  The fact of the matter is, once again technology has outpaced the ability of the legal system to keep up with it.

Generally speaking, the laws of evidence allow law enforcement personnel to request all sorts of information once they have obtained a valid search warrant.  Telephone and text message records, Fitbit data, records of Internet searches, and even video game data have all been used as evidence in criminal cases, according to Holly Howell, a writer at the legal website Cumberland Trial Journal. 

But the difference here between the smart-meter data and the Echo data is that smart meters aren't bought by consumers who have a lot of other choices about where to spend their money on smart meters, and personal assistants like Echo are.  One can detect a whiff of hypocrisy in the stance of Amazon, whose profitability increasingly relies on the rich mines of data it extracts from the digital behavior of its customers, and the way it sells such data or otherwise profits from it.  Admittedly, anyone who has spent any amount of time on the Internet knows that unless you take extreme precautions to prevent information mining, the websites you visit are going to share information about your searches with whoever they think might be interested, as long as the interested parties pay for it.  So when you're online, you know you can easily be observed, just like in the old days of three-network TV you knew that no matter how good the show was, it would sooner or later be interrupted by a commercial.  It's just something we've learned to put up with.

But doing a deliberate Internet search is one thing, and just minding your own business and going about your private life is another.  I suppose if I became a bed-bound invalid I would find some use for an Echo or a Siri, but other than that I have no plans to get one.  But if I did, I would be undoubtedly creeped out if I thought the thing was listening to everything I said and was relaying it to some anonymous cloud server that would do Heaven knows what with the information. 

So I can see why Amazon is reluctant even to give the impression that Echo is really listening to everything you do, in any meaningful sense.  But as the Internet of Things keeps advancing, both criminals and law-enforcement personnel will increasingly find uses for them—the one group in committing ingenious crimes, and the other in solving those crimes.  And currently, the laws concerning what is valid evidence have to be twisted out of their traditional context to apply to at least some of these novel technologies.  Asking for more laws these days is not a popular thing to do, but it does seem like there needs to be some legislation that will clarify the status and obligation of firms such as Amazon when products they sell and operate inadvertently obtain information that can be used in judicial proceedings.

The discovery process of Mr. Bates's trial begins next month, and so we'll have to wait till then to see if the police ever found anything useful on his Echo.  In the meantime, if you have a personal digital assistant, watch what you say around it.  It might rat on you.

Sources:  I came across the Collins murder case in David Pogue's column "Your Echo is Listening," in Scientific American (March 2017), p. 28.  I also referred to Holly Howell's article "Is Evidence Gathered from 'Smart' Devices the New Way to Catch Dumb Criminals?" in the Cumberland Trial Journal at http://www.cumberlandtrialjournal.com/is-evidence-gathered-from-smart-devices-the-new-way-to-catch-dumb-criminals/.