I’ll Take My AI with a Side of Skepticism, Please

For all the hype around AI, we are starting to get some objective measures that suggest many of these tools are not all they’re cracked up to be. And if you’re wondering what an AI story is doing in a newsletter focused on science and medicine, consider that biomedical applications were some of the earliest targets for AI. When IBM Watson came on the scene more than a decade ago, for example, the company’s first commercial focus was on improving healthcare decisions for lung cancer. After billions of dollars in development and various attempts at demonstrating utility for DNA sequencing, diagnosing cancer, and other applications related to the life sciences (among many other fields), IBM ultimately sold the division at a steep loss.

Recently, scientists in Europe performed a fascinating study on the use of AI by medical experts. They focused on endoscopists and everybody’s favorite imaging test: the colonoscopy. AI tools are now being marketed as assistants that can help endoscopists be even more accurate in their interpretation of colonoscopy results, such as for spotting polyps. This research team wanted to know what actually happens when endoscopists use these tools.

The results were unexpected. After three months of using AI tools, endoscopists were measurably worse at detecting benign growths in colonoscopies on their own than they were before. For this study, scientists looked at colonoscopy results three months before the AI tools were introduced, and then again three months after; they reviewed data from nearly 1,500 patients at four endoscopy centers in Poland whose results were interpreted without AI tools. Various statistical analyses showed that it was continuous exposure to AI tools, rather than other factors, that were most associated with the decline in endoscopists’ performance. Researchers chalked up this finding to an over-reliance on AI support.

One of the scientists in the study, Marcin Romańczyk from the Medical University of Silesia, told MedPage Today, “Imagine that you want to travel anywhere, and you’re unable to use Google Maps. … We call it the Google Maps effect. We try to get somewhere, and it’s impossible to use a regular map. It works very similarly.”

Years ago I asked a friend why she’d taken a predictably slow route to my house instead of the route she knew would be much faster, and she blamed GPS (and herself), saying that she had abdicated her executive function to the GPS. With all AI tools, we can turn to them for information while critically considering what they tell us, or we can turn off our own brains and blindly follow what they say. It seems the colonoscopy study supports the idea that many of us do the latter.

In another study, this one outside the realm of healthcare, researchers assessed how AI tools affect software developers who use them. They ran a randomized trial with experienced developers who had a fair amount of familiarity with AI tools. After conducting a series of coding tasks, the developers were asked how the use of AI had made a difference; on average, they estimated that it sped up their work by 20%. In fact, the developers took 19% longer to complete tasks when using AI than they did on their own.

I’ll admit that I laughed out loud when I read that study. When I think of how much time I spend each day swatting away unwanted offers to let AI help me — to post a note on LinkedIn, to read a PDF, to search my email — it struck me as self-evident that these tools are taking time away from us rather than giving it back. But it’s nice to have a little evidence to back that up.

In one final study, a team at MIT looked very broadly at generative AI tools and concluded that “95% of organizations are getting zero return” from the tens of billions of dollars they have collectively invested in enterprise-grade products. While there might be some benefit to individual productivity from these tools, they report, there is “no measurable P&L impact” for most companies.

None of this is to say there is not, or will not be, any benefit from AI. But I do think we’d all benefit from thinking more critically about these tools and how much we choose to rely on them.