The fooled-by-data effect

by Leland Teschler, Executive Editor

About a year ago, everyone in my office got a fitness watch. This was at the height of the IoT wearables craze. Fitness watches were supposed to do wonderful things such as measure steps taken, distance traveled, stairs climbed, active minutes, elapsed time, calories burned, pace, elevation and improve sleep quality.

An informal poll of my colleagues revealed that after about six months, roughly half those fitness watches sat unused in bedroom drawers. The problem for most of them was data overload. They just weren't all that interested in continually monitoring their calorie burn or steps. And all the data the watch generated seemed overwhelming.

It might be just as well. There are those who argue that having too much data about medical conditions can be more harmful than having too little. One in that camp is Nassim Nicholas Taleb. Taleb is a professor of risk engineering at N.Y. University's Polytechnic Institute. But he made his fortune trading options on equities and futures contracts. His strategy was to buy cheap options that paid off big when widely unanticipated events occurred—he is said to have cleaned up in the dot-com crash around Y2K. He outlined his philosophy in a best-seller he authored called Fooled By Randomness.

Taleb has little good to say about the Big Data coming out of IoT endeavors. Judging by what he said in his most recent book, Antifragile, he thinks even less of the reams of data about vital signs generated by items such as fitness watches. "If you want to accelerate someone's death, give him a personal doctor. I don't mean provide him with a bad doctor: just pay for him to choose his own. Any doctor will do," wrote Taleb. "Did you ever wonder why heads of state and wealthy people with access to all this medical care die just as easily as regular persons? Well, it looks like this is because of overmedication and excessive medical care," he posits.

Taleb's criticism of Big Data, and particularly big medical data, is that most of it should be ignored. The reason: Medical data is noisy. Looking at data more frequently is likely to increase the chance you are merely looking at the noise in a signal rather than at the signal itself. So it is easy to end up reacting to spurious readings.

"More data means more information, perhaps, but it also means more false information. … Someone looking at history from the vantage point of a library will necessarily find many more spurious relationships than one who sees matters in the making, in the usual sequences one observes in real life," Taleb said.

The IoT is worsening the situation. The same kind of noisiness that describes medical data can be found in numerous other disciplines that are increasingly the subject of analytics made possible by pervasive connected sensors. "The fooled-by-data effect is accelerating," wrote Taleb. "There is a nasty phenomenon called Big Data in which researchers have brought cherry picking to an industrial level. Modernity provides too many variables (but too little data per variable), and the spurious relationships grow much, much faster than real information, as noise is convex and information is concave," he said.

Those are points to consider before betting your business on an IoT Big Data initiative.