What's the Big Deal About Big Data?


Big Data -- that is, the amassing and analysis of gigantic amounts of raw information from a wide range of sources -- might sound scary to people who imagine government supercomputers sifting through phone call records and toll-booth surveillance video footage to track their daily movements, or malevolent corporate giants siphoning up details of their web browsing and credit card purchases to subtly manipulate them into buying certain products. 

But not all of our civilization's vast reservoir of personal data is being gathered by the National Security Agency or Amazon.com.

Mouth-Watering Maps Continue to Mine Big Data

Increasing numbers of people are gathering amassing vast amounts of information about themselves. Case in point: Economist data editor Ken Cukier, co-author of the 2013 book “Big Data: A Revolution That Will Transform How We Live, Work, and Think,” who wears a small device on his wrist that measures health-related data, ranging from how many steps he takes each day to how much sleep he gets.

Exercise can be as good or better than prescription medication at healing the body.

"If you wanted to measure your sleep patterns 15 years ago, you had to go to a university lab and have them hook up $100,000 worth of sensors and equipment," he explained. "Today, you can do it with a band on your wrist for $100."

Eventually, Cukier hopes that his data, aggregated with other device wearers, will provide a precise answer to the question of how exercise affects sleep. "If you exercise more, do you sleep better, or worse? Or is it that you sleep less when you exercise, but it's more profound sleep? Before, we didn't really have enough information to pose the right questions. But now we will."

Cukier belongs to a seemingly growing group of DIY collectors of Big Data about themselves, sometimes called self-trackers or members of something called the quantified self (QA) movement.

The Pew Internet and American Life Project reports that 7 percent of U.S. adults use a smart phone app to continuously gather and tabulate some sort of health-related data about themselves, such as diet, blood pressure, sleep patterns, or frequency of headaches. But within that group there's a smaller, undefined slice who use phones, computers, wearable sensors and sophisticated software apps to amass minute data on everything from their daily movements, exercise routines and social contacts to fluctuations in their emotional states and how many cups of iced coffee they drink.

By analyzing the data and turning it into visual charts and graphs, some are striving to fine-tune their health, boost their work productivity, and even improve their romantic relationships. Others simply are indulging their self-curiosity, and hoping to understand themselves a little better.

Harvard University social psychologist Justin J. Lehmiller thinks that the personal Big Data movement is an outgrowth of a natural human tendency to engage in self-perception. “We look to our own behaviors to determine who we are,” he said. “We can infer how we feel about something, based upon how we behave toward it.”

Already, organizations and companies are springing up to cater to personal Big Data users. One example is Quantified Self, founded in 2008 by Wired magazine journalists Kevin Kelly and Gary Wolf. QS is holding an annual conference in San Francisco this week in which 400 self-trackers, makers of tools for Big Data, and researchers will gather to hear presentations with titles such as “The Journey to Existence Mapping,”  “Are You What You Eat or What You Buy?” and “What I Learned from 30 Days of Continuous ECG.”  The last is a talk by a self-tracker who wore a cardiac monitor continuously for a month, so that she could gather a data set about her body’s physiological patterns.

In a TED talk in 2012, QS co-founder Wolf explained that amassing and analyzing personal data is useful “when we reflect, learn, remember and want to improve ... if we want to act more effectively in the world, we have to get to know ourselves better.”

Recommended for you