Big data: the greater good or invasion of privacy?

Google maintains massive amounts of user data, and the "Do Not Track" icon of its browser has been shown not to function. Photograph: Uncredited/AP Spying on ordinary citizens' internet searches is usually considered politically unpalatable, especially if the government's at hand. Nobody is thrilled about social media applications like Facebook doing the same, yet many of us indulge anyway even if we are aware that our activity can be easily tracked. As we spill more and more of our secrets posting drinking sessions on Twitter and asking Google indiscreet questions, the kind we would never ask our closest confidants some say that there's a way to harness our online activity for the betterment of humanity, like spotting potential health threats. But is there also a downside to "big data" (the new term among information technology circles for the vast quantities of information now stored online, possibly forever) being opened up to researchers? Last week, scientists at Columbia and Stanford, led by Russ B Altman, published their analysis of 82m online searches, which were provided to them by Microsoft to learn about the symptoms and conditions of certain drugs. In an article written for the Journal of the American Medical Informatics Association, the team announced that they had uncovered an unexpected medical find: the combination of two drugs paroxetine, an antidepressant, and pravastatin, a cholesterol-lowering drug caused high blood sugar. Such an approach could well support the Sentinel Initiative, a US Food and Drug Administration (FDA) project meant to track the impact of drugs after they enter the marketplace or at least, that has been suggested by some, like John Markoff at the New York Times. Confusingly, however, the FBI has a new mega-database with exactly the same name Sentinel to track suspected criminals and terrorists. And yes, the FBI also wants to mine "social networks, including Facebook and Twitter, and immediately translate foreign language tweets into English" to follow what ordinary people are doing online. How similar are the tracking technologies used by medical researchers and security agencies? Well, they're really not that different. And they can be easily used against you, says Paul Ohm, an associate professor at the University of Colorado law school, who wrote a paper titled "Broken Promises of Privacy: Responding to the Surprising Failure of Anonymization", in which he explains that scientists have demonstrated that they can often "reidentify" or "de-anonymize" individuals hidden in "anonymous" data with astonishing ease. In another paper, this one for the Harvard Business Review, Ohm wrote:

"In my work, I've argued that these databases will grow to connect every individual to at least one closely guarded secret. This might be a secret about a medical condition, family history, or personal preference It is a secret that, if revealed, would cause more than embarrassment or shame; it would lead to serious, concrete, devastating harm."

Indeed, Jon Leibowitz, the chairman of the US Federal Trade Commission (FTC), recently coined the word "cyberazzi" to describe data companies that trawl the internet for information on consumers. Like paparazzi who stake out restaurants in Hollywood, and who snap pictures of celebrities in indiscreet situations, the cyberazzi stake out your web browsers and mobile phones to quietly harvest data on what you like, where you go and what kind of questions you ask. In addition to medical researchers and the security state, all this data is a gold mine for marketers. A fascinating Times feature about the statistician Andrew Pole and the scientist Andreas Weigend, who work for Target and Amazon, respectively, explained how retailers subtly track life-altering episodes, like pregnancy. Pole told the author:

"Just wait. We'll be sending you coupons for things you want before you even know you want them."

"It knows who you are. It knows where you live. It knows what you do," wrote another Times reporter, this time about Acxiom, the biggest company in the world of database marketing.

"It peers deeper into American life than the FBI or the IRS, or those prying digital eyes at Facebook and Google."

Acxiom is just one of many Corelogic, Datalogix, eBureau, ID Analytics, Intelius, Peekyou, Rapleaf and Recorded Future who sell data to Amazon and Target, who then pitch us products that we don't need, convincing us to waste our hard earned money. And a number of these companies are also quietly selling our information to government security agencies. So how does one ensure that big data is used in a way that is good for everyone? The policymakers working at the FTC recently offered a three-point plan privacy by design, simplified choice for businesses and consumers, and greater transparency. Ohm's advice?

"We need to slow things down, to give our institutions, individuals, and processes the time they need to find new and better solutions Do not push the privacy envelope. Companies that use personal information in ways that go well beyond the practices of their competitors risk crossing the line from responsible steward to reckless abuser of consumer privacy."

In theory, most browsers and website allow you to permanently opt out of online behavioral advertising, but they don't actually prevent tracking. (The "Do Not Track" button in Google Chrome doesn't actually work) In fact, the current rules allow websites to decide what to do with information. Much of this may change if the World Wide Web Consortium comes up with amenable rules, but for now, activists and database marketers have yet to find common ground. As Jonathan Mayer, a Stanford privacy researcher, told CNN:

"The advertisers have been extraordinarily obstructionist, raising the same issues over and over again, forcing new issues that were not on the agenda, adding new issues that have been closed, and launching personal attacks."

Until such an agreement is reached, I recommend reading an article by Stanton McCandlish, who put together a dozen ways to protect your online privacy. A more detailed and up-to-date guide from the Privacy Rights Clearinghouse can be found here.