What is the big brother problem?

The “Big Brother Problem” is a timely, difficult and sweeping topic, covering digital surveillance by both public and private actors and its implications for human rights. I’ll be moderating a session on the subject at the Annual Meeting in Davos this week, and I thought I’d share my thoughts on both process and substance as I prepare for it.

We have one hour on a very broad topic, including audience and online questions, with six participants. How to use that hour to make progress? Let’s start with the lineup. Since there won’t be prepared remarks, I’ll be looking for lead-off questions that allow each participant to highlight what he or she finds most important, while also answering something that might be a bit off the beaten path.

There’s Salil Shetty, Secretary-General of the Nobel Peace Prize-winning Amnesty International, which has been a beacon for human rights since its inception in 1961. For someone concerned about human rights, it may not make much sense to try to rank abuses: too often the observation that rights are abused “worse” elsewhere is an excuse for a particular government not to clean up its own practices. But it may be fair to ask a general question about how informational privacy ranks next to possibly more fundamental concerns about physical integrity – like freedom from arbitrary detention or torture.

Indeed, surveillance might be best understood metaphorically, as a precursor chemical to an undesirable concoction: spying can facilitate human rights abuses. But it can also be a powerful tool for good – particularly in places where there’s no one on the ground to understand what’s going on. Intelligence agencies around the world may be able to gain perspective on events in, say, Syria, thanks to the tools and practices that have been making global headlines since last June.

If surveillance is to be cut back, what restrictions do we impose on the collection end, the attempt to raise the cost of knowing something secret across the board? What bulwarks would be sufficient to safeguard human rights and dignity? (For example, someone against nuclear weapons proliferation might focus on a regime limiting the use and transfer of nuclear arms; their elimination entirely; and at the end of the spectrum a restriction on peaceful nuclear power for fear that enrichment technologies are just too easily dual-purpose.)

Here, too, it may be helpful to explore what one might want in theory, and what is judged politically attainable. Is there room for compromise and negotiation here, or is the role of an organization like Amnesty to anchor and stick to what it perceives as the purest of truths?

Next up is US Senator Patrick Leahy. He is no stranger to an issue likely to be prominent in the discussion: the activities of the US National Security Agency (NSA) that have been the subject of ongoing leaks, originating most recently with Edward Snowden, and the topic of a US presidential address covering reform last week. Leahy chairs the Senate Judiciary Committee, which has been a focus for debate over legislation such as the Patriot Act, and has often weighed in as one of the chamber’s civil libertarians. He’s also a former prosecutor.

It will be interesting to see how he views the situation: will he lean towards “I told you so” on certain activities having gone too far – perhaps because authorized by US law, from which he dissented, rather than by a rogue agency – or will he emphasize a realpolitik defence of countries gathering whatever information they can to protect their security (something every country does)? Many US officials have expressed bemusement that the NSA has been in the spotlight so singularly, when agencies the world over are thought to conduct parallel activities, and few, if any, states can offer a clear map for satisfying (much less transparent) intelligence oversight.

It might be good to ask Leahy if he thinks there has been an impact not only on the global reputation of the US government, but on US business. Should the fact that legal process can yield so much intelligence from “local” companies with significant global customer bases be a reason for forbearance on collection from them, lest the customer base go elsewhere (even understanding that elsewhere is still somewhere: if not the US there will be some other government empowered by the new presence of activities in its jurisdiction).

At the introduction of the business question, this might be a good time to turn to Augie Fabela and Brad Smith. Fabela is the former chairman of VimpelCom, a worldwide telecommunications provider that originated in Russia in the early 90s and is now based in the Netherlands, with more than 200 million mobile customers worldwide. Telecoms providers are natural places to seek to spy, and indeed some governments – including in Europe – have imposed data retention requirements upon them, so as to facilitate law enforcement or national security-based investigations on subjects that have not yet been conceived. How should we think about those requirements? Fabela may have a particularly interesting view, as since leaving VimpelCom he’s honed an interest in public safety and law enforcement: in 2012 he was appointed to the rank of sheriff commander in Cook County, Illinois, USA, assisting with the restructuring of its intelligence centre.

Brad Smith is the general counsel of Microsoft, and has been with the company for more than 20 years.  He has contended with requests from law enforcement and intelligence agencies around the world, and will no doubt have strong and long-thought-through views on where to go from here. You can see some of his thinking in an interview he gave to Corporate Counsel magazine, where he says it’s difficult to ask companies to step in when governments fall short. Governments are, he says, the “ultimate decision-makers”. When we think of checks and balances in the gathering and use of data, what role, if any, can companies play versus public authorities exercising their power?

Orit Gadiesh is the chairman of global management consulting firm Bain & Company. She has a background in intelligence policy from a time before the Internet went mainstream, along with corresponding digital surveillance, but she may remind us that the Big Brother Problem, contra Orwell, is not limited to the state. Companies gather and maintain an enormous amount of data about us, and moreover have incentive – particularly if advertiser-supported – to use that data to influence our choices, even if those choices are as prosaic as what airline to fly or pet food to buy.

As the cost of data gathering and processing goes down, what was formerly the province of massively funded state intelligence operations can become the territory of “mere” companies – and perhaps, ultimately, individuals. Gadiesh’s participation on the panel may help the exploration include private as well as public intelligence policy. Can we count on use restrictions as a hedge against abuse, or must safety lie in more limited collection?

Finally, what will happen as the power of surveillance devolves from governments and companies to individuals themselves? This very session, in an acknowledgment of the realities of the early 21^st century, will be webcast live. Others labelled “off the record” may seem increasingly quaint as anyone in the room is in a position to record – or livestream – what’s taking place. How can we contend with this when ubiquitous data gathering can happen anywhere, not just at conferences?

Shyam Sankar may have some thoughts on that. He’s a director of Palantir Technologies, which, in its own words, offers “a suite of software applications for integrating, visualizing and analysing the world’s information”. Customers include the US government – which also invested in it early on. Palantir has made a point of emphasizing its claim that its data analytics are geared to recognize and implement privacy protections. Sankar will no doubt have much to say about the relationship between the private and public sectors in data gathering and processing, and lately he’s been speaking a lot – including at TED – about “human-computer cooperation”, how algorithms alone aren’t the key to big data, but rather a “symbiosis” between person and machine.

There might be some solace in the centrality of people, rather than algorithms, to the infrastructure of surveillance: Big Brother ultimately will comprise people. This might make it fallible, yes, but constitutions and the rule of law were framed with fallible humans in mind. That may mean that the challenges posed by the ocean of data around us are difficult and new, but not entirely alien to what has come before, and for which there are models of successful vindication of our rights and dignity.

I hope you’ll join this session, whether in person or online.

Author: Jonathan Zittrain is a Professor of Law and Computer Science at Harvard University.

Image: Berlin’s Reichstag building is pictured though a flag depicting fugitive former NSA contractor Edward Snowden, 18 November 2013. REUTERS/Tobias Schwarz