In this long–read, Stephen Williams and Nathan Mladin discuss the fresh relevance of Shoshana Zuboff’s analysis of ‘surveillance capitalism’, focusing on the question of human autonomy. 07/10/2020

As we frantically try to come up with a vaccine for COVID–19, contact tracing capabilities are crucial for public health. Governments, whose ties to big tech run deep, are ditching their own projects and striking deals with the tech giants for track and trace solutions, while dismantling ethics oversight committees in the process. A mightier than ever Big Tech sector, working with governments drawn to the power of big data, makes Shoshana Zuboff’s analysis of The Age of Surveillance Capitalism even more urgent and important today, over a year on from its publication, than it was when it first hit the shelves.

What is surveillance capitalism?

Despite the length of her book (around 700 pages), Shoshana Zuboff, Emerita Professor at Harvard Business School, puts forward a simple and straightforward thesis. The three words in the title of the book say it all. We are dealing with something which marks (in fact, dominates) our age and not simply with one feature of the contemporary scene. Surveillance is the name of the gruesome game in town: big tech companies have both acquired frightening capabilities of surveillance, and are exercising them; Google is at the top of that game. How so?  By operating in accordance with the economic principles of laissez faire neo–liberal capitalism laid out by Friedrich Hayek. These principles have created a space in which advanced surveillance technology perpetrates behavioural control to the point of automation, so that human life is ordered to the interests of market profits.

Consider the following: a person’s call history, phone–charging patterns, facial features, voice inflections, medical records, all social media activity, including posts, clicks, likes, and shares, but also meta–data such as typing speed and mouse movements – more or less all the facets of modern human experience, online but increasingly offline (the line between the two becoming blurred with the Internet of Things and the spread of ‘smart’ devices). These are all tracked, aggregated as data and mined extensively with the use of sophisticated machine learning tools to yield increasingly accurate predictions about our behaviour – our movements, emotional states, what we’ll want and buy next. These predictions, or what Zuboff calls “predictive products”, are sold for guaranteed outcomes by surveillance capitalists on “behavioural future markets”, primarily to advertisers, but also to insurance and finance companies. The whole operation of predicting and ultimately modifying behaviour is profoundly opaque and takes place under the radar of human awareness. “They know us better than we know ourselves”, says Zuboff, pointing to the stark asymmetries of knowledge and power between surveillance capitalists and the population at large. An exaggeration? Not when we properly appreciate the power of billions of data points and machine learning capabilities.

Zuboff holds that this is all an unprecedented assault on democracy because surveillance capitalism aspires to possess our persons in all their rightful privacy. Autonomy, or self–determination, is the mainstay of our civilization, she argues – and it is under dire threat. The focus of the book is on the detail of how all this works, rather than on strategies of resistance, though Zuboff does issue an urgent call to arms: “Be the friction”, as she entitles the final section of her book. Her elaboration is sober and chilling. The infrastructure for predicting and controlling human behaviour for others’ commercial and political ends is orientated towards, and produces, what Zuboff calls “instrumentarian power”. She calls the regime that results “instrumentarianism”.

A new totalitarianism 

Although Zuboff draws a distinction between ‘instrumentarianism’ and totalitarianism, the end result of both seems to be the same: the erosion of human dignity and the overwhelming of society. She argues as follows. The politics (or rather anti–politics) which surveillance capitalism breeds leans heavily in a totalitarian direction: there is significant concentration of power – but this power is in the hands of a small corporate elite rather than the state (though China, we should note, is a striking exception). Such power, derived from wide knowledge asymmetries, enables population control which undermines human agency and dignity. Totalitarian power, Zuboff observes, was geared towards the production of human souls (“the new man”) and used a plethora of violent means for “re–education” (a chilling euphemism for abuse and torture). Instrumentarian power, by contrast, is supposedly indifferent to interiority, to the life of the soul. External behaviour is the frontier where the surveillance capitalist gaze stops. Zuboff explains: “Trained on measurable action, it only cares that whatever we do is accessible to its ever–evolving operations of rendition, calculation, modification, monetization, and control” (435). Unlike traditional totalitarianism, it can achieve its purposes without violence; when sophisticated and stealthy means of behaviour modification and control are available, who needs tanks and rifles? Furthermore, if totalitarianism was primarily a political project, ‘intrumentarianism’ is a market project powered by digital technology. But the differences between totalitarianism and instrumentarianism, which Zuboff is right to note, should not obscure the dark unity of purpose between the two: controlling and ultimately overwhelming society. 

Orwell’s “Big Brother” becomes, in Zuboff’s analysis, the “Big Other”, which stands for the “ubiquitous digital apparatus” that helps realise a vision of absolute behavioural control in ways that are aligned with the commercial and political interests of surveillance power.Under surveillance capitalism and the reign of instrumentarian power, democratic politics dissolves into computation. Machine–enabled certainty and algorithmic decision–making replace the messy, non–linear, “‘inefficient” practices of listening, negotiation and deliberation at the heart of democratic politics. When individuality and democratic participation are abolished or rendered insignificant, the sense of isolation and existential loneliness that ensues “stoke[s] the fires of totalitarian terror” (624). Autonomy, the mainstay of our civilization, is enlisted in combat with a new totalitarianism. 

Shoshana Zuboff writes extremely well, and if such serious material should ever be described as a pleasure to read, then this volume amply merits that description. It goes without saying that she is a highly–qualified commentator and analyst. Non–expert readers will find her volume well–documented, though it must be left to those who are as familiar as she is with the world of surveillance capitalism to comment on the accuracy and balance of her account. Tech critic Evgeny Morozov, for example, differs from Zuboff on the wider front, holding that she is too lenient with capitalism itself. He also criticizes her on the narrower front, for ignoring (among other things) the way data extraction, not just surveillance capitalism, is at the heart of the digital economy more broadly, as its engine of economic value.[1]  Whatever the merit of his criticism and whatever omissions there are in Zuboff’s account, we want to concentrate specifically on the central issue, the heart of the civilizational matter, as Zuboff sees it: the question of autonomy. 

Autonomy: the bulwark of civilization? 

For Zuboff, autonomy is self–determination. A telling passage reveals the heart of her concern:

No matter how much is taken from me, this inward freedom to create meaning remains my ultimate sanctuary. Jean–Paul Sartre writes that “freedom is nothing but the existence of our will,” and he elaborates: “Actually it is not enough to will; it is necessary to will to will.” This rising up of the will to will is the inner act that secures us as autonomous beings who project choice into the world and exercise the qualities of self–determining moral judgment that are civilization’s necessary and final bulwark (291).

This is what surveillance capitalism threatens: “self–determining moral judgment”, which is “civilization’s necessary and final bulwark”; judgment is attained, and civilization is sustained, by the will to will; the fruit of it all is freedom. For Zuboff, the most important principle is that each person should have the freedom to choose how to live and what to believe. Surveillance capitalism removes this choice by filtering the knowledge and options available to us. She emphasises this principle elsewhere. On Vox’s Recode Decode podcast, for example, she talks about surveillance capitalism as threatening “moral autonomy” and “individual sovereignty”, which she identifies as “the elements that are the constituent forces of democracy”. 

Jean–Paul Sartre, to whom Zuboff appeals in the quote above, is not casually imported into this discussion. Zuboff refers to “the Sartrean age” (439), pitting his thought against the behaviourist philosophy of the renowned Harvard psychologist, B. F. Skinner. Skinner thought that belief in human freedom was illusory and that behavioural control was the remedy for social ills. Surveillance capitalism is the technological implementation of his thought. In Zuboff’s account, Sartre and Skinner are pitted against each other at the very heart of civilizational contest over freedom of the will. This is not one contest among many: it is the ideological contest of our time.

Skinner may deserve the opprobrium which he gets from Zuboff, but Sartre should not command her allegiance. To put it bluntly, she is profoundly mistaken in saying that “self–determining moral judgment” is the bulwark of civilization. On the contrary, taken in isolation, and in accordance with her formulations, it might be closer to being the nemesis of civilisation. How so? 

One question worth pondering is this: what is to prevent the surveillance capitalists whom Zuboff opposes from following in her footsteps and adopting the principle she gets from Sartre? “Sure”, they will say, “autonomy, the creation of meaning, self–determination in moral judgement, are all welcome. They are what led us to implement surveillance.” Zuboff’s riposte is not hard to imagine; she would presumably say that individual self–determination and autonomy are “elemental” universal human rights, so they cannot be exercised legitimately by surveillance capitalists to thwart the autonomy of others. 

But how has she concluded that they are universal human rights? Presumably by exercising her autonomous moral judgement. But suppose I do the same, and exercise my own autonomous judgement and conclude she is wrong – that there are no universal human rights such as she trumpets. On what grounds will Zuboff resist me? She would surely declare my judgement morally wrong. What she really believes is that I have a right to create my own meaning and morally determine myself just as long as this does not prevent others doing the same thing. She is thus issuing a moral judgement which is prescriptive (that is, binding) on all autonomous people. In doing so she is qualifying others’ autonomy. For Zuboff, there are tacit and undeclared moral limits to the meaning which she tells me that I am entitled to create. Zuboff may readily agree that she has those tacit added moral convictions. But why leave them undeclared? If she declared them, what would that do to her insistence on the over–riding value of radical autonomy? 

It is just as well that her actual moral beliefs are not consistent with her robust declarations about autonomy – that (presumably) she regards human rights as grounded in something deeper than her and others’ personal choice of values. When someone makes the independent value of autonomy the very basis of civilization, what they are exalting is the form of moral judgement (“It is for me to decide”.) However, civilization would be very ill–served indeed if only the form of moral judgement and not any determinate content (“Such and such is right or wrong”) were its bulwark. I do not think that Zuboff would really defend the belief that civilization is better grounded on the freedom of the individual to decide whether it is right or wrong to torture children than on the belief that the torture of children is wrong.

More than one thing is going wrong in her argument here but let us consider matters from the following angle. As a child, I was taught that it was wrong to steal. I was not told to decide for myself. In adulthood, I still believe that it is wrong to steal but now I believe it not because I was taught it (although some psychologists are welcome to try to take that conviction to pieces) but because I have gladly internalised the teaching – I see that it is wrong. I do not rejoice in my absolute self–determination, the naked freedom to decide for myself in the matter of theft, irrespective of whether there is such a thing as moral law in the matter. If I rejoice in anything, it is that I have appropriated for myself a truth which is a truth for all, not one which I have determined for myself. The main thing in which I rejoice is not that I have autonomy in this matter.Zuboff exalts self–determination under the heading: ‘When They Come for My Truth’ (290). Supposing she or anyone else says to me: “What stabilises civilisation is precisely the freedom for you to make up your mind what the truth about stealing is for you”.Then I will say: “Well, let me tell you about my little corner of civilization. I can tell you what my neighbourhood needs to make it a good place in which to live. It needs people with the conviction that it is wrong to steal. Yes, indeed, I most certainly want people to graduate to their own independent judgement in the matter; I should like them to see for themselves that stealing is wrong. But if, in my little corner of civilization, I had to choose between people being told: ‘Thou shalt not steal’ or being told: ‘You decide whether or not it’s okay to steal’, I assure you that my little corner of civilization would be much better off in the former case.” 

Is this unfair to Zuboff, who might quarrel with the terms of our imaginary dialogue by saying that she would obviously agree with the person who is being cast in the role of her critic? No: it is not unfair. We are pointing out the implications of giving autonomy such a dominant place as Zuboff does. Supposing she were to say: “Yes, of course, I believe that it is wrong to steal and civilization cannot exist in a healthy state if people believe otherwise.” Her claims for autonomy as the bulwark of civilization are now undercut. If she had said that autonomy is just one value alongside others, there might be no problem. She does not; she accords to autonomy the supreme place. Critics of Sartre have long pointed out the dangers of his rejection of the objectivity of value. Once the objectivity of value is admitted, autonomy ceases to occupy its commanding place. Zuboff should have taken this on board. The fiercely anti–Christian Nietzsche insisted that ‘autonomy’ and ‘morality’ are antitheses. People may protest against that claim but what Nietzsche saw clearly was that the modern age was trying to do two things at the same time: (a) cling to the idea of objective moral value in selected areas and (b) exalt autonomy to the status of a governing value. He flatly and firmly denied it could consistently be done. His challenge cannot easily be shrugged off. So how can Zuboff consistently rebut her dialogue partner in this imaginary dialogue? And what does it do to her attack on surveillance capitalism if she dethrones autonomy and admits that it must share its power as a moral force in society? 

Christianity and autonomy 

Christians should take a keen interest in the question of autonomy. Although this is not the place to trace it, there is a connection between Sartre’s understanding of the will (or those whom he represents) and the turn against Christianity in the West. Church and State came to be regarded over the centuries as oppressive forces, bent on stifling autonomy. “Men will not be free until the last king has been strangled with the entrails of the last priest”, to quote a version of a cheerful statement which goes back to Jean Meslier in the early Enlightenment. It took acts of will on the part of Western individuals to achieve, express and promote self–determination as a dominant value. Rebellion against a God conceived as dominantly oppressive will is an important strand in Western atheism. Church and State are his hands outstretched in oppression. The human will must lash out at both of them with self–determining resolve. Autonomy must come to reign.

This protest is itself a rich intellectual strand, but there is also a long Christian heritage of wrestling with the nature of autonomy. Christian thinkers of all theological stripes have attended to it in one way or another. One influential twentieth century treatment came at the hands of the theologian, Paul Tillich. He distinguished between autonomy, heteronomy and theonomy. In the case of autonomy, we are a law to ourselves; in the case of heteronomy, we are under the law of another. The law of God is different again. How so? If, as Christians, we believe that we are under the law of God, are we not heteronomists? We are not, because the word ‘heteronomy’ has entirely the wrong connotation here. Heteronomy is subjection to an extrinsic law, an imposition on our being. God’s law is not like that. By ‘God’s law’ we mean anything which God reveals to us is right or wrong, good or bad to do, although that is putting it very coldly from a Christian point of view. It is typically experienced by us humans as heteronomous and as an imposition only when we are alienated from God. However, in truth, the design of God’s law is to reveal to us the inner law of our being, the law by which we humans flourish. What takes the form of a divine command is actually God informing us about how to live in accordance with our own, created nature. That is why the divine injunction not to steal (to revert to our earlier example) is not really heteronomous. If we keep up Tillich’s language, we would say that what God is doing when he instructs us in how to live is recalling us from our alienated existence to our proper creaturely essence. That is what theonomy is about. Moreover – and this is not a casual postscript – the God in whom Christians believe enters into the unity of communion with us in Christ through the Spirit. It is in this context that we receive God’s word to us about how we should live.

It’s been important to challenge Zuboff’s articulated philosophy of civilization precisely because her warning about surveillance capitalism is so timely and true. Although she published her volume before the emergence of Covid–19, we indicated at the beginning of our article the specific relevance which her volume has in our current situation. In these days of coronavirus, we should be on heightened alert with respect to her warning. Zuboff remarks on the fact that the terrorist attack of 9/11 in the United States furthered the cause of surveillance by providing security with an impetus to trump (yes, yes, we know) privacy. If coronavirus is most effectively combated by national or international tracking, using the tools deployed in surveillance, this will considerably fortify the hand of security over privacy. Whatever our stand on contentious questions surrounding how to manage the twin requirements of security and privacy, if we wish to fortify the hand of privacy in this deadly serious game, let us not do so as Zuboff does, isolating it as a value grounded in radical autonomy. Let us instead, without apology, speak of those things which are good in themselves and not just of what is good about choosing. Then, to change the metaphor, privacy will be nourished in more fertile soils. 

Capitalism and contentment 

But there is another side to the story. What about those, what about us, who have been lured into the surveillance net as she describes it? Zuboff makes much of the fact that we should not blame the technology itself – still less label it ‘inevitable’ – but, rather, blame the driving capitalism or capitalists for the surveillance situation. But should not the blame be shifted around a little more? Is it not the case that we consumers have been dragooned into the surveillance net only because we have swum happily down the stream into its maw? Capitalism needs some willing consumers, does it not?

 Biblically–aware thinkers who mull over capitalism and consumerism will be reminded of the Bible’s teaching on contentment. The apostle Paul was notifying rather than nagging Timothy when he starkly said: “If we have food and clothing, with these we will be content” (1 Timothy 6:8). How many of us who have adequate food and clothing can affirm this from the heart rather than just assenting to it from the head? Just two verses earlier, Paul has used the abstract noun: “contentment”. Used on the lips of the Stoics of his day, it would have meant “self–sufficiency”.  The two words do not mean the same thing. We bring Stoicism into the picture because a version of it has been popularized in Silicon Valley. Even if scholars rightly judge it removed from its classical historical form, its ethos makes it more proximate to self–sufficiency than to biblical contentment. Contemporary secularized self–sufficiency, in its turn, is more at home in a world which prizes autonomy than one which prizes contentment in the presence of God. 

There is a deep historical vein here which is well worth tracing in order to understand the forces behind the modern world. A neo–Stoic philosophy helped shape modern Europe. The way it prized autonomy brought it into collision with the Christian tradition. In Paul’s day, there was a noticeable contrast between Stoic emphasis on self–sufficiency and Christian emphasis on contentment. In many ways, we might find that self–sufficiency and contentment can harmoniously co–exist in an individual. But what about the relationship between these two values in their capacity to shape culture? A culture shaped by contentment before God is going to look different from one shaped by the central value of self–sufficient autonomy. We have to take that into account in evaluating capitalism.

Before surveillance was associated with capitalism, capitalism was associated with materialism. “If industrial capitalism dangerously disrupted nature, what havoc might surveillance capitalism wreak on human nature?” asks Zuboff in words which she herself italicized (346). Juxtapose this to what Paul says when he follows up the words quoted above: “Those who desire to be rich fall into temptation, into a snare, into many senseless and harmful desires that plunge people into ruin and destruction. For the love of money is a root of all kinds of evils.” For “plunge”, “ruin” and “destruction” in Paul, read – in the spirit of literary correspondence, not literal equivalence – “wreak”, “disrupt” and “havoc” in Zuboff. The connections of thought apparent in Paul’s mind surely make us thoroughly uncomfortable if we are willing to agree that he might be speaking to us. We may sheepishly concede that we need more than food and clothing to be content, and hold up our hands in confession. However, we shall also emphatically deny that we have a love for money, and hold up our hands in self–defence. Will Paul permit us to draw such a definite distinction, or to draw it so quickly, so confidently? Surely we cannot avoid that question. Has not surveillance capitalism got where it has because it has built on the materialism and greed of us who are now ensnared in its net? 

In ‘Setting the Stage for Surveillance Capitalism’ in the first chapter of her volume, Zuboff observes that “[i]ndustrialization modernity and the practices of mass production capitalism at its core produced more wealth than had ever been imagined possible’” (35). She makes this observation in the course of telling “the individualization story”. Zuboff wants to establish a positive though not a simplistic connection between the historical process of individualization and her moral elevation of individual autonomy. But surely the reference to wealth ought to unsettle us on that path. True, it is all too easy to take cheap and enthusiastic shots at “wealth”, “capitalism”, “autonomy” and “individualism” without troubling to be careful or informed.  Nonetheless, questions have to be asked about how the story of peculiarly Western insistence on autonomy is connected with the story of peculiarly Western or Western–led material growth. Have the historical forms of material growth emanating from the West been enabled, in part, by a positive evaluation of autonomy? Or is a positive evaluation of autonomy the result, in part, of this material growth? Or, likelier still, are both these things the case? These are questions which have to be faced and pondered before we divide the world into the autonomous Good in conflict with the surveillance Evil. We may not need Paul to provoke these questions, but what he tells us about creaturely contentment surely alerts us to their importance. 

Our focus in this piece has been Zuboff’s use of the notion of autonomy, her misguided elevation of it as the cornerstone of moral values. But autonomy can have positive uses as well. Those who have worked in countries in the Majority World subject to the most appalling deprivation will know the positive value of autonomy for human flourishing. Those who have worked with the physically or intellectually disabled will know that there are more important things than autonomy for human flourishing. Far more important, and first and foremost, the poor and the disabled themselves will know these things. Of course, many experience both conditions at once, so the relation of the benefits of autonomy to its limitations needs to be carefully unraveled. Autonomy is a complex notion and will correspondingly be weighed and appreciated in different ways according to what feature of it we are considering. But we shall not be in a position to do that well so long as we make the autonomous individual the intellectual and moral centre of our approach. Humankind constitutes a vast relational entity.  The primary quality which ought to mark its constitution is love. Wealth, capitalism, the individual, autonomy, freedom – the context for discussing these must always be such things as contentment, deprivation, poverty, relationships and love. We do well to heed Zuboff’s analysis and warning. But we must re–locate them in a spiritually and morally deeper, wider and truer context if the battle against surveillance capitalism is to be a well–fought battle for true humanization.

 Interested in this? Share it on social media. Join our monthly e–newsletter to keep up to date with our latest research and events. And check out our Supporter Programme to find out how you can help our work.