Wednesday, April 14, 2021

The Self as Narrative: Is it good to tell stories about ourselves?

Is the self divided? Since the time of Freud the notion of a fragmented self has taken deep root in how we think of ourselves. Freud thought we were subject to competing forces, some conscious and some unconscious. It was from the conflict of these forces that the self emerged. Enduring conflicts between different elements of the self could lead to mental breakdown and illness.

Many other psychologists and psychotherapists have proposed similar theories, suggesting that we split ourselves into different roles and identities and sometimes struggle to integrate the competing elements into a coherent picture. In his book, The Act of Living, the psychotherapist Frank Tallis argues that achieving integration between the elements of the self is one of the primary therapeutic goals of psychotherapy. Why? Because personal fragmentation is thought to lie at the root of human unhappiness:

The idea that fragmentation or division of the self is a major determinant of human unhappiness, anxiety and discomfort appears in the writings of many of the key figures in the history of psychotherapy. Our sense of self accompanies all our perceptions, so when the self begins to crack and splinter everything else begins to crack and splinter too. The world around us (and our place in it) becomes unreliable, uncertain, frightening, and in some instances untenable. We experience ourselves as a unity and threats to cohesion are deeply distressing. 
(Tallis 2021, 150)


You may have felt this distress yourself. I know I have. There are parts of my identity that I struggle to reconcile with others. There is, for instance, the conflict between the ideals of my working self and parenting self. I often ask myself how can I justify spending time writing articles like this when I could be spending time with my daughter, particularly when she is so young. I don’t know how to reconcile the two sides of myself.

How can we resolve the distress into something more psychologically appealing? One solution is to tell ourselves a story. As Joan Didion famously remarked: we tell ourselves stories to live. We are narrative creatures. If we can knit the elements of our lives into a satisfying narrative perhaps we can achieve some psychological stability. It’s an attractive idea and one that has resonated with many thinkers over the years.

Or is it? Some people disagree. One of the most prominent is the philosopher Galen Strawson. Echoing millennia of Buddhist thinking on the matter, Strawson has argued that he (at least) does not experience his life as a narrative, nor does he think it is a good thing to do so. On the contrary, he thinks that integrating one’s life into a narrative may be an impediment to insight and ethical decision-making.

In this article I want to consider the conflict between Strawson and defenders of narrativity. I start by outlining some of the research that has been done on the psychological value of self-narratives. I then move on to consider Strawson’s critique of this idea.

1. The Value of Self-Narrative

Those who think of the self as narrative are proponents of something Strawson calls the ‘narrativity thesis’. There are many different construals of the narrativity thesis. Some people argue that the self is, necessarily or quintessentially narrative. Daniel Dennett, for instance, has developed a theory of self that maintains that what we call the self is the ‘centre of narrative gravity’ in our conscious minds. In other words, our consciousnesses are constantly writing multiple draft stories about who we are. The self is the draft that emerges from the melee. Others have adopted a similar view suggesting that we are essentially narrative beings who experience and make sense of our lives in a story-like way.

There is, however, another construal of the thesis. Instead of maintaining that there is something essentially narrative about us we could hold that we are often fragmented but that it would be a good thing for us to integrate the elements into a coherent narrative. In other words, even though we may struggle to tell a coherent story about ourselves, we need to do this in order to maintain psychological stability and well-being. This normative or axiological version of the narrativity thesis is the one that interests me.

Is there any reason to endorse it? There are some intuitive reasons to do so. As noted in the introduction, the failure to integrate various aspects of your life into a common framework can be psychologically unsettling. It can lead to distress, procrastination and anxiety. For instance, when I think about my own personal ideals as both a parent and an academic, I find myself landed in an internal conflict not dissimilar to the dilemma faced by Buridan’s Ass (the mythical creature that was equally poised between the water and the food). The net result of this conflict is that I default to procrastination and apathy, living up to neither set of ideals. This is deeply frustrating and, from what I gather, not an uncommon experience.

But we can go further than mere intuition. There has been a lot of psychological research on the value of telling a coherent story about yourself. One of the chief researchers in this area is Dan P McAdams. Over the years, McAdams has defended two key claims about the importance of narratives to the human condition. First, he claims that narrativity is a central part of our self identity. In other words, we come to understand who we are through the stories we tell about ourselves. This occurs through what McAdams calls synchronic and diachronic integration. In other words, we integrate different roles or elements of our lives as they arise at one time (synchronically) and across time and space (diachronically). McAdams claims that it is during later adolescence that people form distinct and coherent self narratives that become a key part of their self-identity. This doesn’t mean that they lack a ‘self’ at an earlier age but they do lack a coherent self-identity:

To the extent that a person’s self-understanding is integrated synchronically and diachronically such that it situates him or her into a meaningful psychosocial niche and provides his or her life with some degree of unity and purpose, that person "has" identity. Identity.. is something people begin to "work on" and have…[in] the emerging adulthood years. At this time…people begin to put their lives together into self-defining stories. It is an internalized and evolving story of self that integrates the self synchronically and diachronically, explaining why it is that I am sullen with my father and euphoric with my friends and how it happened—step by step, scene by scene—that I went from being a born-again Christian who loved baseball to an agnostic social psychologist. 
(McAdams 2001, 102)


In addition to this, McAdams argues that the kinds of stories we tell about ourselves have a significant impact on our psychological well-being. If you tell yourself the story that you are a perpetual failure and a complete fraud, you won’t be as happy and fulfilled as someone who tells themselves a story highlighting their successes and triumphs over adversity. In a study published in 2013, McAdams and his co-author Kate McLean review the psychological literature on this topic and highlight several specific self-narratives that seem to be associated with psychological well-being. They suggest that research shows that people who tell stories that emphasise personal growth, redemption and an overarching sense of purpose or meaning in their lives are psychologically better off than others. Some examples of such research findings from their paper include:

Bauer and colleagues have examined negative accounts of life-story low points as well as stories about difficult life transitions. People who scored higher on independent measures of psychological maturity tended to construct storied accounts that emphasized learning, growth, and positive personal transformation. 
(McAdams and McLean 2013, p 235 reporting on Bauer et al 2006)


In a longitudinal demonstration, Tavernier and Willoughby (2012) reported that highschool seniors who found positive meanings in their narrations of difficult high-school turning points showed higher levels of psychological well-being than those students who failed to construct narratives about turning points with positive meanings, even when controlling for well-being scores obtained 3 years earlier, when the students were freshmen. 
(McAdams and McLean 2013, p 235)


The psychological importance of such self-storying seems to be confirmed by other sources. For instance, a centrepiece of cognitive behavioural therapy is to reexamine the core beliefs that we have about ourselves. These core beliefs are often the central narrative hinges of our lives. If your core belief is that you are unlovable, you tend to tell yourself a story that fits all your experiences with that core belief. By targeting automatic thoughts and assumptions that are linked to these core beliefs, cognitive therapists have demonstrated considerable success in helping people improve their psychological well-being. Similarly, and just because I happened to be interested in the topic, Houltberg et al’s study of the self-narratives of elite athletes suggests that athletes that have a purpose-based narrative identity score better on independent measures of psychological well-being than those with a performance-based narrative identity (i.e. those that emphasise perfectionism and fear of failure).

There are some important caveats to all of this. The association between certain self-narratives and psychological well-being may be highly culturally dependent. McAdams himself notes this, emphasising that stories of personal redemption and growth have a particular resonance in American culture. In cultures with a more communitarian ethos, other stories may have greater value. This is something to bear in mind as we turn to consider Strawson’s critique of narrativity.

2. Strawson’s Critique of Narrativity

Strawson doens’t buy the narrativity thesis. He doesn’t deny that many people do experience their lives as a narrative, or that some people derive great meaning and value from the stories they tell. He does, however, deny that this is necessary to the human condition and suggests that it is not always a good thing. Indeed, in practice self-storying may be an impediment to the good life.

Strawson has critiqued the narrative view in several venues over the years. As best I can tell, he offers four main lines of criticism.

First, he argues that the narrativity thesis is often unclear. What does it actually mean to say that one does or should live one’s life as a narrative? Does this mean that your life has to fit some well-recognised narrative structure, such as the ‘hero’s journey’ or the ‘rags to riches’ adventure? Or does it simply mean that you have to put some form or shape to your life? If it means the former, then it seems like an obviously false thesis. Not everyone’s life fits a recognised narrative structure and trying to make the events in one’s life fit that structure will lead to a unjustified selectivity in how one remembers and orders events. If it means the latter, then it is arguably too loose a requirement. Virtually any sequence of events can be given some shape or form. Modern literature, for example, is replete with novels that defy traditional narrative structures and yet stil have some loose form.

Second, he argues that narrativity is not psychologically necessary. In other words, contrary to what people like Dennett and McAdams might argue, we are not all essentially narrative beings that come to understand ourselves through stories. Some people don’t think of the events in their lives slotting into some overarching narrative. Strawson usually cites himself as an example of such a person, proudly pronouncing that he has no sense of himself as a narrative being. Consider:

And yet I have absolutely no sense of my life as a narrative with form, or indeed as a narrative without form. Absolutely none. Nor do I have any great or special interest in my past…Nor do I have a great deal of concern for my future. 
(Strawson 2018, p 51)


And also:

…I’m blowed if I constituted my identity, or if my identity is my life’s story. I don’t spend time constructing integrative narratives of myself — or my self — that selectively recall the past and wishfully anticipate the future to provide my life with some semblance of unity purpose, and identity. 
(Strawson 2018, 192)


Doth he protest too much? Perhaps but what he says resonates with me to an extent. I don’t really invest much time in telling myself some long story about my past. That said, I do sometimes sequence certain events in my life into a story, e.g. intention-obstacle-overcoming that obstacle or not. Furthermore, the notion that someone could have no concern for their past or future is alien to me. Surely we all naturally have a little bit of concern for the past and future? After all, unless we have some serious cognitive deficit, we all naturally remember the past and plan for the future? It’s hard to imagine a human life without that sense of oneself in time and space.

Strawson is cognisant of this and elsewhere in his writings he insists that one can have a sense of oneself as an enduring human being without have a sense of oneself as a narrative being. We occupy the same human biological form over time and we remember what happened to our bodies and we have to plan for the future of what will happen to those bodies, but we don’t have to fit all of these happenings into a psychological narrative. He goes on to suggest that an enduring non-narrative being might represent their lives as a list of remembered events and not a narrative. This suggests to me that Strawson might see his life as a ‘listicle’ and not a story.

Third, Strawon argues that no ethical virtue or duty hinges on narrativity. Some people have the opposite view. They think that interpersonal duties and values such as loyalty and friendship depend on having a sense of oneself as an continuing narrative being. After all, how can you motivate yourself to be moral if you don’t care that much for your future (as Strawson proudly proclaims for himself)? Strawson rejects this. In his paper ‘Episodic Ethics’ he maintains that people who experience their lives as a series of short episodes can still have relatively normal moral lives. A sense of concern and empathy for others does not depend on narrativity. Indeed, it often depends more on attentiveness and awareness of the other in the moment. Projecting into the far future or past can dissociate you from the moral demands of the moment. He also cites examples of prominent people that profess an episodic experience of life and yet live morally normal lives.

Fourth, and adding to the previous argument, Strawson maintains that narrativity is often counterproductive both from the perspective of individual psychological well-being and from the perspective of one’s relations to others. If we constantly try to fit our lives into a narrative, there is a danger that we tell ourselves a false story. This falsity can work in different directions. Sometimes we might tell ourselves a false positive story, painting ourselves in a better light than we deserve, suggesting that we are more generous and caring than we really are, giving ourselves a free pass when we fail to do the right thing in the moment. Sometimes we might tell ourselves a false negative story, painting ourselves in a worse light than we deserve. Many people with depression and anxiety do this. They see themselves as failures, as useless, pathetic, unfit for life. They often experience profound shame and guilt as a result, and they can be ’toxic’ to other people as a result of these stories.

Against this, Strawson argues that some of the best chroniclers of the human condition — the ones that display the greatest empathy and understanding of what it means to be human — are non-narrative in their outlook. Contrary to the therapeutic interventions of psychotherapy, these non-narrative people often have the best kind of self-knowledge. Michel de Montaigne is a favourite example:

Montaigne writes the unstoried life — the only life that matters, I’m inclined to think. He has no “side”, in the colloquial English sense of this term. His honesty, though extreme, is devoid of exhibitionism or sentimentality (St. Augustine and Rousseau compare unfavorably). He seeks self-knowledge in radically unpremeditated life-writing: “I speak to my writing paper exactly as I do to the first person I meet.” He knows his memory is hopelessly untrustworthy, he concludes that the fundamental lesson of self-knowledge is knowledge of self-ignorance.
(Strawson 2018, 196)

There is something to this, I believe. An honest and ethically sensitive appraisal of one’s life requires a dispassionate observance of what is going on, almost like a sustained form of mindfulness. If the narrative self takes over, you often lose a true appreciation of who you are and what you owe to others.

3. Conclusion

So should we be narrative or not? My stance on this is equivocal. On reflection, I don’t think there is as much distance between the two views outlined in this article as might initially seem to be the case. I agree with Strawson on two main points. First, we are not necessarily narrative in nature and, indeed, the suggestion that each of us does (and should) fit the entire course of our lives into a single overarching narrative strikes me as absurd. If I asked any one of my friends to tell me the story of their lives, I doubt any one of them could do it. They don’t think about themselves in those terms. Second, I agree with Strawson that narratives often distort the truth and this can be unhelpful. Narratives can lead to overconfidence and excessive pessimism.

But I don’t think we can completely dismiss the psychological appeal and moral value of narratives either. While I don’t think of my entire life as a single extended narrative, I do think episodes within my life have some narrative-like structure. There are often important lessons to be learned from our experiences and telling stories about those experiences can be an effective way to remember those lessons. The empirical work done my McAdams and his colleagues cannot be rejected out of hand. The stories we tell about ourselves can have a positive and negative impact on our mental health and well-being.

The important thing is to do your best to avoid narrative distortion. Have a realistic sense of your strengths and weaknesses. If you have a tendency to fit everything within a narrative, try to take a step back from doing so. List the events within your life. Gather the evidence carefully. Avoid assuming that you are the hero of your life story; avoid assuming that you are the villain.

Tuesday, April 6, 2021

From Mind-as-Computer to Robot-as-Human: Can metaphors change morality?

Over the past three years, I have returned to one question over and over again: how does technology reshape our moral beliefs and practices? In his classic study of medieval technology, Lynn White Jr argues that simple technological changes can have a profound effect on social moral systems. Consider the stirrup. Before this device was created, mounted warriors had to rely largely on their own strength (the “pressure of their knees” to use White’s phrase) to launch an attack while riding horseback. The warrior’s position on top of the horse was precarious and he was limited to firing a bow and arrow or hurling a javelin.

The stirrup changed all that:

The stirrup, by giving lateral support in addition to the front and back support offered by pommel and cantle, effectively welded horse and rider into a single fighting unit capable of violence without precedent. The fighter’s hand no longer delivered the blow: it merely guided it. The stirrup thus replaced human energy with animal power, and immensely increased the warrior’s ability to damage his enemy. Immediately, without preparatory steps, it made possible mounted shock combat, a revolutionary new way of doing battle. 
(White 1962, p 2)


This had major ripple effects. It turned mounted knights into the centrepiece of the medieval army. And since the survival and growth of medieval society was highly dependent on military prowess, these knights needed to be trained and maintained. This required a lot of resources. According to White, the feudal manor system, with its associated legal and moral norms relating to property, social hierarchy, honour and chivalry, was established in order to provide knights with those resources.

This is an interesting example of technologically induced social moral change. The creation of a new technology afforded a new type of action (mounted shock combat) which had significant moral consequences for society. The technology needed to be supported and sustained, but it also took on new cultural meanings. Mounted knights became symbols of strength, valour, honour, duty and so forth. They were celebrated and rewarded. The entire system of social production was reoriented to meet their needs. There is a direct line that can traced from the technology through to this new ideological moral superstructure.

Can something similar happen with contemporary technologies? Is it already happening? In the remainder of this article I want to consider a case study. I want to look at social robots and the changes they may induce in our moral practices. I want to argue that there is a particular mechanism through which they may change our moral practices that is quite subtle but significant. Unlike the case of stirrup — in which the tool changed the social moral order because of the new possibilities for action that it enabled — I want to argue that social robots might change the social moral order by changing the metaphors that humans use to understand themselves. In particular, I want to argue that the more humans come to view themselves as robot-like (as opposed to robots being seen as human-like), the more likely it is that we will adopt a utilitarian mode of moral reasoning. I base this argument on the theory of hermeneutic moral mediation and some recent findings in human-robot interactions. This argument is highly speculative but, I believe, worth considering.

Terminological note: By ‘robot’ I mean any embodied artificial agent with the capacity to interpret information from its environment and act in response to that information. By ‘social robot’ I mean any robot that is integrated into human social practices and responds to human social cues and behaviours, e.g. care robots, service robots. Social robots may be very human like in appearance or behaviour but they need not be. For example, a robot chef or waiter in a restaurant might be very unhuman-like in appearance but may still respond dynamically and adaptively to human social cues and behaviours.

1. Hermeneutic Moral Mediation

In arguing that technology might alter human moral beliefs and practices, it is important to distinguish between two different understandings of morality. On the one hand, there is ‘ideal morality’. This is the type of morality studied by moral philosophers and ethicists. It consists of claims about what humans really ought to value and ought to do. On the other hand, there is ‘social morality’. This is the type of morality practiced by ordinary people. It consists in people’s beliefs about what they ought to value and what they ought to do. Social morality and ideal morality may not align with each other. Indeed, moral philosophers often lament the fact that they don’t. In considering how technology might change human morality, I am primarily interested in how it might change social morality, not ideal morality.

That said, there can be connections between ideal morality and social morality. Obviously, moral philosophers often use claims about ideal morality to criticise social morality. The history of moral reform is replete with examples of this including anti-slavery arguments in the Enlightenment era, pro-suffragette arguments in the late 1800s, and pro-same sex marriage arguments in the late 1990s early 2000s. But changes in social morality may also affect ideal morality, or at least our understanding of ideal morality. If people adopt a certain moral practice in reality, this can encourage moral philosophers to reconsider their claims about ideal morality. There is often a (suspicious?) correlation between changes in social morality and changes in theories of ideal morality.

How can technology induce changes in social morality? There are several theories out there. Peter Paul Verbeek’s theory of technological moral mediation is the one I will rely on in this article. Verbeek argues that technologies change how humans relate to the world and to themselves. To use the academic jargon: they mediate our relationships with reality. This can have moral effects.

Verbeek singles out two forms of mediation, in particular, for their moral impact: (i) pragmatic mediation and (ii) hermeneutic mediation. Pragmatic mediation arises when technology adds to, or subtracts from, the morally salient choices in human life. This forces us to consider new moral dilemmas and new moral questions. The impact of the stirrup on medieval warfare is an example of this. It made mounted knights more effective in battle and military commanders were thus forced to decide whether to use these more effective units. Given the overwhelming value attached to military success in that era, their use became a moral necessity: to not use them would be morally reckless and a dereliction of duty. Hermeneutic mediation is different. It arises when technology changes how we interpret the world, adding a new moral perspective to our choices. Verbeek argues that obstetric ultrasound is a classic example of hermeneutic moral mediation in action because the technology presents the foetus-in-utero to us as an independent being, situated inside but still distinct from its mother, and capable of being treated or intervened upon by medical practitioners. This alters our moral understanding of pre-natal care.

[If you are interested, I wrote a longer explanation of Verbeek’s theory here]

The widespread diffusion of social robots will undoubtedly pragmatically mediate our relationship with the world. We will face choices as to whether to deploy care robots in medical settings, whether to outsource tasks to robots that might otherwise have been performed by humans, and so on. But it is the hermeneutic effects of social robots that I want to dwell on. I think the diffusion of social robots could have a profound impact on how we understand ourselves and our own moral choices.

To make this point, I want to consider the history of another technology.

2. The Mind-as-Computer Metaphor

The computer was the defining technology of the 20th century. It completely reshaped the modern workplace, from the world of high finance, to scientific research, to graphic design. It also enabled communication and coordination at a global scale. In this way, the computer has pragmatically mediated our relationship with the world. We now think and act through the abilities that computers provide. Should I send that email or not? Should I use this bit of software to work on this problem?

Not only has the computer pragmatically mediated our relationship to the world, it has also hermeneutically mediated it. We now think of many processes in the natural world as essentially computational. Nowhere is this more true than in the world of cognitive science. Cognitive scientists try to figure out how the human mind works: how is it that we perceive the world, learn from it and act in it? Cognitive scientists have long used computers to help model and understand human cognition. But they go further than this too. Many of them have come to see the human mind as a kind of computer — to see thinking as a type of computation.

Gerd Gigerenzer and Daniel Goldstein explore this metaphorical turn in detail in their article ‘The Mind as Computer: The Birth of a Metaphor’. They note that it is not such an unusual turn of events. Scientists have always used tools to make sense of the world. Indeed, they argue that the history of science can be understood, at least in part, as the emergence of theories from the applied use of tools. They call this the ‘tools-to-theories’ heuristic. A classic example would be the development of the mechanical clock. Not long after this device was invented, scientists (or natural philosophers as they were then called) started thinking about physical processes (motion, gravitation etc) in mechanical terms. Similarly, when statistical tools were adopted for use in psychological experimentation in the 1900s, it didn’t take too long before psychologists starting to see human psychology as a kind of statistical analysis:

One of the most widely used tools for statistical inference is analysis of variance (ANOVA). By the late 1960s, about 70% of all experimental articles in psychological journals already used ANOVA (Edgington 1974). The tool became a theory of mind. In his causal attribution theory, Kelley (1967) postulated that the mind attributed a cause to an effect in the same way that a psychologist does — namely, by performing an ANOVA. Psychologists were quick to accept the new analogy between mind and their laboratory tool. 
(Gigerenzer and Goldstein 1996, 132).


The computational metaphor suffered from a similar fate. The story is a fascinating one. As Gigerenzer and Goldstein note, the early developers of the computer, such as Von Neumann and Turing, did work on the assumption that the devices they were building could be modeled on human thought processes (at either a biological or behavioural level). But they saw this as a one-way metaphor: the goal was to build a machine that was something like a human mind. In the 1960s and 70s, the metaphor turned around on itself: cognitive scientists started to see the human mind as a computational machine.

One of the watershed moments in this shift was the publication of Allan Newell and Herbert Simon’s Human Problem Solving in 1972. In this book, Newell and Simon outlined an essentially computational model of how the human mind works. In an interview, Simon documented how, through his use of computers, he started to think of the human mind as a machine that ran programs:

The metaphor I’d been using, of a mind as something that took some premises and ground them up and processed them into conclusions, began to transform itself into a notion that a mind was something that took some program inputs and data and had some processes which operated on the data and produced some output. 
(quoted in Gigerenzer and Goldstein 1996, 136)


This theory of the mind was initially resisted but, as Gigerenzer and Goldstein document, when the use of computational tools to simulate human cognition became more widespread it was eventually accepted by the mainstream of cognitive scientists. So much so that some cognitive scientists find it hard to see the mind as anything other than a computer.

3. The Human-as-Robot Metaphor

What significance does this have for robots and the moral transformations they might initiate? Well, in a sense, the robot is a continuation of the mind-as-computer metaphor. Robots are, after, all essentially just embodied computational devices, capable of receiving data inputs and processing them into actions. If the mind is seen as a computer, is it not then natural to see the whole embodied human as something like a robot?

We can imagine a similar metaphorical turn to the one outlined by Gigerenzer and Goldstein taking root, albeit over a much shorter timeframe since the computational metaphor is already firmly embedded in popular consciousness. We begin by trying to model robots on humans (already the established practice in social robotics), then, as robots become common tools for understanding human social interactions, the metaphor flips around: we start to view humans as robot-like themselves. This is already happening to some extent and some people (myself included) are comfortable with the metaphor; others much less so.

This thought is not original to me. Henrik Skaug Sætra, in a series of papers, has remarked on the possible emergence of ‘robotomorphy’ in how we think about ourselves. Many people have noted how humans tend to anthropomorphise robots (see e.g. Kate Darling’s work), but as robots become common Saetra argues that we might also tend to ‘robotomorphise’ ourselves. In a paper delivered to the Love and Sex with Robots Conference in December 2020, he remarks:

Roboticists and robot ethicists may similarly lead us to a situation in which all human phenomena are understood according to a computational, mechanistic and behaviourist logic, as this easily allows for the inclusion of robots in such phenomena. By doing so, however, they are changing the concepts. In what follows, our understanding of the concept, and of ourself, changes accordingly. 
(Saetra 2020, 10)*


But how does it change? Saetra has some interesting thoughts on how robot ethicists might use our interactions with robots to encourage a behaviourist understanding of human social interactions. This could lead to an impoverished (he says ‘deficient’) conception of certain human relationships, including loving relationships. Humans might favour efficient and psychologically simple robot partners over their more complex human alternatives. Since I am a defender of ‘ethical behaviourism’, I am, no doubt, one of the people that is guilty of encouraging this reconceptualisation of human relations (for what it’s worth, I don’t think this necessarily endorses an impoverished conception of love; what I do think is that it is practically unavoidable when it comes to understand our relationships with others).

Fascinating though that may be I want to consider another potential transformation here. This transformation concerns the general moral norms to which we are beholden. As moral psychologists have long noted, the majority of humans tend to follow a somewhat confusing, perhaps even contradictory, moral code. When asked to decide what the correct course of action is in moral dilemmas, humans typically eschew a simple utilitarian calculus (avoid the most suffering; do the most good) in favour of a more complex, non-consequentialist moral code. For example, in Joshua Greene’s various explorations of human reasoning in trolley-like scenarios (scenarios that challenge humans to sacrifice one person for the greater good), it is found that humans care about primary intentions, the physical proximity to a victim and other variables that are not linked to the outcome of our actions. In short, it seems that most people think they have act-related duties — not to intentionally harm another, not to intentionally violate trust, not intentionally violate an oath or duty of loyalty to another etc — that hold firm even when following this duty will lead to a worse outcome for all. This isn’t always true. There are some contexts in which outcomes are morally salient and override the act-related duties, but these are relatively rare.

Recent investigations into human moral judgment of robots paints a different picture. Studies by Bertram Malle and his colleagues, for example, suggests that we hold robots to different moral standards. In particular, we expect them to adopt a more utilitarian logic in their moral decision-making. They should aim for the greater good and they are more likely to be negatively evaluated if they do not. We do not (as readily) expect them to abide by duties of loyalty or community. Malle et al’s findings have been broadly confirmed by other studies into the asymmetrical moral norms that humans apply to robots. We think robots should focus on harm minimisation; we don’t judge them based on their perceived intentions or biases. For example, Hidalgo et al’s recent book-length discussion of a series of experiments done on over 6000 US subjects, How Humans Judge Machines, seems to be broadly consistent with the moral asymmetry thesis.

Now, I would be the first to admit that these findings are far from watertight. They are, for the most part, based on vignette studies in which people are asked to imagine that robots are making decisions and not on interactions with real-world robots. There are also many nuances to the studies that I cannot do justice to here. For instance, there are some tentative findings suggesting that the more human-like a robot’s actions, and/or the more harm it causes, the more inclined we are to judge it in a human-like way. This might indicate that the asymmetry holds, in part, because we currently dissociate ourselves from robots. 

Nevertheless, I think these findings are suggestive and they do point the way toward a hermeneutic moral effect that the widespread deployment of robots might have. If it becomes common wisdom for us to interpret and understand our own behaviour in a robot-like manner, then we may start to hold ourselves to the same moral standards as machines. In other words, we may start to adopt a more outcome-oriented utilitarian moral framework and start to abandon our obsession with intentions and act-related duties.

Three factors convince me that this is a plausible potential future. First, there is a ready-made community of consequentialist utilitarian activists that would welcome such a moral shift. Utilitarianism has been a popular moral framework since the 1800s and has resonance in corporate and governmental sectors. The effective altruist movement, with its obsessive focus on doing the most good through evidence-based personal decision-making, may also welcome such a shift. Second, there is some initial evidence to suggest that humans to adapt their moral behaviours in response to machines. Studies by Ryan Jackson and colleagues on natural language interfaces, for instance, suggest that if a machine asks a clarificatory question implying a willingness to violate a moral norm, humans are more willing to violate the same norm. So we can imagine that if machines both express and act in way that violates non-consequentialist norms, we may, in turn, be more willing to do the same. Finally, there are now some robot ethicists that encourage us to take the metaphorical flip, i.e. to make human moral behaviour more robot-like and not to make robot moral behaviour more human-like. One interesting example of this comes from Sven Nyholm and Jilles Smids article on the ethics of autonomous vehicles in ‘mixed traffic’ scenarios, i.e. where the machines must interact with human-driven vehicles. A common approach to the design of mixed traffic scenarios is to assume that the machines must adapt to human driving behaviour but Nyholm and Smids argue that sometimes it might be preferable for the adaptation to go the other way. Why? Because machine driving, with its emphasis on harm-minimisation and strict adherence to the rules of the road, might be morally preferable. More precisely, they argue that if automated driving is provably safer than human driving, then humans face a moral choice, either they use automated driving or they adapt to automated driving standards in their own behaviour:

If highly automated driving is indeed safer than non-automated conventional driving, the introduction of automated driving thereby constitutes the introduction of a safer alternative within the context of mixed traffic. So if a driver does not go for this safer option, this should create some moral pressure to take extra safety-precautions when using the older, less safe option even as a new, safer option is introduced. As we see things, then, it can plausibly be claimed that with the introduction of the safer option (viz. switching to automated driving), a new moral imperative is created within this domain [for human drivers]. 
(Nyholm and Smids 2018)


If we get more arguments like this, in more domains of human and robot interaction, then the net effect may be to encourage a shift to a robot-like moral standard. This would complete the hermeneutic moral mediation that I am envisaging.

4. Conclusion

None of this is guaranteed to happen nor is it necessarily a good or bad thing. Once we know about a potential moral transformation we can do something about it: we can either speed it up (if we welcome it) or try to shut it down (if we do not). Nevertheless, speculative though it may be, I do think that the mechanism I have discussed in this article is a plausible one and worth taking seriously: by adopting the robot-as-human metaphor we may be inclined to favour a more consequentialist utilitarian set of moral norms.

* I’m not sure if this paper is publicly accessible. Saetra shared a copy with me prior to the conference.


Tuesday, March 30, 2021

Technology and the Value of Trust: Can we trust technology? Should we?

Can we trust technology? Should we try to make technology, particularly AI, more trustworthy? These are questions that have perplexed philosophers and policy-makers in recent years. The EU’s High Level Expert Group on AI has, for example, recommended that the primary goal of AI policy and law in the EU should be to make the technology more trustworthy. But some philosophers have critiqued this goal as being borderline incoherent. You cannot trust AI, they say, because AI is just a thing. Trust can exist between people and other people, not between people and things.

This is an old debate. Trust is a central value in human society. The fact that I can trust my partner not to betray me is one of the things that makes our relationship workable and meaningful. The fact that I can trust my neighbours not to kill me is one of the things that allows me to sleep at night. Indeed, so implicit is this trust that I rarely think about it. It is one of the background conditions that makes other things in my life possible. Still, it is true that when I think about trust, and when I think about what it is that makes trust valuable, I usually think about trust in my relationships with other people, not my relationships with things.

But would it be so terrible to talk about trust in technology? Should we use some other term instead such as ‘reliable’ or ‘confidence-inspiring’? Or should we, as some blockchain enthusiasts have argued, use technology to create a ‘post-trust’ system of social governance?

I want to offer some quick thoughts on these questions in this article. I will do so in three stages. First, I will briefly review some of the philosophical debates about trust in people and trust in things. Second, I will consider the value of trust, distinguishing between its intrinsic and extrinsic components. Third, I will suggest that it is meaningful to talk about trust in technology, but that the kind of trust we have in technology has a different value to the kind of trust we have in other people. Finally, I will argue that most talk about building ‘trustworthy’ technology is misleading: the goal of most of these policies is to obviate or override the need for trust.

1. Can you trust a thing?

Philosophers like to draw a distinction between trust and mere reliance. The distinction is usually parsed like this: trust is something that exists between people; mere reliance can exist between people and things. One person trusts another when they expect the other will act with goodwill towards them and live up to their obligations. Mere reliance involves the expectation that someone or something will follow a predictable pattern of behaviour.

I believe this distinction was first articulated by Annette Baier in her 1986 article ‘Trust and Antitrust’. More recently, Katherine Hawley has made it the centrepiece of her theory of trust. In her article ‘Trust, Distrust and Commitment’ she opens with a section entitled ‘Trust is not Mere Reliance’. Why not? Hawley accepts that this distinction is not one that is respected in ordinary language. Much to the annoyance of philosophers, people do talk about trusting their cars, their appliances and even the ground on which they walk. But Hawley thinks people are wrong to do so. They should make the distinction because trust is a normatively richer concept than mere reliance. 

Here is the core of her argument:

The distinction is important because trust, not mere reliance, is a significant category for normative assessment. Trust, unlike mere reliance, is connected to betrayal. Moreover trustworthiness is clearly distinguished from mere reliability. Trustworthiness is admirable, something to be aspired to and inculcated in our children: it is a virtue in the everyday sense, and perhaps in the richer sense of virtue ethics too. Mere reliability, however, is not. A reliable person is simply predictable: someone who can be relied upon to lose keys, or succumb to shallow rhetoric, is predictable in these respects, but isn't therefore admirable. Even reliability in more welcome respects need not amount to trustworthiness: when you reliably bring too much lunch, you do not demonstrate trustworthiness, and nor would you demonstrate untrustworthiness if you stopped. 
(Hawley 2012, 2)


This is a strange argument. There seem to be two main parts to it. The first is the claim that trust is linked to betrayal while mere reliability is not. I guess that’s true, but that is probably just an artefact of the conceptual vocabulary we use. Betrayal is the flipside or negative of trust: it’s what happens when trust goes bad. There is, presumably, a negative side to reliability too. Unpredictability? Randomness? The second claim is that trustworthiness is admirable and normatively assessable in a way that mere reliability is not. But is that really true? It seems to me that many people think that being 'reliable' is an admirable quality. I often overhear people talking about work colleagues being reliable with the implication being that they exhibit some virtue. It is true that people can be reliably bad, but that doesn’t say much. After all, people can misplace trust in others or their trust can be betrayed. In other words, just as reliability has its ups and downs so too does trust. I can’t help but wonder if the modifier ‘mere’ is doing a lot of the work in this conceptual distinction. If we said ‘mere trust’ instead of ‘trust’ would we have a similarly dismissive attitude?

In any event, neither of these points is particularly pertinent to the issue at hand. Even if there is this important conceptual distinction between trust and mere reliance, it does not follow that you cannot trust a thing. To make that argument, you would have to suggest that there is some condition of trust that is linked to a property that people have but machines or things lack. What might that be?

The typical answer appeals to mental properties. The idea is that trust depends on having a mind. Since things cannot have minds, they cannot be proper objects of trust. Mark Ryan develops this critique in his article ‘In AI We Trust: Ethics, Artificial Intelligence and Reliability’. In the article, Ryan identifies a number of conditions that must be satisfied in order for trust to exist between two entities or parties. They include things like believing that the other party is competent to perform some action or function, having confidence that they will perform those functions, and being vulnerable to them if they do not. Ryan accepts that machines, specifically AIs, can satisfy these three conditions and so a form of ‘rational’ trust in machines (which might be equivalent to what others call ‘reliance’ or ‘confidence’) is possible. But machines cannot satisfy two other critical conditions for the normatively richer form of trust: (i) they cannot be motivated to act towards us out of a sense of goodwill or out of a desire to live up to their moral obligations toward us; and (ii) they cannot betray us.

Without getting too into the details, I think there are problems with the second part of this argument. Ryan’s claim that machines cannot betray appears to be circular. In essence, his position boils down to the claim that you cannot betray someone unless you are a proper object of trust but you cannot be a proper object of trust unless you have the capacity for betrayal. But that just begs the question: how do you become a proper object of trust or develop the capacity for betrayal? 

That leaves the other part of the argument: the claim that machines cannot have the right kinds of motivation or desire for action. What does Ryan say about this? A lot, but here is one critical quote from his paper:

While we may be able to build AI to receive environmental input and stimuli, to detect appropriate responses, and program it to select an appropriate outcome, this does not mean that it is moved by the trust placed in it. While we may be able to program AI to replicate emotional reactions, it is simply a pre-defined and programmed response without possessing the capacity to feel anything towards the trustor. Artificial agents do not have emotions or psychological attitudes for their motives, but instead act on the criteria inputted within their design or the rules outlined during their development [reference omitted] 
(Ryan 2020, p 13)


In other words, we might create machines that look and act they care about us or look and act like they are motivated by reasons similar to our own, but this is all just an illusion. They don’t feel anything or care about us. They are just programmed artifacts; not conscious, caring humans. They have no minds, no intentions, no inner life.

If you have read any of my previous work on ‘ethical behaviourism’ (e.g. here, here, here and here), you will know that I do not like this kind of argument. To me, it smacks of an unwarranted form of human exceptionalism and mysterianism: humans have this special property that cannot be replicated by machines, but how that property is instantiated in humans is both mysterious and never fully specified. My own view is that while there are important differences between humans and machines (particularly as they are currently designed and operated) there is no ‘in principle’ reason why machines cannot be motivated to act toward us with goodwill and moral rectitude. After all, the only reason we have to believe that other humans are so motivated toward us is because of how they look and act. Looking and acting, broadly defined, are the epistemic hinge on which perceptions of mindedness turn. We can rely on the same evidence when it comes to machines. If they look and act the right way, we can trust them. Similarly, the notion that machines are somehow different from us because they act on the basis of ‘criteria inputted within their design or rules outlined during the development’ also strikes me as being misleading and false. Humans have also been manufactured through a process of evolution by natural selection and personal biological development. We are constrained by both processes and we act on the basis of decision rules and heuristics acquired during these developmental processes. We may sophisticated and complicated biological machines, but there is nothing magical about us.

If I’m right, then even on Ryan’s account of trust it is, in principle, possible for us to trust machines. But this assumes that Ryan (and Hawley and Baier) are right in supposing that trust depends on mental properties like goodwill and a desire to do the right thing. What if that is the wrong way to think about trust?

One of the most interesting recent papers on this topic comes from C. Thi Nguyen. It is called ‘Trust as an Unquestioning Attitude’. In it, Nguyen argues that we can have a normatively rich form of trust in objects and things. Indeed, hearkening back to the point made by Hawley, he suggests that reference to this non-interpersonal or non-agential form of trust are common in everyday language. He cites several examples of this, including climbers who talk about ‘trusting’ their climbing ropes, and people who have lived through earthquakes talking about feeling ‘betrayed’ by the ground beneath their feet.

What is it that unites these non-agential forms of trust? Nguyen argues that this form of trust arises when we have an unquestioning attitude toward something. In other words, when we take it for granted that it will act in a certain way and we depend on it do so. In this respect, we all trust the ground beneath our feet. We don’t wake up in the morning and assume that it will suddenly tear apart and swallow us up. We rely upon this assumption to live our lives. It is only in the extreme case of an earthquake that we realise how much trust we place in the ground. Other examples of this form of trust abound in our everyday discourse.

But what about all those philosophers who insist that trust can only exist between people? Nguyen says something about this:

I have found that philosophers who work on trust and testimony think that this use of “trust” is bizarre and unintuitive — especially locutions like “trusting the ground” and feeling “betrayed by the ground”. But it seems to me that, in fact, these expressions are entirely natural and comprehensible, and it is only excess immersion in modern, narrowed philosophical theories of trust that renders these locutions odd to the ear. 
(Nguyen, MS p 10)


There is a general lesson for philosophers here. For instance, I have encountered a similar phenomenon when writing about gratitude. I once tried to publish a paper on whether atheists could be grateful for being alive. It was repeatedly rejected from journals by reviewers who insisted that gratitude is necessarily interpersonal. According to them, it makes no sense to be grateful for things or for some natural state of affairs. You can only be grateful toward other people. This always struck me as bizarre and counterintuitive but, according to these reviewers, I was the outlier. (If you are interested, you can find the unpublished paper here. Before you say anything, I’m sure there are other reasons why it should have been rejected for publication)

Assume Nguyen is right. What is normatively significant about his version of trust? Nguyen sees trust as an unquestioning attitude as something that is integral to our sense of agency. We are cognitively limited beings. We cannot be constantly suspicious and questioning of everything. By accepting that things (cars, climbing ropes, mobile phones) will work in a certain way, or that people (lovers, friends, fellow citizens) will live up to their obligations, we give ourselves the freedom to live more enriched and open lives.

This doesn’t mean that we can never be suspicious of them. This trust can be misplaced and its wobbly foundations can be revealed in certain circumstances (like in the midst of an earthquake). When this happens we may critically interrogate our previous unquestioning attitude. We may search for data to confirm whether we are right to trust this thing or not. Depending on the outcome of this inquiry, we may find our trust restored or we may find that we can no longer take the thing for granted. Either way, trust as an unquestioning attitude is a normatively essential part of what it means to be human. Given our cognitive limitations, we couldn’t get by without it.

I like Nguyen’s theory of trust. I think it captures something important about our relationship to the world around us. We don’t just rely on our friends, or on the ground beneath our feet or on the smartphones in our pockets. We trust them to act or to persist in certain way so that we can get on with the business of living.

2. Technology and The Value(s) of Trust

If Nguyen’s right, then it does make sense to talk about trust in technology. But this raises a deeper question. Everyone talks about the value of trust but what form does this value take. Is trust valuable in and of itself? In other words, is it a good thing to have trusting relationships in our lives, irrespective of their consequences? Or is trust valuable purely for consequential reasons?

There is a common philosophical distinction that is relevant here: the distinction between intrinsic and instrumental value. It is possible to argue that trust has both kinds of value:

The Intrinsic Value of Trust: Trust is valuable in and of itself (irrespective of its consequences) because it expresses an attitude of respect or tolerance toward the object of trust. For example, if you trust another human being you are signalling to them that you recognise and respect their moral status and moral autonomy.


The Instrumental Value of Trust: Trust is valuable because it is practically essential to human life. It allows us to cooperate and coordinate with others, which allows us to innovate and develop and explore more opportunities. A life without trust would be impoverished because it would lack access to other valuable things.


From my reading of the literature, the instrumental value of trust tends to be emphasised more than the intrinsic value of trust . There is a good reason for this. Everyone that writes about trust notes that trust is a double-edged sword. Whenever you trust a person or a thing you cede some control and power to them. When I trust my partner to look after our daughter, I give up my own attempts to manage and control all aspects of childcare. When I trust my calendar app to keep a record of my appointments and meetings, I give up my own attempts to keep a mental record of my appointments and meetings. The irony is that ceding power and control in this manner can actually be empowering. By not having to worry about childcare or scheduling (at least temporarily) you unlock other opportunities and overcome some of your own cognitive and temporal limitations. This is Nguyen’s argument. But ceding power and control can be risky. The trust can be betrayed. My partner might not look after our daughter properly, my calendar app might fail to update or record a meeting. When this happens I may lose, rather than gain, something that I value.

It is because the consequences of misplaced trust can be so terrible that people tend to emphasise the instrumental value of trust. Even if trust has some intrinsic value this can be swamped by its negative consequences. Imagine if my partner, through neglect, causes our daughter to become seriously ill. By trusting her to look after our daughter I will have expressed my respect for her moral status and autonomy, but that will be of little consolation if our daughter is seriously ill. The intrinsic value of trust is present and cannot be denied, but it has been overridden by the negative instrumental value of trust. It is the instrumental value that matters most.

How is this relevant to the debate about trust in technology? Well, if we accept that we can trust technology (and that it is meaningful to talk about such trust), then we can also accept that this form of trust can have significant instrumental value. It can help us to access other values that would be impossible (or very difficult to obtain) without that trust. But the intrinsic value of trust does seem to be absent when it comes to our relationships with technology. If we accept that most technologies as they currently exist lack an independent moral autonomy and moral status, then we cannot express respect or tolerance for technology by trusting it. This means that the value of trust in technology hinges entirely on the consequences of this trust: if the consequences are good, then it has instrumental value; if the consequences are bad, it does not.

There are three counterarguments to this claim that trust in technology lacks intrinsic value. The first is to claim that even if technology currently lacks independent moral autonomy and status, it may someday acquire this. The typical way to run this counterargument is to suggest that sophisticated machines might acquire the mental properties that we typically associate with moral autonomy and status and, once they do, we will be able to express respect and tolerance toward them by trusting them. Given my earlier critique of Mark Ryan’s views on trust in technology, and my defence of ethical behaviourism, I am quite sympathetic to this argument. I’m just not sure that any present technology rises to requisite level of sophistication.

The second counterargument is to claim that entities do not need to posses mental properties in order for them to have a moral status that is worthy of respect. Environmental ethicists, for example, might argue that aspects of the natural world have an independent moral status that is not derived from human enjoyment of or dependence on the natural world. It is, consequently, not absurd to suggest that we can express respect or tolerance toward aspects of the natural world. If that is right, then it may be less of a stretch to say that trust in technology in its current form has some intrinsic value (remembering, at all times, that this intrinsic value can be swamped by the negative consequences of misplaced trust).

The third counterargument is to claim that technology is a product of human moral agency and autonomy and hence it can have a kind of derived moral status. In other words, it makes sense to express respect for the technology because in doing you are expressing respect for its human creator. There may be some plausibility to this argument in certain contexts. For example, I trust the chef at my favourite restaurant not to poison me. As a result, I don’t test the chemical composition of his food every time it comes out to my table. I just eat it. By trusting that the food will be fine I am, in a sense, expressing my respect for him. But whether this reasoning holds up in the case of technology is much less clear. Most technologies are created by teams of humans. You are not singling any one of them out for respect and, arguably, it is just as mistaken to respect an entire group of humans as it is to respect a thing. But even if you can, the value of trusting their product is still only a derived value and it is quite a nebulous and partial one at that.

In conclusion, trust in technology can have instrumental value (or disvalue as the case may be), but it probably lacks the intrinsic value that arises from trust between human beings. That said, the intrinsic value of trust is quite limited and can easily be swamped by the negative consequences of misplaced trust. So to say that trust in technology lacks intrinsic value is not to say all that much.

3. Concluding Thoughts

None of this is to suggest that we ought to trust technology. It is simply to say that it is meaningful to talk about trusting technology and this type of trust can have significant instrumental value in our lives. Whether it does, in fact, have such value depends on the properties and dynamics of the technology. What does it actually do in our lives? Does it empower us? Or does it act against our interests? Does it do more of the former than the latter?

These are the very same questions we should ask about our relationships with other human beings. We shouldn’t trust all humans. That would be a mistake. Whether we should trust them, or not, depends on who they are and what they do to us. If we take an unquestioning attitude toward them, does this unlock other opportunities and goods for us? Or does it leave us exposed to exploitation and abuse?

It is undoubtedly true that many of us trust technology in our daily lives and are rewarded for doing so. Right now, as I write these words, I’m trusting my computer and my word processing software to safely record and save them for later retrieval. I don’t doubt that the files will be there tomorrow morning when I wish to work on them again. Similarly, I trust my car not to breakdown when I drive to collect my daughter this afternoon. I don’t meticulously check the undercarriage or the engine every time I hop into the driver’s seat.

The problem is that this trust is sometimes betrayed. Modern technologies can let us down. Digital technology is vulnerable to security hacks and data leaks. Mass surveillance can compromise our privacy. Apps can work more for their creator’s interests than for those of their human users. To use a trite example, it is in Facebook’s interests to keep you hooked on their newsfeed and clicking on their ads. Whether this is in your interest is much more doubtful. In many cases, assuming that the technology has a benign effect on your life can be mistaken. This is the dark side of trust.

What can we do about this? Efforts to create trustworthy technology can help, but many of these efforts must be understood for what they really are. Sometimes they are not about encouraging or facilitating trust in technology. They are, instead, about making it possible for us to critically scrutinise the technology. In other words, to make it possible for us to take questioning attitude toward it when we feel unsure about its bona fides. This is why there is such a significant emphasis on transparency, accountability and audit trails when it comes to creating trustworthy technology.

These are laudable goals, and once the mechanisms of accountability are put in place people may well slip back into an unquestioning attitude toward technology. Trust could then be restored. But the policy itself is motivated by the belief that the technology in its current form is not trustworthy.

Friday, March 26, 2021

89 - Is Morality All About Cooperation?

What are the origins and dynamics of human morality? Is morality, at root, an attempt to solve basic problems of cooperation? What implications does this have for the future? In this episode, I chat to Dr Oliver Scott Curry about these questions. We discuss, in particular, his theory of morality as cooperation (MAC). Dr Curry is Research Director for Kindlab, at He is also a Research Affiliate at the School of Anthropology and Museum Ethnography, University of Oxford, and a Research Associate at the Centre for Philosophy of Natural and Social Science, at the London School of Economics. He received his PhD from LSE in 2005. Oliver’s academic research investigates the nature, content and structure of human morality. He tackles such questions as: What is morality? How did morality evolve? What psychological mechanisms underpin moral judgments? How are moral values best measured? And how does morality vary across cultures? To answer these questions, he employs a range of techniques from philosophy, experimental and social psychology and comparative anthropology.

You can download the episode here or listen below. You can also subscribe on Apple PodcastsStitcherSpotify and other podcasting services (the RSS feed is here).


Show Notes

Topics discussed include:

  • The nature of morality
  • The link between human morality and cooperation
  • The seven types of cooperation 
  • How these seven types of cooperation generate distinctive moral norms
  • The evidence for the theory of morality as cooperation
  • Is the theory underinclusive, reductive and universalist? Is that a problem?
  • Is the theory overinclusive? Could it be falsified?
  • Why Morality as Cooperation is better than Moral Foundations Theory
  • The future of cooperation

Relevant links

Thursday, March 18, 2021

The Importance of Collective Intelligence in a Sustainable Future

[This is the text of a short - 15 minute talk - I delivered to the Viridian Conference on 17th March 2021. The purpose of the conference was to discuss the Viridian Declaration, which advocates for technological and social reform in order to make the environment sustainable. One aspect of the declaration focuses on the importance of resilient and adaptive social institutions. That’s where I focused my energies for my talk. As with all short talks of this nature, the ideas and arguments here are programmatic and provocative. They are not rigorously defended. I’m fully aware that there are some holes in what I have to say, but I hope it provides food for thought nonetheless. I have changed the title and made some minor updates to this text from the version I used during the actual talk].

I am very pleased to be able to talk to you this evening and I would like to thank the organisers for inviting me to participate in this exciting and optimistic event. I am a philosopher and ethicist of technology and it is, sadly, rare for me to participate in an event in which people talk positively about the future.

In the short time I have with you I want to make an argument, or maybe even a plea: that we need to think carefully, systematically and scientifically about how best to harness human and machine intelligence to solve the ecological, sociological and technological problems we currently face. Indeed, I would argue that thinking systematically and scientifically about how to harness human and machine intelligence is the greatest challenge of our times. There are some tentative and fascinating attempts to do this already, but they are insufficient. We need to do much more.

To support this argument there are four propositions I wish to defend in the remainder of my talk.

  • Proposition 1: In order for our societies to have a bright and sustainable future, we need to invest in our problem-solving capacity. Since intelligence is equivalent to problem-solving capacity another way of putting this is to say that we must invest in our intelligence capital. This type of capital, not simply labour or technology or natural resources, is crucial to our long-term survival.

Why is this true? I take my support for this from Joseph Tainter’s argument in The Collapse of Complex Societies. For those who do not know, Tainter’s book tries to explain why complex historical civilisations collapsed. He looks at all the famous examples: the Western Chou Empire, The Egyptian Old Kingdom, The Hittite Empire, the Western Roman Empire, and Lowland Classic Maya. After reviewing and dismissing typical explanations for this collapse — including the caim that environmental damage and destruction is the main culprit — Tainter concludes that insufficient of problem-solving capacity is the root cause of civilisational decline. His argument is relatively simple and consists of five claims:

  • Human societies are organisations that solve basic existential and psychological problems for their members. They thereby generate benefits (B) for their members and, in order to sustain themselves, they must continue to solve problems and generate benefits.

  • Like all organisations, human societies must capture and expend energy (C) in order to sustain their problem-solving capacity. Classically, societies have captured energy by foraging, farming, burning fossil fuels, and also through war and imperial expansion.

  • Therefore, in order to sustain themselves, societies face a basic cost-benefit equation: the benefits of increased energy expenditure on problem-solving capacity must exceed the costs (i.e. B must be > C)

  • Increased investment in problem-solving capacity usually yields higher returns (B ↑) but this often comes at an increasing per capita cost so that, at a certain point, the marginal benefits (mB) of increased investment are outweighed by the marginal costs (mC).

  • If mC exceeds the mB at an increasing rate, societies will collapse (i.e. experience a rapid and significant decline in socio-political complexity: this can encompass war, famine, or partition and fragmentation).

The only way that societies avoid this dilemma is by developing some breakthrough technology (broadly understood) that increases the net benefit or reduces the per capita cost of problem-solving. Past examples of such breakthroughs include new methods of food production, facilitation of open markets and trading, new techniques of bureaucratic management, new forms of energy production and, most recently, computing power. But each of these technological breakthroughs is made possible by human intelligence. Hence it is our intelligence capital, not technology or innovation per se, that should be the main target for investment if we are to sustain our civilisation.

  • Proposition 2: The most effective forms of intelligence are collective in nature, not individualistic. In other words, when investing in intelligence capital we should focus primarily on making teams of humans and machines enhance their collective intelligence, and not solely on making individuals more intelligent.

We all like to celebrate lone geniuses such as Einstein, Newton and Darwin. But the reality is that lone geniuses play a limited role in the sustainability and success of human civilisation. It is collections of individuals, coordinating and cooperating their intelligences that makes this possible. There is a certain amount of common sense behind this argument: look around and note how dependent you are on the intelligence of others to solve your basic existential problems. But there have been some books that defend this common sense idea in more detail. One of my favourites is the psychologist Joseph Henrich’s book The Secret of Our Success, which is a systematic defence of the importance of collective intelligence in human history. Henrich uses an arresting thought experiment to set up his main thesis. Imagine you took a random human and a random chimpanzee and dropped them both in the jungle to fend for themselves. Who would you bet on surviving for more than a few days? Henrich argues that you would bet on the chimpanzee every time. Why? Because individual humans are far too dependent on their cultures, their collaborators and their technologies to get by.

Technologies, including artificial intelligence, play a crucial role in collective intelligence structures (or ‘groupminds’ as we might call them). Humans are a technological species. One of the main products of our intelligence is technology: from stone axes to nuclear reactors, groups of humans create and use technologies to enhance our collective intelligence and solve our existential problems. Artificial intelligence is just the latest product of human collective intelligence. Some worry that it threatens to replace or undermine human intelligence. Some of these fears may be well-founded, others less so. That’s not a debate I wish to get into right now. What I do wish to say, however, is that ideally we shouldn’t think of AI as something separate or alien from human collective intelligence. Ideally, we should view AI as a new partner or assistant to human intelligence — as something that needs be incorporated effectively into our collective intelligence structures. Indeed, doing this could give us the technological breakthrough we need to avoid Tainter’s dilemma.

  • Proposition 3: We can think systematically and scientifically about collective intelligence structures and how best to design them to maximise their benefits relative to their costs.

We know that some collective intelligence structures are more successful than others. We also know that there are many different designs or structures that we could implement. We are now starting to develop a scientific field of inquiry into collective intelligence and it is important that we double-down on this effort if we are to survive and thrive in the future.

There are several disparate fields of inquiry that are relevant to this, from organisational theory and management theory, to human-computer interaction studies. At the moment, most of these inquiries are siloed in their own academic subfields and focused on a narrow set of problems. For example, organisational theory and management theory often measures the success of groups in terms of their efficiency or profitability, not their intelligence per se. What we need to do is unite these fields of inquiry into a common discipline centred on collective intelligence.

The work of Thomas Malone and his colleagues has been particularly insightful and innovative in this regard. They have developed group intelligence tests that replicate individual IQ tests. In initial experiments on this idea they find that are ways to measure collective intelligence and that there are some factors that seem to correlate with increased group intelligence. For example, while individual intelligence (as measured by IQ) is one relevant variable, it is not the only one, nor perhaps even the most important one. Indeed, having many high IQ individuals in a group can hinder rather than help collective intelligence. Other crucial variables include cognitive diversity, social perceptiveness of group members (including capacity for empathy and understanding), equality of participation and proportion of women. Malone cautions against overinterpreting these initial findings. There may be more noise than data in them (though the link between cognitive diversity and collective intelligence seems robust) but this is indicative of the kind of inquiry I think we need.

Linked to this, we need to think carefully about the different types of collective intelligence structure and their distinctive costs of benefits. In principle, there are as many structures as there are groups, but Malone argues that we can bring some order to the apparent chaos. He suggests that there are five main types of collective intelligence structure that humans have used to solve our problems in the past:

  • Hierarchies - These are group structures in which a single individual or small group of individuals solves problems for a larger group. In doing so, they are often supported by a legal- bureaucratic agency (or set of agencies).
  • Democracies - These are group structures in which all defined members of the group have a say in group problem-solving, usually through a formal voting procedure. The group decision or output is the aggregated sum of these votes.  
  • Markets  - These are group structures in which individuals are left to themselves and then have to transact and trade with other individuals to solve problems. The group decision or output is then an emergent property of these individual transactions. Markets usually require some external support from a hierarchy or community to ensure that the transactions and trades are enforced.
  • Communities - These are group structures that lack any formal hierarchy or voting procedure and instead involve individuals informally agreeing upon and enforcing some set of social norms for cooperation and coordination. The group solves problems by coordinating on these norms.
  • Ecosystems (state of nature) - These are not group structures but, rather, a form of anarchical social interaction. There are no norms, no voting procedures, no enforceable market transactions. The most powerful individual typically thrives while others scramble for survival.

Malone argues that each of these structures has different benefits and costs for the members of the groups and different ways of distributing the benefits to members of the group. For example, hierarchies often produce significant group benefits (aggregate social surplus) but don’t distribute those benefits evenly among the members of the society. They also tend to be quite costly to maintain: you need create cultural myths/religions, laws and enforcement agencies to maintain the  hierarchy. Democracies are somewhat similar — insofar as they produce significant group benefits and are costly to maintain — but they usually distribute group benefits more evenly. Markets and communities are different again. They can produce group benefits at lower costs (there is less formal infrastructure needed to maintain the group) but can be more unpredictable in how they distribute those benefits. Technology plays a key role in each type of structure too by both reducing the costs of group decision-making and increasing the benefits of working with the group.

Malone’s framework is just a preliminary one but it shows how we might begin to think carefully, systematically and scientifically about collective intelligence. Other work that is useful in this regard would include the series of studies by my colleague Michael Hogan on using collective intelligence in policy-formulation and political decision-making, as well as Geoff Mulgan’s thoughts on collective intelligence in his book Big Mind.

  • Proposition 4: We should be pluralists, not monists when it comes to investment in and design of collective intelligence structures. 

It is tempting to think that we need to find the ‘best’ collective intelligence structure or to assume that one structure is necessarily better than the others (e.g. that democracy is better than all the rest). But we should avoid this temptation. The reality is that modern societies blend different kinds of structure. To stick with Malone’s framework, even in liberal democratic states it is not the case that we only rely on democratic intelligence structures. Our democratic political institutions include elements of legal-bureaucratic hierarchy, may well depend on informal community norms for their success, and in turn function as a support structure for markets. In other words, we cannot have the benefits of democracies without hierarchies and communities, nor can we have the benefits of markets without hierarchies and democracies. The different structures work together and have different strengths and weaknesses.

Furthermore collective intelligence structures can decay, stagnate over time. This may mean they cease to function as effective problem-solving mechanisms. We will need to adapt and redesign them when this happens. Many people argue that this is happening now to mature democratic states. A democracy’s primary tools for increasing its problem-solving capacity and its perceived legitimacy is to enlarge the voting franchise and increase transparency and accountability. But many mature democracies may have hit the point of declining marginal returns on these mechanisms. Political stalemate and polarisation are manifestations of this. How we can redesign democracies to reinvigorate their problem-solving capacity is, I believe, a theme that David Wood will be taking up in the next talk and another major challenge for our time.

My bottom line is this: different kinds of structure are optimal for different kinds of problem. Figuring out which structure is optimal for the set of challenges we now face — ecological, sociological, and technological — is the project we should focus on. We have started to think about this in a systematic and scientific way, but we need to do much more.

Thank you for your attention.