The Case for Irrational Optimism: A Pessimistic Perspective

Tweet

I am a pretty pessimistic person.* At least that’s what I am told. I rarely see a silver lining without a cloud. When things are going well, I assume that they are about to get worse. When things are going badly, I assume that I am still some distance from rock bottom. Some of this pessimism is protective. If you are always expecting the worst you can be pleasantly surprised when reality fails to meet your expectations. But many times it is destructive. This is particularly true in personal relationships where constantly assuming the worst can be pretty frustrating for your partners and friends.

When challenged on my pessimism, I usually respond by saying that I am ‘realistic’ as opposed to pessimistic. I see things as they are, without the distorting lens of those rose-tinted glasses that others seem to wear. I explain to people that there is something known as ‘depressive realism’, which is the observed tendency for those with depression to see the world for what it is not what we would like it to be. Depressive realists are, sometimes, better able to cope with tragedy and misfortune. It’s what they expect: it doesn’t unsettle them all that much. Most people are the opposite. They are default optimists. They not only think that things are better than they are, but that they are going to get better. Their lives are viewed as upward cycles of progress.

I don’t understand these people. Surely such beliefs are irrational and without warrant? But maybe I should try to understand them? Maybe I should join their ranks and become an irrepressible and irrational optimist? This might be easier said than done, but there is some hope. My pessimism is not monolithic and unwavering. It has some subtlety. I tend to be pessimistic about particular parts of my own life, not all of it, and not about the lives of others, nor about the world as whole. I assume that I am inferior to most people and that I won’t be able to cope with the challenges life throws my way, but I think the opposite about my friends and family. Furthermore, I know that there is some evidence suggesting that those with an irrepressibly optimistic outlook — even if that optimism is unrealistic or unwarranted — do better in life, e.g. in managing potentially fatal illnesses and difficult relationships.

So should I be an irrational optimist? Should I cultivate a healthy disregard for the depressive aspects of reality? The philosopher Lisa Bortolotti has done a lot of work on this topic. Through a combination of philosophical analysis and psychological investigation, she has tried to figure out exactly which kind of irrational optimism is best. And she has a theory, which she sets out in a recent paper ‘Optimism, Agency and Success’. I want explain her theory in what follows and consider how it might apply to my own life. This isn’t intended as an exercise in self-therapy. I think the theory is interesting and worth considering. Furthermore, I hope that other people with a pessimistic outlook can learn something from this inquiry.


1. The Varieties of Irrational Optimism
Let’s start by considering the nature of optimism. Psychologists have long noticed that most people are irrational optimists. They believe things about themselves and the world that are not warranted by the evidence. For example, most people think they are above average in just about any trait or capacity you care to measure (height, intelligence, generosity etc). Some experimental findings suggest that this optimism is adaptive: it helps people deal with the stresses and strains of life. But other findings suggest that it is not: that people with irrational optimism do worse than others when the going gets tough.

The currently favoured explanation for these differences has to do with the degree of ‘reality distortion’ involved in the optimism. Optimistic beliefs that mildly distort reality are thought to be beneficial; ones that stretch the fabric of reality too far are not. So if you are like Steve Jobs, with his famous ‘reality distortion field’, you should watch out: it’s going to catch up with you eventually.
But when you think about it that can’t be quite right. After all, Jobs’s reality distortion field served him pretty well through some difficult times at Apple and elsewhere. It may even have helped him turn the company into the success it now is, even if the cancer got him in the end. Maybe reality distortion itself has nothing to do with the beneficial effects of optimism?

This is, roughly, what Bortolotti argues: that the benefits of optimism have nothing to do with the degree of reality distortion it involves. There are two reasons for this. The first is that some optimistic beliefs, even if they are not epistemically warranted, are not really ‘distortions’ at all: they are actually true. The second is that some empirical findings suggest that highly distortive beliefs — perhaps like those of Steve Jobs — can be beneficial despite their lack of realism. She consequently thinks we need a new theory of optimism that clarifies exactly when it is beneficial and when it is not. She thinks she has one.

To understand Bortolotti’s theory you need to understand some of the conceptual terrain in which it is situated. One important distinction within that terrain concerns the different kinds of optimism. The first involves positive illusions:

Positive Illusions: Positive beliefs about yourself or the world that are the product of biased reasoning and hence are not always warranted by the evidence. They come in three main forms:

Illusions of control: Assuming that you can control external events that are not really or easily within your control.

Illusions of superiority: Assuming that you are better than average with respect to various traits and capacities.

Optimism bias: Assuming that the future will be largely positive and that negative events are unlikely to feature.

Bortolotti is very clear that illusions should be kept conceptually distinct from distortions. Illusions are not always false. At least some of the people who think they are better than average with respect to intelligence or attractiveness must be better than average. The point is that they will tend to think this anyway, irrespective of whether they have good reason to. Their optimism is the product of a systematic bias in how they think about the world, not necessarily in some mismatch between their beliefs and the world. The systematic bias is what we are calling an illusion.

That’s just the first kind of optimism. The second kind is dispositional optimism. This isn’t a set of discrete beliefs about yourself or the world around you. It is a stable character trait that dictates how you react and orient yourself to the world. If you are a dispositional optimist you will tend to assume the best and respond positively toward challenges, without necessarily having specifically optimistic beliefs about what is going to happen or about those challenges. Bortolotti explains in her article that dispositional optimism is measured in a different way than positive illusions and seems to be a reasonably fixed trait that people have over the course of their lives. Positive illusions are different because they can wax and wane in response to different events.

Most of the discussion in therapeutic psychology has focused on the benefits or disadvantages of positive illusions. Since positive illusions are systematic biases, and since they seem to be responsive and malleable, the question arises as to whether we should discourage or encourage them. When it comes to irrational beliefs more generally (i.e. those not specifically related to optimism) two theories have emerged:

The Traditional View: Associated with some of the early founders of cognitive behavioural therapy, holds that irrational beliefs and biases are contrary to our well-being. One major goal of psychotherapy is thus to remove these biased beliefs about reality and replace them with a more evidence-based view of reality.

The Tradeoff View: Some irrational beliefs are good, some are bad. It’s all about achieving the right balance.

Both of these views make predictions about positive illusions and their link to psychological well-being. The traditional view predicts that positive illusions are counterproductive and should be removed. The tradeoff view holds that they should be encouraged in certain circumstances. It’s just a case of being able to identify those circumstances. Although she has problems with both views — because they conflate bias with distortion of reality — Bortolotti develops a theory that is more in line with the tradeoff view.


2. Bortolotti’s Theory - Agency-Based Optimism
The essence of Bortolotti’s theory is that it is an agency-based view of optimism. One of the defining features of being a human is the experience of agency. We have goals, projects and plans and we use our capacities to achieve those goals, projects and plans. When we act as agents we assume, to at least some extent, that our actions allow us to control the outcome. This may not always be true, of course. On some occasions our actions may have little causal effect on the outcomes we desire. Buying a lottery ticket and squeezing it tightly to your chest during the draw doesn’t make it more likely that you will win the lottery. But oftentimes we have an illusion of control: we really think we can control the outcome even though we have no good reason to believe this. This positive illusion of control, along with a positive belief concerning one’s capacities to learn and develop one’s talents, is central to Bortolotti’s theory. She argues that those who think they have the capacity to realise their goals, even when this is a distortion of reality, are more likely to do so than those with a more fatalistic outlook.

But that’s only part of it. It’s not enough that people have positive illusions concerning their capacity to achieve certain outcomes. They must also have positive attitudes concerning those outcomes. In other words, they must believe that what they are doing is valuable or worthwhile. This gives them a sense of purpose and optimism, which combined with the belief in their capacity to bring it about, sustains them in the face of a world that doesn’t always agree. Those with more doubts about the merits of their plans will have less resilience and perseverance. Again, this is true even if there is some distortion of reality involved. Many people might question whether developing the iPhone, or a sleeker laptop, really is good thing for humanity. What mattered for Steve Jobs was that he thought it was. It was a mission that gave him a sense of purpose and meaning. I suspect the same is true for many other high-achieving individuals.


I have tried to illustrate this below.

To be clear, this is not a theory that Bortolotti plucks out of thin air. She develops it by inference from two particular case studies of optimism. The first concerns positive illusions in personal relationships. The second concerns positive illusions in healthcare. I’ll briefly describe both.

The relationship case study focuses on positive beliefs concerning one’s partner. It’s common enough for those in the first blushes of infatuation to idealise their partners. They think their partners are better than average and only see the good in them. This idealisation is usually unwarranted. Nobody is perfect. Nevertheless, it is commonly thought to help solidify a relationship in the short-to-medium term.** The danger is in the long-term. What happens when evidence mounts suggesting that the partner isn’t that great after all?

According to one theory — the disappointment theory — we’d expect the initial idealisation to have a negative effect on the relationship. As the partner fails to live up to your original conception of them you will grow disappointed and weary, maybe eventually ending the relationship. According to another theory — the self-fulfillment theory — we’d expect it to have a positive effect. The idealising partner will tend to ignore or downplay evidence that contradicts the idealisation and work positively to ensure that the partner lives up to the initial expectations. This latter theory has been endorsed by the empirical work of Sandra Murray and her colleagues. They find that those who idealise their partners do better in the long-run. They argue that there are three mechanisms that help to do this: (i) buffering, i.e. the idealising partner has a strong sense of security and confidence in the relationship and are not swayed by conflict or doubt; (ii) transformation, i.e. the idealising partner reinterprets weaknesses as strengths and confronts problems rather than runs away from them; and (iii) reflective appraisals, i.e. the idealised partner starts to see themselves as the idealising partner does and they try to live up to the idealisation. This is all consistent with the agency-based theory of success: the initial idealisations might be wildly distorted but if you have them, you are likely to work to narrow the gap between reality and perception.

The healthcare case study focuses on positive beliefs concerning one’s likelihood of recovering from or coping with a serious illness. Looking at some famous studies done by Shelley Taylor and colleagues on patients with HIV and breast cancer, Bortolotti finds evidence suggesting that those with both the illusion of control over health outcomes and optimism about their future health prospects do better than those with a more pessimistic outlook. There are two alleged mechanisms at play here: (i) those who are optimistic experience less stress, which may have a deleterious effect on health and (ii) those who are optimistic about their capacity to control their health outcomes are more likely to engage in protective behaviours. This is disconcerting for someone like me who thinks that we should be sceptical about many health-related claims — particularly those concerning which lifestyle choices are protective — but is again consistent with the agency-based theory of optimism. Also, as Bortolotti is keen to point out, in none of these case studies does the degree of realism appear to be a relevant factor: sometimes the patients had wildly distorted views about their health and their capacity to control it.

Although she doesn’t mention it, I think Philip Tetlock’s work on ‘superforecasters’ also lends credence to the agency-based theory. In trying to figure out who was best at predicting future events, Tetlock found that people who thought it was genuinely possible for them to do this and, more importantly, that this was a skill that they could hone and develop, did best.

You might wonder whether there is a paradox in all of this. If people with irrationally optimistic beliefs about their agency do better than those with more pessimistic and fatalistic beliefs, is their optimism actually irrational? Well, I guess that all depends on what you mean by ‘irrational’. Bortolotti doesn’t use the term ‘irrational’ in setting out her theory, but I think it’s clear that she would make a distinction between subjective and objective rationality. The better-performing optimists might be subjectively irrational (i.e. acting without epistemic warrant), even if they are objectively rational (i.e. there is evidence to suggest that they do better than the pessimists). Even then, the degree of objective rationality might be in doubt. An irrational optimist might do better than a pessimist, but their degree of optimism might not match how much better they actually do.


3. So should I be an irrational optimist?
The upshot of all this is that there does seem to be some reason to endorse the agency-based theory of optimism. Obviously, I would like more research to be done to see whether it holds up under a variety of different conditions, but I am willing to accept, for the sake of argument, it for the time being. The critical question for me is whether, assuming it’s correct, I should take it onboard in my own life. Should I try to cultivate an irrationally optimistic outlook? Should I be more confident in my own aspirations and abilities? Should I try to develop the belief that I can control outcomes that I currently think are beyond my reach? Should I assume that it is possible to hone my talents and abilities to achieve my goals?

Maybe. It’s worth noting that Bortolotti herself is cautious. Although she thinks the evidence does support the idea that optimistic, agency-related beliefs are positively correlated with achieving outcomes, she warns us against assuming that we are invulnerable. That seems like good advice to me: believing that you are invulnerable is just an optimistic form of fatalism, which is not in line with the agency-based theory. But even if I accept this note of caution, I find myself at odds with the theory.

The problem for me is with the other half of it: the optimism about the goals themselves, and not just the capacity to achieve them. This is where I fall down. There are times when I am pretty optimistic about my own capacities. For example, I’m optimistic about my capacity to write this piece. I think I’m better than average at writing pieces like this; that I can improve at doing so; and that I will succeed in finishing it. I don’t suffer from writer’s block or some paralysis of self-confidence. But that’s because I think what I am doing has some value. It’s fascinating to learn about the relationship between optimism and life outcomes; I feel like I’m learning something about myself through the process of writing; and I’m sharing something that might be of value to others. All of this sustains me in my actions.

But a lot of the time I doubt the wisdom of what I am doing, or I am deeply conflicted about it. On these occasions, I tend to lose all positive self-belief. This is particularly true in relation to my academic work and my personal relationships. I have no idea whether what I am doing with my life is worthwhile. This is true at both an objective and subjective level. In other words, I have doubts about whether what I am doing is good for the world as a whole and whether it contributes to my own well-being and happiness. I worry, for example, that much of the academic work I do is without value or that it’s value is, at the very least, highly uncertain. Some of my academic colleagues are moral crusaders. They are trying to make the world a better place, for example by arguing for human rights and justice for all. They seem very confident that they are doing the right thing. But I have no idea whether my talents and expertise can be used to make the world a better place. Does the world need another piece on the ethics of sex robots? Probably not. Do I get a sense of purpose from writing about it? I’m not sure. Indeed, I find that it’s only really when I ignore the bigger picture, and focus purely on curiosity and intrinsic fascination, that I attain some degree of optimism about what I’m doing. But then I feel that I’m being selfish and self-indulgent and so I step back and try to get a wider perspective, from which I lose all sense of optimism.

I’m sure these thoughts are not uncommon.*** I suspect that even those people I envy for their irrepressible optimism and moral conviction entertain them from time to time. But when they are pervasive, these thoughts undermine the potential therapeutic uses of irrational optimism. Cultivating irrational optimism about one’s capacities and talents might be possible, and hence valuable, if one already has goals about which one is motivated and optimistic. But if you lack those goals, and if you have been trained to question and critique nearly every goal, it’s difficult to find the motivation for doing this. You somehow need to fall into a set of goals that gets your juices flowing to get the process started. If you stay at the reflective, critical level, you never will.

The only thing I can think of that might work for someone like me would be something like Will MacAskill’s combination of effective altruism and moral uncertaintism. For those that don’t know, Will MacAskill is one of the founders of the effective altruism movement. This movement is dedicated, at the broadest level, to doing the best for the world, whatever that turns out to be. Proponents of effective altruism do advocate specific policies for doing good, several of which have been controversial (if you’re interested, I wrote a series of blog posts on critiques of effective altruism). But what interests me more is the way in which MacAskill’s academic work on moral uncertainty complements his approach to effective altruism. MacAskill’s academic work acknowledges that there is plenty of uncertainty about what the best thing to do is. It then tries to work out a decision procedure for doing the right thing even when you don’t know what the right thing to do is. Figuring this out seems like it might be only thing that someone like me could be optimistic about since it at least acknowledges and tries to work with pervasive uncertainty about the value of one's actions. But there’s a problem with this too since there is room for doubt and uncertainty about the best approach to dealing with uncertainty.

So, while I would like to be an irrational optimist, I’m not sure that I can be one given my general attitude to the world and my place within it. I can, at best, give a conditional thumbs up to following Bortolotti’s model: if I can find the right goals, then I probably should cultivate irrational optimism about my ability to achieve them. But until I do that, I’ll have to hang back. I may like the idea of putting a dent in the universe, but before I proceed I want to make sure it’s the right kind of dent.


* I don’t know if this has always been true. I’ve probably become more pessimistic as I’ve aged and I’m probably currently at a personal peak of pessimism. That said, I have always been drawn to a darker view of the world and to the sense that much of human life is tragic. Friends of mine from school will confirm. 

** For what it’s worth, this seems inconsistent with my own experience. I often see the worst in partners in the short-term but then eventually grow to like them more as evidence of their goodness accumulates. 

*** For those who are familiar with it, Thomas Nagel’s article ‘The Absurd’ echoes some of these thoughts.

Subscribe to the newsletter