Tuesday, December 29, 2009

Explanations: A Gentle Introduction

Welcome to yet another series! This time out I am going to be covering explanations. The series is going to be quite wide-ranging but will have one question at its core: what makes for a good explanation?

There is a danger here. The form of this question could tempt us into an arcane and esoteric exercise in academic self-indulgence. You know the kind. If we are not careful, we could well spend all our time with sentences like the following: "x is a good expanation iff it has P, Q, and R, and has no S, T or U."

I want to avoid this kind of dry treatment. Explanations are the glue that hold together the fragile and disparate territories of the intellectual commonwealth: understanding their complexities and their simplicities, and their successes and their failures, is essential.

So, in order to give the topic the treatment it deserves, I want to avoid abstract analyses and use instead the medium of practical illustration. In other words, I want to compare and contrast several types of explanations across several different domains: philosophical, historical, scientific and religious.

This introductory post has two goals. First, it gets the necessary abstractions out of the way by considering the form that a good explanation should take. Second, it offers a simple example of explanation-in-action by going over one of Sherlock Holmes's fictional cases.

1. Abductive Inference and Explanatory Virtues
When I speak about explanation, I speak in particular about explanations that are arrived at through abductive inference. This was a method first formalised by Charles Sanders Pierce, one of the three great American pragmatists.

An abductive inference looks like this:

D is some collection of data (includes facts, observations, and givens)
H is some hypothesis that would, if true, explain D
No other hypothesis explains D as well as H
Therefore, H is probably true

As can be seen, abductive inference reaches probabilistic conclusions, not definitive conclusions. It also involves the comparison of different hypothesis: weighing their respective merits against one another. Human inquiry always begins in the middle, i.e. with the contradictions and tensions in our present worldview, we must always consider different ways of resolving these tensions and contradiction. We cannot consider one hypothesis in isolation from everything else that we know.

But how can we weigh one explanation against another? How can we know when one explanation is stronger? This is where the idea of explanatory virtues becomes important. The virtues are a set of criteria that can be used to assess explanations. Luke over at commonsenseatheism provides a good list of these virtues here. I can't improve upon it.

The image below summarises the formal aspects of abductive inference and includes a list of the explanatory virtues.

2. Explanation in action: Sherlock Holmes and the Adventure of the Three Students
To ease our way into future practical examples, it will be useful to consider something frothy, frivolous and yet instructive.

Everybody's favourite fictional detective, Sherlock Holmes, is often thought to be the embodiment of abductive reasoning. In solving the cases presented to him by the troubled citizens of London he marshals the evidence, considers the different hypotheses, eliminates the impossible ones and accepts the remaining one, however improbable it may at first appear to be. (This is a paraphrase of one of Holmes's famous sayings, but it is questionable, see here - an improbable answer is more likely to be the result of an improper question).

To show how Holmes uses abduction, I will look at how he solves the case presented in the story "The Adventure of the Three Students". It is one of the more forgettable of Conan Doyle's efforts, but its academic setting appeals to the student in me.

The Problem
The story begins with Holmes and Watson firmly ensconced in one of England's famous university towns (the precise location is never divulged). They are interrupted by one of the professors, a man named Soames, who presents Holmes with a problem befitting his intellect.

Earlier that afternoon, Soames was going over the proofs for the Greek scholarship exams, which are being held the following morning. He had to visit a colleague and so left the proofs on the desk in his room.

When he returned he found that the proofs had been disturbed. They were left in various positions around the room, including one left by the window overlooking the quad. There were other notable disturbances: strange globules of mud were deposited in various locations, and a broken pencil had been left behind. The latter was presumably used by the intruder to copy the proof.

Soames alerted his servant, named Bannister, as soon as he suspected that there had been an intruder. Bannister became quite upset upon hearing this and collapsed into one the chairs in Soames's room. After calming him, Soames walked straight over to Holmes to seek his help.

Soames informs Holmes that the intruder is likely to be one of the three students with whom he shares the building in which he lives. All three are due to sit the exam, so all three have an incentive to cheat.

The student living on the ground floor is a hard-working, scholarly and athletic but disadvantaged youth named Gilchrist. His father had once been rich, but had lost all his money in that all too common vice of the English gent: gambling.

The second floor is occupied by an Indian student named Daulat Ras. He is quiet and intelligent, although Greek is his weakest subject.

The third floor is inhabited by Miles McLaren. He is brilliant but intemperate, wild, and morally circumspect. He has had previous run-ins with the college authorities and was almost expelled on one occasion.

Holmes goes over Soames's room with his usual care, interviews the servant Bannister and visits each of the student's rooms, with the exception of McLaren's who wouldn't let them in. Satisfied with his efforts, he promises Soames that he will resolve the case first thing the following morning (just before the exam is due to start).

The following morning Holmes delivers the goods: Gilchrist is revealed to be the guilty party and, what's more, the servant Bannister is implicated in the events. Gilchrist confesses, withdraws from university and all is right in the world.

How did Holmes manage to correctly identify the intruder?

The Explanation
As noted, all three students had an incentive to cheat on the exam. So we are weighing three hypotheses (i.e. potential explanations of the events) against each other: (i) Gilchrist did it; (ii) Ras did it; or (iii) McLaren.

For the most part, the available data is equally well accounted for by each hypotheses. But there are three crucial points at which the Ras- and McLaren-hypotheses break down:
  • Holmes reasons that the intruder was likely to have seen the proofs on Soames's desk as they passed his window - to have simply stumbled into the room and find them was too much to ask. Neither Ras, nor McLaren were tall enough to see in the window.
  • The strange balls of mud found in the room came from the sandpit over on the long jump practice ground. Gilchrist was a long jumper and had been practicing that afternoon. That's almost a QED right there.
  • The other puzzling fact was the behaviour of Bannister. His collapsing into the chair on being told that there may have been an intruder seemed over-the-top to Holmes. He reckoned Bannister was trying to cover up for Gilchrist because Gilchrist was still hiding in the room. After Soames left to see Holmes, Bannister could sneak Gilchrist out. This is revealed to have been true. Indeed, Bannister and Gilchrist were connected because Bannister used to work for his father.
What we have then is a classic example of an abductive inference: three hypotheses are on the table, they are compared on the basis of their explanatory virtues, and one is left standing.

The diagram below illustrates the form and virtue of Holmes's explanation. It fills in some details missing from the summary to this point.

Alas, that brings this post to a close. In future contributions to this series I will look at various scientific, philosophic and religious explanations. All the time adhering to the abductive method.


  1. It is perhaps noteworthy that Holmes is often quoted as saying:

    "When you have eliminated the impossible, whatever remains, however improbable, must be the truth."
    Sir Arthur Conan Doyle (1859-1930), English author. Sherlock Holmes, in The Sign of Four, ch. 6 (1889).

    This is actually deduction (not abduction), but it also fallacious:


  2. I edited the post to include that link.

    I still think that in practice Holmes used abductive reasoning.

  3. Actually that was a slightly ridiculous comment: I haven't extensively analysed every Holmes story so I couldn't really say.

    At a minimum, I think the example I discuss illustrates how one could take account of the explanatory virtues when comparing hypotheses and thereby arrive at the most probable explanation (and not just eliminate impossible explanations).

    My interpretation of the story could be off the mark. Don't think it matters for the purposes of illustration.

  4. Part of the problem (as always) is we induce what is possible "When you have eliminated the impossible.." ( In fact we induce lots of things we need before we can start reasoning at all, the uniformity of nature, for example.

  5. I hope you continue this series and make an index for it. Just don't burn yourself out!

  6. Thanks, I will try. I'm working on related material for my thesis at the moment so I'll add that as soon as I'm done.

    I'm going to look at scientific explanations first, then historical explanations, then naturalism v. theism. I'm trying to get a copy of theism and explanation before those later parts.

    Unfortunately, this is the first week back teaching so I've been preoccupied.

  7. Are there any solid epistemic criteria for ruling out supposed explanations without appeal to other possible explanations? For instance, how would we be justified in saying that "X isn't an explanation" or "X is a bad explanation" without appealing to other alternative explanations?

    In short: when given a choice between professing tentative belief in explanation E for phenomenon P and admitting "I don't know why P, but I don't buy E", how could one rationally choose between the two options?

    Thanks for the great post!

  8. Well, I think the explanatory virtues are solid epistemic criteria. But you are right that they are derived from other explanations. We kind of muddle along, building our explanatory models from the bottom-up rather than from the top-down.

    The other possibility is to analyse explanations using a Bayesian or Likelihoodist framework. That is to say (following your notation), by seeing which E raises or lowers the probability of P the most, or seeing which E is most probable given P. The former being the likelihood approach and the latter being the Bayesian.

    Although this probabilistic perspective works quite well in certain contexts, it runs into problems when the probabilities are inscrutable or unascertainable. Which is more often than we'd like.

    Dawes' theism and explanation touches upon these issues to some extent. And Sober's Evidence and Explanation provides a pretty good introduction to the Bayesian and Likelihoodist approaches, as well as an application of the Likelihoodist framework to evolutionary biology.