Should we stop killer robots? (1) - The Anti-Codifiability Objection

The Campaign to Stop Killer Robots outside Westminster

There is well-publicised campaign to Stop Killer Robots. The goal of the campaign is to pre-emptively ban the use of fully autonomous weapons systems (AWSs). This is because:

Allowing life or death decisions to be made by machines crosses a fundamental moral line. Autonomous robots would lack human judgment and the ability to understand context. These qualities are necessary to make complex ethical choices on a dynamic battlefield, to distinguish adequately between soldiers and civilians, and to evaluate the proportionality of an attack. As a result, fully autonomous weapons would not meet the requirements of the laws of war.

(Campaign Website, ‘The Problem’)

But is this really true? Couldn’t it be the case that sufficiently advanced AWSs would be morally superior to human beings and that their deployment could be safer and more morally effective than the deployment of human beings? In short, is there really a fundamental moral line that is being crossed with the development of AWSs?

These are questions taken up by Purves, Jenkins and Strawser (‘Purves et al’) in their paper ‘Autonomous Machines, Moral Judgment, and Acting for the Right Reasons’. As they see it, existing objections to the use of AWSs suffer from two major problems. First, they tend to be contingent. That is to say, they focus on shortcomings in the existing technology (such as defects in targeting systems) that could be overcome. Second, they tend not to distinguish between weaponised and non-weaponised autonomous technology. In other words, if the objections were sound they would apply just as easily to other forms of autonomous technology (e.g. driverless cars) that are generally encouraged (or, at least, not so actively opposed).

They try to correct for these problems by presenting two objections to the use of AWSs that are non-contingent and that offer some hope of distinguishing between AWSs and non-weaponised autonomous systems. I want to have a look at their two objections. The objections are interesting to me because they rely on certain metaphysical and ethical assumptions that I do not share. Thus, I think their case against AWSs is weaker than they might like. Nevertheless, this does lead to an interesting conclusion that I do share with the authors. I’ll try to explain as I go along. It’ll take me two posts to cover all the relevant bits and pieces. I’ll start today by looking at their first objection and the criticisms thereof.


1. The Anti-Codifiability Objection
Purves et al’s paper must be appreciated in the appropriate context. That context is one in which just war theory plays a significant role. They refer to this and to leading figures in the field of just war theory throughout the paper. Many of their arguments presume that in order for AWSs to be permissibly deployed in warfare requires compliance with the principles of just war theory. I’m willing to go along with that with two caveats: (i) I cannot claim to be an expert in just war theory — I have a very sketchy knowledge of it, but many of the principles that I am aware of have their equivalents in mundane moral contexts (e.g. the principle of right intention, discussed in part two); (ii) we shouldn’t assume that the principles of just war theory are beyond criticism — several of them strike me as being questionable and I’ll indicate some of my own qualms as I go along.

Some of the relevant principles of just war theory will be introduced when I outline their arguments. One basic background principle should be made clear at the outset. It is this: the actors in a war should be moral agents. That is to say, they should be capable of appreciating, responding to and acting for moral reasons. They should be capable of exercising moral judgment. In addition to this, moral accountability should be present in military operations: there must be someone acting or controlling the actions of soldiers that can be rightfully held to moral account. The major concern with AWSs is that they will circumvent these requirements. AWSs operate without the direct control and guidance of human commanders (that’s why they are ‘autonomous’). So the only way for them to be legitimately deployed is if they themselves have the appropriate moral agency and/or accountability. Both of the objections mounted by Purves et al dispute this possibility.

The first of these objections is the anti-codifiability objection. This one is based on the notion that an AWS must be programmed in such a way that they can replicate human moral judgment. In order for this to happen, human moral judgment must be codifiable, i.e. capable of being reduced to an exhaustive list of principles. This is because robots are programmed using lists of step-by-step instructions that are specified (in some fashion) by their designers. The problem for AWSs is that moral judgment may not be codifiable. Indeed, several moral philosophers explicitly argue that it is not codifiable. If they are right then AI could not, in principle, be moral actors of the appropriate kind. Their deployment in war would not comply with the relevant principles of just war theory.


To put this a little more formally:

• (1) In order for AWSs to exercise moral judgment, moral judgment must be codifiable. 

• (2) Moral judgment is not codifiable. 

• (3) Therefore, AWSs cannot exercise moral judgment.

You could tack on further premises to this simple version of the argument. These further premises could lead you to the conclusion that AWSs should not be used in a military context.

Interestingly, Purves et al don’t spend that much time defending the first premise of this argument. They seem to take it to be a relatively obvious truth about the nature of robot construction. I guess this is right in a very general sense: contemporary programming systems do rely on step-by-step instructions (algorithms). But these instructions can take different forms, some of which may not require the explicit articulation of all the principles making up human moral judgment. Furthermore, there may be alternative ways of programming a robot that avoid the codifiability issue. I’ll say a little more about this below.

Purves et al dedicate most of their attention to the second premise, identifying a number of different moral theorists who believe that moral judgment is not codifiable. Two famous examples would be John McDowell and Jonathan Dancy. McDowell famously argued against codifiability in his 1979 paper ‘Virtue and Reason’ (he was focusing in particular on moral virtue, which we can view as part of moral judgment more generally):

[T]o an unprejudiced eye it should seem quite implausible that any reasonably adult moral outlook admits of any such codification. As Aristotle consistently says, the best generalizations about how one should behave hold only for the most part. If one attempted to reduce one’s conception of what virtue requires to a set of rules, then, however subtle and thoughtful one was in drawing up the code, cases would inevitably turn up in which a mechanical application of the rules would strike one as wrong.

(McDowell 1979, 336)

Dancy has a similar view, arguing that general moral principles are not part of human moral judgment. Instead, human moral judgment is particularised, i.e. always tailored to the particularities of a real or imagined moral decision problem. This has led to a school of moral theory known as moral particularism. What’s more, even those who don’t explicitly endorse McDowell or Dancy’s views have argued that moral judgment is highly context sensitive, or that to make use of articulable moral principles one must already have a faculty of moral judgment. The latter view would pose a problem for defenders of AWSs since it would rule out the construction of a faculty of moral judgment from the articulation of moral principles.

If moral judgment is not capable of being codified, what is required for it to exist? Purves et al remain agnostic about this. They highlight a range of views, some suggesting that phenomenal consciousness is needed, others suggesting a capacity for wide reflective equilibrium is the necessary precursor, and yet others focusing on practical wisdom. It doesn’t matter which view is right. All that matters for their argument is that none of these things is, in principle, within the reach of a computer-programmed robot. They think they are on firm ground in believing this to be true. They cite the views of Hubert Dreyfus in support of this belief. He argues that a computer would have to rely on a discrete list of instructions and that these instructions could never replicate all the tacit, intuitive knowledge that goes into human decision-making. Similarly, they note that only a minority of philosophers of mind think that a robot could replicate human phenomenal consciousness. So if that is necessary for moral judgment, it is unlikely that an AWS will ever be able to exercise moral judgment.

That is the essence of their first objection to AWSs. Is it any good?


2. Assessing the Anti-Codifiability Object
Purves et al consider two main criticisms of their anti-codifiability objection. The first argues that they fail to meet one of their own criteria for success — non-contingency — because AWSs could potentially exercise moral judgment if some of their claims are false. The second argues that even if AWSs could never exercise moral judgment, their deployment might still be morally superior to the deployment of humans. Let’s look at both of these criticisms in some detail.

The first criticism points out that two things must be true in order for Purves et al’s objection to work: moral judgment must not be codifiable; and AWSs must (in principle) be incapable of exercising such non-codifiable moral judgment. Both of these things might turn out to be false. This would render their argument contingent upon the truth of these two claims. Hence it fails to meet their non-contingency criterion of success for an objection to AWS.

Purves et al respond to this criticism by accepting that the two claims could indeed be false, but that if they are true they are true in-principle. In other words, they are making claims about what is necessarily* true of morality and AWSs. If they are correct, these claims will be correct in a non-contingent way. It would be a hollow victory for the opponent to say the objection was contingent simply because these two in-principle claims could be false. Many in-principle objections would then share a similar fate:

While it is possible that what we say in this section is mistaken, unlike other objections, our worries about AWS are not grounded in merely contingent facts about the current state of AI. Rather, if we are correct, our claims hold for future iterations of AI into perpetuity, and hold in every possible world where there are minds and morality.

(Purves et al 2015, 858-859)

To some extent, I agree with Purves et al about this. But largely because this is a particularly unfavourable way to frame the criticism. I think the problem with their argument is that the in-principle claims upon which they rely are much weaker than they seem to suppose. Both claims seem to me to rest on metaphysical suppositions that are deeply uncertain. Although there is some plausibility to the anti-codifiability view of moral judgment, I certainly wouldn’t bet my life on it. And, more importantly, I’m not sure anyone could prove it in such a way that it would encourage everyone to reject the deployment of AWSs (the context for the paper is, after all, the need for a campaign against killer robots). The same goes for the inability of AWSs to replicate moral judgment. In general, I think it is a bad idea to make claims about what is or is not in principle possible for an artificial intelligence. Such claims have a tendency to be disproven.

More particularly, I think Purves et al ignore techniques for automating moral judgment that could get around the codifiability problem. One such technique is machine learning, which would get machines to infer their own moral principles from large sets of sample cases. Here, the programmer is not instructing the machine to follow an articulated set of moral principles, rather they are instructing it to follow certain statistical induction rules to come up with their own. There are definitely problems with this technique, and some have argued for limitations, but it is getting more impressive all the time. I wouldn’t be sure that it is limited in the way that Purves et al require. Another technique that is ignored in the paper is human brain-emulation, which could also get around the problem of having to articulate a set of moral principles by simply emulating the human brain on a silicon architecture. This technique is in its infancy, and perhaps there are limits to what it is possible to emulate on a silicon architecture but, once again, I’d be inclined to avoid making absolute claims about what is or is not possible for such techniques in the future.

To be fair, Purves et al do concede the bare possibility of a machine emulating human moral judgment. They argue towards the end of their paper that if a machine could do this it would throw up some deeply troubling questions for humanity. I happen to agree with them about this but I’ll address it in part two.

The second major criticism of the anti-codifiability objection holds that even if it is true, it could be that AWSs are morally superior to human moral agents on other grounds. Purves et al develop this criticism at some length in their paper. If you think about it, there are three main types of mistake that an AWS could make: (i) empirical mistakes, i.e. the system might fail to detect empirical facts that are relevant to moral decision-making; (ii) genuine moral mistakes, i.e. the system might fail to use the right moral judgments to guide its actions (ii) practical mistakes, i.e. the system might implement a morally sound decision in the wrong manner (e.g. reacting too late or too soon). They concede that AWSs might be less inclined to make empirical and practical mistakes. In other words, the AWS might be better at selecting appropriate targets and implementing morally sound decisions. We could call this kind of behaviour ‘moral conformism’. Their objection is simply that even if the AWS was better at conforming with moral requirements it would not be acting for the right moral reasons. Hence it would continue to make moral mistakes. The question is whether the continued presence of moral mistakes is enough to impugn the use of the AWS.

Let’s put it in practical terms. Suppose you have a human soldier and a conforming AWS. The human soldier is capable of exercising sound moral judgment, but they are slow to act and make mistakes when trying to discover morally salient targets in the field. Thus, even though the soldier knows when it is morally appropriate to target an enemy, he/she selects a morally inappropriate target 20% of the time. The AWS is much faster and more efficient, it is more likely to select morally salient targets in the field, but it is incapable of using moral reasons to guide its decision. Nevertheless, the AWS selects morally inappropriate targets only 5% of the time. Should we really favour the use of the human soldier simply because they are acting in accordance with sound moral judgment? The critics say ‘no’ and their view becomes even more powerful when you factor in the moral imperfection of many human agents.

Purves et al concede some of the force of this objection. Their main reply is to argue that, in order to comply with the requirements of just war theory, agents in a combat environment must act in accordance with right moral reason. This leads to their second, in principle, objection to the use of AWSs. I’ll take a look at that in part two.


* I say nothing here about the particular brand of necessity at work in the argument. It could be logical or metaphysical necessity, but I suspect Purves et al only need physical necessity for their argument to work (at least when it comes to what is or is not possible for AWSs).