109 - How Can We Align Language Models like GPT with Human Values?

Tweet

In this episode of the podcast I chat to Atoosa Kasirzadeh. Atoosa is an Assistant Professor/Chancellor's fellow at the University of Edinburgh. She is also the Director of Research at the Centre for Technomoral Futures at Edinburgh. We chat about the alignment problem in AI development, roughly: how do we ensure that AI acts in a way that is consistent with human values. We focus, in particular, on the alignment problem for language models such as ChatGPT, Bard and Claude, and how some old ideas from the philosophy of language could help us to address this problem.

You can download the episode here or listen below. You can also subscribe the podcast on Apple, Spotify, Google, Amazon or whatever your preferred service might be.

Relevant Links

• Atoosa's webpage

• Atoosa's paper (with Iason Gabriel) 'In Conversation with AI: Aligning Language Models with Human Values'

Subscribe to the newsletter

Oxford Philosophy Mid-20th Century: A Five Book Review

Tweet

Over the past year, I have read several books about Oxford philosophy in the mid-20th Century. This was the golden era of linguistic philosophy -- the time when Gilbert Ryle, John Austin and the ghost of Ludwig Wittgenstein stalked the seminar room.

It's odd that I have dedicated so much time to reading about this era. If asked for my opinion, I would say that I don't think much of it, or the philosophy it produced, although I appreciate its impact. But these books caught my attention and I thought it might be interesting to offer some very quick reviews of them.

The books are, for the most part, exercises in biographical or narrative history. They are about personalities as much, if not more, than philosophical doctrines. Consequently, my reviews won't focus on the substance of the philosophical positions defended by the different members of the Oxford School. I'll just focus on whether the books were insightful and enjoyable to read.

First, a little bit of background. The central tenets of linguistic philosophy are, perhaps, best summed up by one of its intellectual forefathers, the now-obscure Cook Wilson (1849-1915), who taught and inspired a number of its protagonists:

The authority of language is too often forgotten in philosophy with serious results. Distinctions made or applied in ordinary language are more likely to be right than wrong. Developed, as they have been, in what may be called the natural course of thinking, under the influence of experience, and in the apprehension of particular truths, whether of everyday life or science, they are not due to any preconceived theory...On the other hand, the actual fact is that a philosophical distinction is prima facie more likely to be wrong than what is called a popular distinction, because it is based on a philosophical theory which may be wrong in its ultimate principles. 

(Wilson 1926, taken from Rowe 2023, p 80)

 

There is wisdom in what Wilson said. The error rate of grand metaphysical theories is likely to be high, and the zeal to create a coherent and compelling philosophical theory can lead us astray. Whether ordinary language is a repository of wisdom is, I think, more debatable, and whether ordinary language philosophy, as practiced by the likes of Austin and Ryle, ever got close to studying ordinary language (as opposed to the 'ordinary' language of a narrowly-circumscribed elite) is questionable. Although it is now widely condemned, I think Ernest Gellner's critiques of ordinary language philosophy -- in his controversial book Words and Things -- is more right than wrong.

Still, it would be churlish to deny that ordinary language philosophy exerts considerable influence over modern analytic philosophy. Close attention to words and concepts, precise analysis, technical detail, and argumentation, are all features of the modern philosophical literature, particularly in the Anglo-American world. Even in the areas of philosophy with which I am most familiar -- moral, political, legal and religious -- one feels the dead hand of Oxford gently nudging one in the back. Furthermore, Oxford philosophy in the middle 20th-century was not a monolith. As practiced by John Austin, ordinary language analysis was a pedantic and priggish exercise. But in the hands of others, there was something fun and exploratory about it. And there always internal resistance to it. Bernard Williams, Stuart Hampshire and Peter Strawson, for instance, both criticised and moved beyond the constraints imposed by the likes of Austin. And a group of female philosophers (more on them below) were also persistent gadflies.

Anyway, here are my reviews of the five books.

1. A Terribly Serious Adventure by Nikhil Krishnan

If you want a basic, very readable, introduction to ordinary language philosophy, this is the place to start. It's a comprehensive, yet breezy, overview of the movement from 1900-1960 (going by the cover), though primarily focused on the 1930s-1960s. It covers all the key movers and shakers, from Ayer to Wittgenstein, and many more in between.

What I most appreciated about this book was its attempt to put linguistic philosophy in its context, as partly a reaction to British Idealism, which dominated in the late 1800s, and partly an effort to modernise and professionalise the discipline. I also enjoyed the writing, peppered with mordant and colourful observations, such as this:

Under whatever name, linguistic philosophy -- or something descended from it -- still dominates the academic philosophy of the English speaking world. Rumour has it that brave evangelists have even managed to find converts in deepest France. (p7)

 

The occasional observations about the impact that the two world wars had on Oxford philosophy, both in terms of its personnel (who survived or was lucky enough to avoid the draft) and how they acted, were sobering. To be fair, this is a theme that crops up in all the other books I read, but consider this passage about Gilbert Ryle, who went on to have an outsized role in the movement:

He was born in the late summer of 1900, a lucky year to be born an English boy. Just a year older and there was every chance that he would have been one of the 149 boys from Brighton College who died at Ypres, the Somme or in Palestine...As it was, Ryle survived, eighteen years old at the Armistice, and ready to head for - or 'to go up' to - Oxford. (p 15)

 

In one of the other books -- I think it was the one by MW Rowe -- it is noted that there were no Oxford philosophers born between 1890-1897 that left any record of publications or influence. The reason 'why?' does not need to be stated.

2. The Fly and the Fly Bottle by Vid Mehta

This is the oldest book I read. Published in the early 1960s, it is a collection of articles by the New Yorker writer Vid Mehta. I read it largely because it kept cropping up as a reference in the other books. The articles bear the hallmarks of the New Yorker style (if you read the New Yorker, you will know). The articles are exercises in reportage: colour pieces about academics and their opinions and very well written for that. The opening chapter, in particular, is a masterpiece of metaphorical construction. Mehta, a former Oxford student himself, visited academics, interviewed them, and reported back about what they said, with some observations and asides of his own.

The book is divided into two parts. Only the first part is about philosophy and it centres on the controversy arising from Ernest Gellner's book-length critique of ordinary language philosophy. This is the book Words and Things that I mentioned in the introduction. As such, it ends up being something of an extended obituary for ordinary language philosophy, summarising the movement as it was dying out. Mehta is like a contemporary Gibbon, reporting on the decline and fall of a once proud empire. Parts of it are quite sad (relatively speaking). I was particularly struck by this reflection on John Austin (dead by the time Mehta wrote his book) by one of his close friends, Geoffrey Warnock:

"He was really a very unhappy man," Warnock said quietly. "It worried him that he hadn't written much. One lecture, "Ifs and Cans"...became famous, but it is mainly a negative work, and he published very few articles and, significantly, not a single book...To add to his writing block, he had a fear of microphones, and this prevented him from broadcasting...this was another source of unhappiness. He took enormous pride in teaching, but this began to peter out in his last years, when he felt that he had reached the summit of his influence at Oxford. Toward the end of his life, therefore, he decided to pack up and go permanently to the University of California in Berkeley...But before he could get away from Oxford, he died. (p 62)

 

Why did this strike me? There are a few reasons. Austin had a major influence on the development of ordinary language philosophy, post-WWII. He crops up everywhere. Any book you read about the period singles him out as its most influential figure. Not all the portraits are kind. Some suggest he was a tyrant and bully. But he was undeniably a piquant and influential figure who, as we will see below, played a crucial role in military intelligence during WWII, possibly critical to ensuring the defeat of Hitler. And yet here we have one of his closest allies and friends remarking on how unhappy he was.

Was this just the mistaken impression of a friend? Was it really true? I suspect it is more true than false. It resonates with my own experience of academia. With relatively few exceptions, many of my academic colleagues strike me as being unhappy people, at least when it comes to their professional lives. They seem overwhelmed with disappointment, frustrated by lack of motivation or foiled ambition, always creating the impression that they should have done (or should be doing) more. I don't exclude myself from this diagnosis either. I wouldn't say that I am unhappy, per se, but I certainly feel frustrated and unfulfilled more often than I would like. In my darker moments, I'm haunted by the image of Einstein furiously scribbling equations on his deathbed for his (never-to-be-completed) grand unifying theory. It makes me wonder whether academia is a bit like politics: do all academic careers end in failure (or at least the sense of failure)?

Anyway, back to Mehta's book. The second half of the book is not about philosophy. It is about history and historical method. Should historians try to discover the mechanics of civilisation (as Arnold Toynbee hoped to do) or should their aims be more modest? I found this half of the book more interesting than the first half, largely because the material was less familiar, but to explain why would take me away from the themes of this review.

Overall, enjoyable as it was at times, Mehta's book didn't quite work for me. There is not enough 'colour' to make his portraits interesting. While it is nice to know that Richard Hare used to write his philosophy in a caravan outside the front of his house, you would expect more details like this from a New Yorker-style piece. There were a few too many, unexplained, extended quotes from the figures themselves. It's like a book that doesn't know where it belongs: philosophical exposition, collection of interviews, investigative journalism or colour reporting?

3. The Women are Up to Something (by Benjamin Lipscomb) and 4. Metaphysical Animals (by Clare Mac Cumhaill and Rachael Wiseman)

Sometimes the publishing world gets wind of something and decides to capitalise on it. This often results in a flood of books hitting the market at the same time about the same topic. While this is, perhaps, understandable when it comes to contemporary and popular affairs, it is more surprising when the topic in question is four dead female philosophers and their experiences at Oxford in the mid-20th century. Nevertheless, that's exactly what happened with these two books, published in quick succession, by two different sets of authors. Both books are about the lives and philosophies of Elizabeth Anscombe Philippa Foot, Mary Midgely and Iris Murdoch. The Somerville Quartet, as one reviewer described them.

Each of these women is well known in their own right. Anscombe was Wittgenstein's literary executor and translated his final work (Philosophical Investigations) to English. She also published ground-breaking and highly influential work of her own, particularly in the philosophy of action. Philippa Foot was one of the leading figures in moral philosophy in the latter part of the 20th century. We have her to thank (or blame?) for the modern obsession with the trolley problem. Iris Murdoch was an influential writer and, to a lesser extent, moral philosopher. She is probably best known for her complex, psychological novels. Mary Midgley is a slightly more eclectic figure. She lived to be nearly a hundred and became influential later in life as a critic of modern evolutionary perspectives on human behaviour and morality, particularly as practiced and preached by the likes of Richard Dawkins.

Before reading these two books, I was most familiar with the work of Philippa Foot, and, to a lesser extent, Iris Murdoch (I've struggled through several of her novels). I had read some of Mary Midgley's papers critiquing Dawkins, but never thought much of them (they seemed, to me, to rest on a misreading and misunderstanding of his position). I knew Anscombe's work only by reputation and found some of its terminology intimidatingly obscurantist.

Both of these books take as their foundational conceit an observation once made by Mary Midgley to the effect that the advent of World War II, and the fact that the young men who would otherwise have dominated the philosophy seminars of Oxford were called up to serve, had a liberating effect on the women who were left behind:

The effect was to make it a great deal easier for a woman to be heard in discussion than it is in normal times. Sheer loudness of voice has a lot to do with the difficulty, but there is also a temperamental difference about confidence--about the amount of work that one thinks is needed to make one's opinion worth hearing. 

(Mary Midgley, quoted in Lipscomb, p 39)

 

Both books are well written and very readable. I can recommend either (or both) to anyone interested in the four women. They share a similar thesis: that the four women developed their philosophical positions largely in opposition to the dominant trend in ordinary language philosophy. In particular, the books claim that all four thought that philosophy should be less about solving technical problems in linguistic expression and more about addressing real human problems, particularly in the land of morals and values. Lipscomb runs with this thesis the most, arguing that the women were all opposed to something he calls the 'Dawkins Sublime' (essentially, a reductionistic and scientific view of mankind). MacCumhaill and Wiseman are bit more circumspect, suggesting that the women were more interested in the idea of mystery and returning philosophy to metaphysics.

Jennifer Frey has written what, to my mind, seems like a plausible critique of both books, arguing that the picture of opposition that they paint is misleading. For instance, Anscombe, given her obvious links to Wittgenstein, was clearly less outside the Oxford norm than one might be inclined to think from reading these books. And Philippa Foot's work was largely a defence of a scientific and naturalistic view of morality, not an alternative to it. Overall, according to Frey, there is more disunity to what the four women had to offer than the books suggest. Based on what I have read of them, this sounds right to me, though I am not an expert on any of them. Even if Frey is right, there would still be a reason to write a book about all four of them. They did go to Oxford at the same time, they were friends and they did develop interesting philosophical positions of their own. Indeed, Frey suggests that what binds the women together, and what should be celebrated, is their intellectual friendship, not their common ideology. That's something to celebrate in its own right

Is one of the two books better than the other? I wanted to like MacCumhaill and Wiseman's more: it seemed appropriate to me that a book aimed at reinvigorating the inquiry into four female philosophers should be written by two women. But, on balance, I marginally preferred Lipscomb's. This was because it covered more of their lives and philosophical views (MacCumhaill and Wiseman rush through the final years). Also, MacCumhaill and Wiseman's book had some slightly (to me) grating historical speculation in it. They would frequently imagine occasions when the women may have met up or spoken to one another about a particular topic. For example, there are several scenes that start with phrases like "we might imagine" or "perhaps" X or Y happened. Obviously this helps with storytelling, and perhaps (look! I'm doing it too) all historians invent narrative details, but by the end of the book I found it jarring.

5. JL Austin: Philosopher and D-Day Intelligence Officer, by Mark W Rowe

The last book in the sequence is the most recently published (May 2023) and, as a consequence, the one that is freshest in my memory. It is an extended biography of the aforementioned, and deeply unhappy, JL Austin. As already noted, Austin was, along with Gilbert Ryle, the centre of the ordinary language movement in Oxford philosophy. Although he published little in his lifetime, through his Saturday morning seminars and extensive teaching, he exerted a significant influence over the method and style of mid-20th century Oxford philosophy. Every book about the era makes reference to this fact. And yet, until Rowe, there has been no book-length biography of him.

I went into this book with low expectations. Austin is not a philosopher that speaks to me. I think aspects of speech act theory are interesting, and I have made use of them in some of my own work on legal interpretation, but I never really took that directly from Austin, whose preferred vocabulary ('illocutionary acts', 'perlocutionary effects' and the like) always put me off. Also, none of the portraits of him in other works make him sound like an appealing figure. Consequently, this book was something of a revelation, not because it made me warm to Austin (it didn't and Rowe doesn't shy away from his flaws), but because I just really enjoyed reading it.

It is a classic 'fat' biography, full of historical detail and clearly the work of tremendous, painstaking research. If you want to know about every philosophical paper, debate, class, or seminar series Austin participated in, then this is the book for you. That might make it sound boring, but it is really not. I could have, perhaps, done without the extensive family tree and prehistory in the opening chapter, but beyond that I learned more from this book about the style, personalities and views of Oxford philosophy in the early to mid 20th century than from all the other books reviewed above.

Also, at the heart of the book, is a fascinating inquiry into what Austin did during WWII. It has long been known that Austin played a key role in British intelligence during WWII. Again, if you read histories of ordinary language philosophy, you will constantly come across allusions to Austin's 'glittering' war career and how he returned to Oxford after ascending the heights of the British military. But what exactly did he do? Austin never shared the details during his life. To work it out, you have to piece together fragmentary accounts and indirect evidence from numerous sources. That would take a long time. Fortunately, Rowe has done all this work for us and he shares his results in the middle portion of the book.

This was grist to my mill. I am not a WWII buff, by any stretch of the imagination, but I loved all the detail about the progression of the war, the intelligence operations, the brave work of the French resistance, the military strategy and planning, and the internal politics of the military bureaucracy. In very short outline, Austin rose to fame within the intelligence services for some early work on the North African Campaign (mainly in 1941). He was one of relatively few people to guess, based on the evidence, that German forces in North Africa were stronger than others were claiming. Then, from 1942 onwards, Austin led a unit within the intelligence service (nicknamed the Martians) that was instrumental in planning the D-Day landings. This required a lot of patient and piecemeal work, figuring out appropriate landing sites and the distribution of German defensive fortifications. It might sound dull, but it was detailed, high pressure work, and it was important to the success of the Normandy invasion.

One thing that emerges from this section of the book, of course, is that Austin was not some lone genius in the military complex, single-handedly responsible for the success of the D-Day landings. It was the coordinated work of many individuals that made the crucial difference. This is something you get from all good histories of WWII and is, I think, one of the positives to take from the war: when our backs are to the wall, we humans can cooperate on a large scale to achieve a desired end.

There is also plenty of philosophy in the book. Rowe takes the time to explain and critique a lot of Austin's post-war philosophical work. The end result is not a light and breezy read, but it is a rewarding one.

And that's it. Five books. Reviewed.

Subscribe to the newsletter

Generative AI Entails a Credit-Blame Asymmetry

Tweet

That's the title of one of my new papers, co-authored with multiple others, just published in Nature Machine Intelligence. The full paper is behind a paywall, sadly, but you can access a read-only version here. If anyone would like a PDF copy, just let me know via email and I will happily share one.

The core thesis of the paper is summed up in this quote:

Traditional theories of blame, reflected in many legal standards, suggest that if we are reckless or negligent with respect to bringing about a negative outcome, even if we did not intend to do so, we can still be held responsible for it. By contrast, to deserve credit for a positive outcome, we must exert some effort, or display some form of talent, or make some sacrifice to bring it about. 

These differences lead to what we term a credit–blame asymmetry: the use of generative AI elevates the bar for earning credit, but standards for assigning blame remain the same. 

Applied to LLMs, this asymmetry suggests that society might be justified in holding persons accountable for deliberate or careless errors in generated text if they put such text to use in ways that negatively impact others, even if they did not put much skill and effort into generating that text. But we might not think people deserve credit for text generated without much skill and effort..

 

The paper goes on to discuss the ramifications of this asymmetry in multiple domains, with a particular focus on responsibility for publications.

Subscribe to the newsletter

Artificial General Intelligence and the Problem of Cognitive Inflation

Tweet

The development of artificial general intelligence (AGI), and, indeed, precursors to AGI (things that 'spark' of AGI), could give rise to cognitive inflation. Cognitive inflation is analogous to monetary inflation. In the case of monetary inflation, the value of money (its purchasing power) goes down, oftentimes due to an oversupply of money. Central banks attempt to control monetary inflation by controlling the supply of money, which can benefit certain groups and institutions more than others. In the case of cognitive inflation, the value of (human) intelligence goes down, partly due to the oversupply of intelligence. This can have profound implications for the functioning of the economy and, more importantly, for social equality. At the moment, a handful of key companies and institutions have the power to control the supply of intelligence. Just as monetary inflation often benefits certain groups and institutions over others; so too will cognitive inflation, and the control over the supply of cognitive power, benefit some people at the expense of others.

In the remainder of this article, I will elaborate on this idea. I will first give a quick overview of monetary inflation. I will discuss what it is, why it is important, and how it can be controlled. I will then draw out the analogy between monetary inflation and cognitive inflation, again, explaining what it is, why it is important and how it can be controlled. I will conclude by addressing some objections to the proposed analogy.

Before I get started, let me confess that this is a somewhat half-baked idea. It occurred to me on the flight home from a conference -- stimulated by a talk I heard by Atoosa Kasirzadeh -- though it goes without saying she bears no blame for what I am about to say. I'm not sure if the idea is valuable. It just struck me as interesting and worth sharing. Please let me know what you think. Is the analogy between monetary inflation and cognitive inflation a productive one? Does it give us some additional insight into the future trajectory of AI development and its impact on society? Or are we better off sticking with more traditional categories and concepts?

1. A Crash Course in Monetary Inflation

For a period of about 6 years, for some unknown reason, I used to teach a course on banking and finance. It's not my area of expertise, though I do have an interest in it. As a result of teaching this course, I acquired some knowledge of how money is made, how it is circulated in the economy and, of course, the problem of inflation. What follows are some of the key ideas I used to share with students. If you want to skip the gory details, jump ahead to the bullet points at the end of this section.

First, it's worth being clear about what money actually does. The standard economic definition of money is functional not ontological, i.e. it focuses not on what money is but, rather, on what it does. Typically, textbooks will say that money performs at least three functions in the economy: (i) it is a medium of exchange (you can use to pay for things and you in turn can be paid in it); (ii) it is a unit of account (it records debts and measures the value of goods and services) and (iii) it is a store of value (it doesn't radically fluctuate in value over time). It may also perform other functions. Some more sophisticated books, for example, claim there are more than four functions. But there is no need to get into that complexity here.

Second, it is worth briefly commenting on some of the disputes about the ontology and history of money. This is a topic that exercises a great many people, probably because there is something ontologically weird about money. Does money have to be linked to a material artifact/resource (gold, silver)? Can it just be created out of nothing? Is it, in essence, a collective hallucination? I'm of the 'collective hallucination' school of thought. If pushed, I would say that money is a collectively agreed upon system of tokens. These tokens perform the three functions mentioned in the previous paragraph (possibly more). In principle, these tokens can be anything -- gold bars, sea shells, large stone discs, balances stored on a cryptographic ledger -- but whatever token chosen in a given society will only perform the functions of money if there is sufficient trust and faith in the system. Historically, money was often linked to physical artifacts (gold, precious metals etc) and this created natural limitations on its supply: you had to find more of those physical artifacts to create money (or, alternatively, change the exchange rate between those physical artifacts and paper currency). This sustained trust and faith in the system because it was hard to fake money or depreciate its value by creating more, though there were always efforts to undermine this through forgery and seigniorage. Nowadays, most countries use fiat money systems, whereby money is created from nothing by governments, usually operating through central banks. Sometimes the money will take the form of physical currency (notes and coins) but the majority of it is purely abstract in nature - balances recorded in digital ledgers. This is an important feature of the modern monetary system and one to which I will return in a moment. It is also worth noting that ordinary banks also have the power to create money -- or at least purchasing power -- through the system of fractional reserve lending. They can lend out more money than they actually have on deposit, which means they can create multiples of the actual sums of money they have. Technically, the money that ordinary banks create through lending should disappear when loans are repaid (although, of course, it is unlikely that all loans are ever fully repaid).

This is where the problem of inflation enters the fray. Inflation is a measured increase in the price of goods and services over a period of time (usually a year). Most countries measure inflation by reference to a consumer price index or CPI (a representative basket of goods and services whose prices are recorded over time). If the average price of goods and services in the CPI goes up over time, then we have inflation. If it goes down, we have deflation.

Although inflation is normally understood as an increase in prices -- and this is how most people perceive its effects in their day to day lives -- it can also be understood as a decrease in the purchasing power of money. In other words, if we have inflation this indicates that the value of money is decreasing over time. This is the way I prefer to understand inflation even though I know this can create confusion (since "inflation" connotes an increase in something, not a decrease).

Both inflation and deflation can have negative effects on an economy. Deflation usually signifies that people are saving too much, that there is not enough money circulating, and consequently not enough ongoing economic activity. This can be a cause or effect of recession. Indeed, post-2008, most European economies underwent a sustained period of deflation correlating with stagnant or minimal economic growth. Inflation usually signifies that there is too much money being spent and/or not enough goods or services being created to meet the demand. At sufficiently high levels, inflation can cause an economy to grind to halt. The most dramatic illustrations of this are during episodes of hyperinflation, such as those that occurred in Weimar Germany in the early 1920s and Zimbabwe in the 2000s. It's worth reading up on the details of those hyperinflationary episodes just to see how disruptive and destabilising they were. That said, hyperinflation is an extreme case, caused by a total breakdown in the trust and faith needed to sustain the monetary system. The modern view is that a minimal level of inflation is a good thing insofar as it can stimulate economic growth. This is why many central banks have a target rate of inflation. For instance, the European Central bank tries to achieve a 2% inflation rate across the Eurozone economies.

Inflation can have weird, and slightly counterintuitive effects. I want to mention one here because it is important for the analogy I want to draw with cognitive inflation. Inflation can involve a redistribution or transfer of wealth. Extended periods of deflation often strike individual consumers as a good thing. To them, it means that the purchasing power of their money is going up, which, in a sense, means that they are getting wealthier (without having to earn additional money for this). For example, house prices, post 2008, underwent significant deflation in Ireland. This was a good thing for some people because it meant they could buy houses for less money. Conversely, from the perspective of most individuals, inflation looks like a bad thing, even at small levels, because it results in a loss of wealth: the purchasing power goes down. This is one reason why fans of older monetary systems - such as the gold standard - critique fiat money systems. They argue that the inherently (or intentionally) inflationary nature of fiat money systems means that they always strip ordinary people of wealth. This is one reason why Bitcoin was designed to be a radically deflationary currency - as long as the system sustains a basic level of trust and confidence (which hasn't always been the case), people that hold onto Bitcoin can expect their wealth to increase. Who gains from inflation? If it gets out of hand, then I would argue that no one really benefits, because it leads to social and economic breakdown, but at certain points in inflationary cycles some people do benefit. In particular those that control the supply of money or get early access to increased flows of money can benefit. They can reinvest the money before the value decreases rapidly, and pass it onto the 'suckers'.

There are many causes of inflation. If an economy undergoes a significant shock and the supply of goods and services goes down (as arguably happened during Covid 19), then you may end up with a situation in which there is more money available than there are goods and services. This can cause inflation. Another thing that influences rates of inflation -- and has been implicated in most hyperinflationary cycles -- is the supply of money. As noted above, we now live in fiat money systems. In these systems, governments - usually through central banks - have power over the money supply. If they abuse this power it can cause inflation. Technically, there is no upper limit on the amount of money they can create (because it is not linked to some limited physical artifact). If they create more and more of it (often referred to as 'printing' money due to the history of using paper currency), they may well cause an inflationary or hyperinflationary cycle: because more money is sloshing around in the economy, prices are bid up, resulting in inflation. This means they have to be careful when it comes to deciding how much money is created. They need to maintain collective belief in the monetary system so that people don't think money is rapidly decreasing or increasing in value. How do they do this?

Modern central banks typically do two things to control the supply of money: (i) they create money (ex nihilo - out of nothing) by purchasing government debt or other financial assets or (ii) they set the baseline interest rate charged to banks within a monetary zone.

The first of these mechanisms is the one most people have trouble understanding. Intuitively, people think that you have to have money to spend it. This is what everyday life teaches us. But the rules of everyday life do not apply to central banks. They can create money without having it. And they do this on a regular basis. This is often obscured by the unusual and euphemistic terminology they use to explain what they are doing. When you hear about central banks engaging in 'quantitative easing' or 'asset purchase programs' -- as they did, with gusto, post 2008 and during the Covid-19 crisis -- you may have no clear sense of what they were up to. What they were up to was creating large sums of money from nothing and pumping into the economy. Where did this money end up? Well, that's the interesting bit. In principle, I suppose that central banks could just give the money they create equally, to all ordinary people (everyone gets an extra 1000 euro in their account, for example). In practice, this is not what they do. Instead, they purchase debt instruments - government bonds most often - from banks and other financial institutions. These banks and financial institutions then get to control how the money that is created gets distributed through society. They have the initial and preferential access to the increased supply of money. Perhaps the hope among central bankers is that banks will lend the money out to productive economic activity, and this will stimulate economic growth in the right way, but that's not always what happens. Indeed, over the past decade and a half, a large percentage of the money that was created has ended up being invested in property and land, oftentimes with deleterious consequences for the price of housing in a range of countries.

The second mechanism is a bit more straightforward and involves setting the price of money (the price attached to loans) in a monetary zone. This affects the flow of money through the monetary zone by affecting the behaviour of banks and borrowers. A low interest rate, in principle, entices people to borrow more money (and banks to lend more) thereby increasing the volume of money circulating; a high interest rate, in principle, has the opposite effect. It's basic supply-demand thinking, although reality doesn't always match theory.

I appreciate some of this may be unfamiliar and complicated to readers. I have tried my best to make it simple. For the purposes of exploring the analogy with AGI and cognitive inflation, there are just a few key points I want people to take away:

• Inflation involves a decrease in the purchasing power of money (the value of money) and, in a sense, a redistribution of wealth away from a person that owns or holds onto money.

• Inflation is often closely linked to the supply and flow of money through an economy. The more money being created, and the more it flows through the economy, the more inflation you are likely to have.

• Traditionally, there were some (imperfect) natural controls on the supply of money. Money was often physically embodied in rare or precious metals (or some other scarce resource) and you had to have more of those metals to increase the supply of money. In fiat money systems, those natural controls are absent. In principle, money can be created at whim.

• In fiat money systems, governments, usually through central banks, have ultimate control over the supply of money. They can create more, whenever they like, and they can speed up or slow down its flow through the economy by setting baseline interest rates (the price of borrowing money)

• In contemporary fiat money systems, money doesn't flow equally in all directions at once. Banks and financial institutions have preferential access to the increase in money supply and have power - subject to regulation - over how it gets redistributed through society.

These are the points I will draw upon in what follows.

2. The Cognitive Inflation Analogy

There are a number of striking parallels between monetary inflation and cognitive inflation. These arise, in the first instance, from an analogy between money and intelligence and from the new reality of artificial intelligence, particularly more generalised forms of it. Let's consider these parallels in sequence.

First, like money, intelligence is a basic store of value and medium of exchange in the human economy (it's probably not a unit of account but no analogy is perfect). Money is a general purpose token. If I have 100 dollars I can use it to buy 100 dollars worth of any good or service. Money is fungible: one unit is the same as any other unit. I can deploy it to a number of ends. Intelligence is a bit like this. Intelligence is a general purpose tool, used, for the most part, by humans to produce goods and services across the full spectrum of the economy. General intelligence is, in principle, fungible: the same basic phenomenon or capacity can be used to pursue any economic ends. All economic activity depends on intelligence, even physical labour or 'low-skilled' labour depends on intelligence, broadly defined. Cutting someone's hair, for example, requires a high level of manual-physical intelligence. Analysing financial data and recommending arbitrage transactions requires a high-level of analytical intelligence. Intelligence is the medium that human workers use to sell themselves to employers. Without intelligence, we are nothing but physical resources.

Second, despite what I just said, there are some obvious disanalogies between intelligence and money, but these disanalogies pertain to the historical norm not the emerging reality. Historically, intelligence was intrinsically linked to and limited by the human body/brain. You couldn't just create intelligence from nothing, whenever you liked. You had to create more humans to get more intelligence. At the margins, changing educational systems could make a difference to the overall level of intelligence (roughly: good quality universal education = more intelligence). But educational systems are often imperfect and don't harness or spot talent in productive. There wasn't the same level of control over the creation and flow of intelligence through the economy as there is with money.

What's more, intelligence was highly differentially distributed. Though it is correct to say that humans have 'general' intelligence, they do not all have the same level of intelligence across all domains. Some are better at maths; some are better at languages; and so on. This may be due, in part, to genetics and in part to environment. As a result, even if people all have same innate level of intelligence or ability, they do not have equal access to opportunities to hone and develop their intelligence. This means that human intelligence is not really fungible or easily transferable across economic domains.

But this is now changing. With the creation of artificial intelligence, particularly generalised systems of artificial intelligence, we have the opportunity to create a fungible, transferable and controllable supply of intelligence. It is like making the transition from the gold standard to fiat currency. Companies that create AGI and control the infrastructure through it is supplied to the broader economy, now function like central banks -- central banks of intelligence. They can speed up or slow down the rate at which intelligence flows through the economy. They can create new units of intelligence whenever they see fit. Consider, as the most prominent example, OpenAI and GPT4. Through its ChatGPT interface (in particular) they have just flooded the market with additional intelligence. What impact will this have?

One potential effect of this will be to trigger cycles of cognitive inflation. In ordinary inflationary cycles, the value/purchasing power of money goes down. In cognitive inflationary cycles, the value of intelligence goes down. The primary impact of this will be on humans, who can no longer trade their intelligence for as much as they used to be able to do. Intelligence will become cheap and abundant. Humans will have to invest more and more in their own education and performance to keep pace. Unless and until there is a major readjustment in the economy, there will be more intelligence chasing the same set of opportunities. This could (will!) have profound implications for how the economy operates, most immediately in terms of job loss and wage reductions.

What's more, similar to monetary inflation, the effects of cognitive inflation will not be equally felt. For the majority, the effect will be a loss of wealth. What you have to offer (your intelligence) is no longer worth as much and you will need more of it to achieve the same ends. But those with the power to create intelligence at whim, and those with early and preferential access to AGI are likely to gain, at least in the short term. In some respects the situation seems better (or more hopeful) with respect to AGI than it is with money. For historical and practical reasons, money is created by and flows through banks and financial institutions. They get to control who accesses it. Artificial intelligence is created by and flows through tech companies. They also get to control who accesses it but many of them are used to providing it to end users for free (as in GPT3.5) or at a relatively low cost. We might then suppose that the initial benefits of an increased supply of intelligence will be more widely shared than an increased supply of money. But I am not so sure about this. Companies and governments can also put up barriers to accessing artificial intelligence. They can create hefty subscription fees or high cost bespoke models for certain institutions (lawyers/financial analysts); they can limit access for regulatory reasons. Relatedly, some people are better able to exploit the new technology because they are more aware of how it works and how it can be leveraged to productive ends. It could well be a case of the cognitively rich getting richer.

In summary, there are, to my mind, a number of compelling analogies between monetary inflation and cognitive inflation:

• Intelligence, at least in its generalised forms, is a universalisable and general purpose tool/token that serves as a medium of exchange and store of value across all economic activities.

• Although historically the supply of intelligence was intrinsically linked to and limited by the supply and education of humans, with the advent of artificial intelligence this is no longer true.

• With artificial general intelligence, the creation and supply of intelligence into the economy is controlled by a number of key tech companies. They can switch the tap/faucet of intelligence on and off at whim. They are like central banks of intelligence.

• If the economy becomes suddenly flooded with lots of newly created intelligence, this is likely to lead to cognitive inflation, i.e. a situation in which the value of intelligence declines over time. There could even be extreme or hyper-cognitive inflation if there are sudden breakthroughs in AGI.

• Not everyone will benefit equally from cognitive inflation. For most people, the main long-term effect will be a loss of wealth, specifically in the value of their own individual intelligence. That said, those that control the supply of intelligence or get preferential early access to it, may benefit, at least in the short term.

	Monetary Inflation	Cognitive Inflation
Definition	Decrease in the value/purchasing power of money	Decrease in the value/exchange price of intelligence.
Causes	Multifactorial but often linked to an increase in the supply of and speed at which money flows through the economy.	An increase in the supply of and speed at which intelligence flows through the economy.
Effects	Loss of individual wealth (though some may gain, at least in the short term); economic slowdown/stagnation; at extremes, complete grinding to a halt of the economy	Loss of individual wealth (though some may gain, at least in the short term); if there is a significant and sudden increase in the supply of intelligence, there could be demand collapse, and potentially economic collapse.
Solution	Reduce the supply of or speed at which money flows through the economy.	Reduce the supply of and/or speed at which intelligence flows through the economy.

3. But Surely You are Joking?

I'll be the first to admit that the analogy I've proposed is imperfect. No analogy is ever exact. If it were, we would have identity, not analogy. Nevertheless, I think there is something compelling about it, even though I can envisage several potential objections.

An obvious first objection would be that money and intelligence are not alike. Money is an operating system for the economy - something you need to price goods and services, pay for them, store wealth and incentivise action. Intelligence doesn't seem to be like that. It is more like an asset or input into the economy. The more appropriate analogy is with something like oil or energy, not money.

I understand this objection.   Setting aside the fact that some people think money is best understood as an asset (e.g. Eric Lonergan), I do think intelligence is a bit like energy. It's something that is used across all economic activity. It is something we invest in and deploy to different productive ends. But I also think there are several ways in which intelligence is not like energy, particularly if it becomes scalable and producible at whim. Energy is a basic input into economic activity but it doesn't have the same diversity and flexibility of application as intelligence. Indeed I would argue that the productive use of energy is itself dependent on the productive use of intelligence. So intelligence is, to my mind, a more fungible and flexible input. Furthermore, energy is still locked into particular deposits and resources. We have to find and exploit those resources to harness energy to productive ends. At times, energy can seem abundant and cheap, but at other times it becomes expensive and scarce. We cannot simply create more energy at will, whenever we like. With artificial intelligence, we can create intelligence at will.

Of course, the infrastructure underlying artificial intelligence is dependent on energy and there may be scaling limitations to how much intelligence we can create, given our current energy resources. But this is true of the infrastructure underlying money too. In fiat money systems, we depend on extensive computerised accounting systems to keep track of the creation and flow of money. This requires lots of energy. So intelligence and money are not disanalogous in this respect.

Another objection to the proposed analogy is that the effects of an oversupply of intelligence are not the same as an oversupply of money. If there is too much money in the economy, prices get bid up, which robs people of wealth and, in extreme cases, can lead to a complete breakdown of trust and faith in the monetary system, causing economic activity to grind to a halt (or, at least, switch to a less efficient, non-monetary, barter-like form). An oversupply of intelligence is unlikely to lead to something similar. It's main effect will be to drive down the price of human intelligence, which will have negative effects for many workers, at least in the short term, but can also have positive effects. It can result in cheaper and more efficient goods and services. This could make people better off, not worse off. It could also result in the creation of more opportunities in the future, as the economy adjusts to accommodate the new powers of cheap and abundant intelligence. To put it another way, it is accepted that there is some socially optimal level of money for an economy (usually assessed by reference to the rate of inflation); it is not clear that there is some socially optimal supply of intelligence. In principle, there is no shortage of problems or purposes to which intelligence can be put.

Here, I recall the argument of Julian Simon in his controversial book The Ultimate Resource. The book was primarily a rebuttal to those that argued we should curb population growth. Contrary to this, Simon argued that more people was actually a good thing because it could lead to more innovation and economic growth, which could benefit a larger population. Why did Simon think this? Because increasing the supply of people increased the supply of intelligence. The argument has been made more explicitly by Ramez Naam in his book The Infinite Resource (the title of which is an homage to Simon). The same reasoning, I would suggest, applies to the increased supply of artificial intelligence. It can be used to fuel innovation and growth. It won't, necessarily, cause a breakdown in economic activity. After all, intelligence, unlike money, is not dependent on collective trust and faith for its functionality. A massive increase in the supply of intelligence may reduce the price of human labour, but it won't denude intelligence itself of its functionality. Intelligence has an intrinsic value in solving problems and producing valuable outputs; money, by contrast, has a purely representative or signal value. You can completely override the signal value through oversupply; you cannot override the intrinsic value of intelligence through oversupply. In the long run, we may all benefit from this.

Again, I can sympathise with this objection, but I have two responses to it. First, I think that a sudden and rapid increase in the supply of intelligence, could lead to a significant short-to-medium term 'adjustment shock' in the economy. There could be a rapid shift away from hiring intelligent humans and a refocusing of attention on how to best deploy artificial intelligence. This will be motivated, initially, by the belief that this can lead to more efficient and profitable activity, but, at a sufficient scale, it could lead to a breakdown of economic activity. This is a long-hypothesised concern about the rapid, scalable automation of labour. The capitalist economy depends on consumer demand. Consumers need money to make good on their demands. If a large segment of them are rendered unemployable as a result of an oversupply of general intelligence, or have their wages significantly cut as a result, this could lead to a collapse in demand. Unless there is some radical shift in how wealth gets redistributed, this could cause the economy to grind to a halt. We already see inklings of this happening. In the week that I write this article, IBM announced that it will stop hiring new people in roles likely to be replaced by AI. If that logic takes root across a broad swathe of the economy -- as is likely if we can create AGI at whim -- you could see effects very similar to those we see in inflation or hyperinflationary cycles. Consequently, I think there is, in some sense, a socially optimal supply of intelligence: we need a supply of intelligence that matches or perhaps slightly exceeds our current capacity to put it to good use. An oversupply creates a glut of intelligence that adversely impacts existing jobs, payment structures and wealth distribution, without creating additional wealth and opportunity. In the long run, the economy may well adjust to the new reality, and we could benefit from increased productivity and innovation, but, as Keynes once said, in the long run we are all dead.

Second, I think there is a sense in which intelligence, like money, depends on collective trust and faith for its functionality. The value of intelligence is not purely intrinsic; it is also representative. The value we place in intelligence is dependent on our belief that it can be deployed effectively, and that it can provide us with the truth, with theoretical or analytical insight, evaluative wisdom and so on. One of the primary ways in which we verify that intelligence can do its job is by scrutinising the information produced and shared by intelligent entities. One of the problems with AI is that it can be used to produce massive volumes of misinformation and disinformation, either intentionally, through malicious human action or oversight, or unintentionally due to inherent limitations in its design (e.g. the propensity for LLMs to hallucinate information). If we flood the economy with information of potentially dubious quality, then we could see a breakdown in the collective trust and faith required for intelligence to sustain its value.

There are some caveats to this. Obviously, human intelligence isn't always trustworthy either. Humans have significant cognitive limitations and they mislead and deceive one another for a variety of reasons. Nevertheless, we have some institutions and practices in place for verifying and checking human intelligence. Hopefully we will have similar institutions and practices in place for AI but, as has long been noted by critics and commentators, we have struggled to create such institutions in recent times. This has been true for human-produced misinformation on the internet; and is likely to be true for AI-produced misinformation too. Indeed, the problem is likely to be exacerbated by AI due to the fact that it is not like human intelligence and operates at a different scale and speed (we already see some evidence for this).

In addition to this, money and intelligence are unlike in the following crucial respect: Money is entirely dependent on collective trust and faith: without it, it cannot perform its functions. Intelligence is only partially dependent on collective trust and faith. There are some disciplines and activities in which there are clear objective standards of success and failure (e.g. building a stable bridge). It is easy to tell whether artificial intelligence meets those standards and, consequently, easy to sustain collective trust and faith in its functionality. Other disciplines and activities have fuzzier or more nebulous standards of success. They are more likely to be prone to the disinformation effect outlined above. They are the areas in which you are likely to see the worst effects of cognitive inflation.

4. Conclusion

This brings me to the end of this article. To quickly recap, I have proposed an analogy between monetary inflation and cognitive inflation. Monetary inflation involves a reduction in the value of money, which can be deleterious to economic activity. Monetary inflation is often linked to the oversupply of money, and so controlling the supply of money into the economy is the main method of controlling inflation. Historically, control over the money supply was limited due to the fact that money was dependent on scarce physical resources like gold and other precious metals. In the modern era of fiat money, there are no intrinsic limits on the supply of money. It can be created at whim. This accentuates the challenge of inflation, but gives greater flexibility and control over it too.

Cognitive inflation involves a reduction in the value of intelligence, which can be deleterious to economic activity. Cognitive inflation can be linked to the oversupply of general intelligence. Historically, the supply of intelligence was limited due to the fact that it was dependent on humans. We could increase it, at the margins, by increasing the population or making adjustments to how they were educated, but this gave only limited control over the supply. This meant that the problem of cognitive inflation was never particularly acute or solvable. In the era of artificial intelligence, particularly in its generalised form, this changes. We now have the power to create intelligence at will. This accentuates the challenge of cognitive inflation, but gives greater flexibility and control over it too.

The analogy is imperfect. No doubt about that, but I think it is a interesting and insightful way of thinking about the dawning new reality.

108 - Miles Brundage (Head of Policy Research at Open AI) on the speed of AI development and the risks and opportunities of GPT

Tweet

[UPDATED WITH CORRECT EPISODE LINK]

In this episode I chat to Miles Brundage. Miles leads the policy research team at Open AI. Unsurprisingly, we talk a lot about GPT and generative AI. Our conservation covers the risks that arise from their use, their speed of development, how they should be regulated, the harms they may cause and the opportunities they create. We also talk a bit about what it is like working at OpenAI and why Miles made the transition from academia to industry (sort of). Lots of useful insight in this episode from someone at the coalface of AI development.

You can download the episode here or listen below. You can also subscribe the podcast on Apple, Spotify, Google, Amazon or whatever your preferred service might be.

Subscribe to the newsletter