The Alignment Problem (2020) is an excellent resource for anyone seeking a clear and informed view of the limitations, as well as the opportunities, of current AI tools. Having read Christian’s last, co-authored, book, Algorithms to Live By (2016), I was keen to read more from this remarkable writer. Christian makes AI intelligible, but moreover, he makes it fun. He writes from the inside of AI teams: you feel he has been there as the discoveries take place, but he remains dispassionate enough to include the references, and is not swept away by the advances. I cannot but admire the thoroughness of the research: there are over 800 citations, and over 65 pages of notes. This is one of those books where the notes carry out a little conversation with you, alongside the main argument. You don’t want to miss the notes, since they provide not only links to the primary sources, but as well a lot of interesting details, often from talking to the researchers.
Christian’s great achievement is to capture the fun of primary research. He turns scholarly papers into excitement, as he describes the stories and people behind the papers. Christian carried out hundreds of interviews he had with researchers. This is one of the reasons why the research comes to life. He himself quotes from one of the researchers he interviewed, who describes the magic of meeting an author of a seminal paper. Michael Littman, for example, describes how, fresh from his studies, starts a new job and expresses an interest in reinforcement learning, and his colleagues suggest that he invites the author of a seminal paper to address his grouop:
And I was like, “What? That’s a thing you can do? You can read a paper and there’s a person’s name on it, and then you can turn them into a human being? … I didn’t think of it as a community; I thought of it as literature. But no, it was a bunch of people, and they knew each other…” [ch 5, p171]
Christian assesses AI in three main areas: prediction (which he terms prophecy), the fundamental capability of current AI tools; agency, including the need for some algorithms such as self-driving cars to never make a mistake, as well as the limitations of reinforcement learning and rewards; and “normativity”, which includes the crucial “inference”: how the AI model may (or may not) infer what we really want it to do, when we humans may be unclear in our goals, or present the machine with an overly simple model to follow.
Christian makes clear, with example after example, the limitations of using only machine learning without any examination of context. That context might be the corpus, or the assumptions that the users bring with them. The book is full of warning signs about the unthinking use of AI tools.
- Word embeddings, for example, can be manipulated arithmetically but reveal (and may perpetuate) bias. [p46]
- “we should be sceptical when the system states is it “99% accurate” [p33]
- “Machine learning is not, by default, fair or just in any meaningful way” [p51]
- You cannot be blind to the variable under consideration: “fairness through blindness doesn’t work” [p64]
- “A machine-learning model … is by definition a tool to predict the future, given that it looks like the past … that’s why it’s fundamentally the wrong tool for a lot of domains, when you’re trying to design interventions and mechanisms to change the world” [p81]
- “Deep neural networks are easily fooled” [the title of a famous paper from 2015]
Or it may be that the so-called “ground truth” turns out, on closer examination, to be very different, as researchers discovered when they realised that recidivism is measured by convictions, not by crimes committed. ImageNet, the corpus of human images used for training machine learning, contains one thousand categories: no image can be tagged outside these thousand terms (and does not allow uncertainty). But this is not a negative book; Christian is arguing simply for a responsible use of AI.
Another sign of the author’s thoroughness is his use of the relevant terminology. He describes the term, provides the relevant citation, but also gives a practical example that makes clear what the term means. I noted over twenty terms that were new, or newly intelligible, to me, from this book, including such colourful phrases as “cross-entropy loss”, the “curse of dimensionality” and “credit assignment problem”.
Perhaps the biggest disappointment of the book is something outside the author’s control: it appeared just before the first generative AI systems. The cover of my paperback edition proclaimed “Be informed on ChatGPT”, which is something of an exaggeration, given that there is no reference in the book to ChatGPT (hardly surprising, as it was only released in 2022, two years after this book was published). Nonetheless, the essential limitations of AI are all here, and all applicable to tools such as Copilot and Claude.
Criticisms? Christian covers such a wide range that it is not easy to keep up, and, perhaps inevitably, the coverage is less compelling when the author describes something not yet solved. Perhaps in these areas, the tone of the book is, for me, a little over-optimistic: “Computer scientists are reaching out to the social sciences as they begin to think more broadly about what goes into the models they build” [p47]. Although the author describes and quotes extensively from various institutes and groups that have sprung up to look at AI and ethics, including Stanford and Oxford, I still think that these groups are not driving the development of new tools at present; they are reacting, rather than driving development.
My experience from working with AI teams is that they get excited about developing the algorithm, and “thinking more broadly” tends to be left behind in the thrill of adding a few percentage points to the F1 score. If those teams could be persuaded to read Christian’s book, they might then go on to present us with a more nuanced, and more usable, set of AI tools.
Leave a Reply