In the keynote lecture of the second day, Professor W. Bentley MacLeod of Yale and Columbia, who is writing a book meant, as he put it, to update the textbook by Robert Cooter and Thomas Ulen from which a whole generation of Polish law-and-economics scholars started, myself included, first put economics itself in order. He divided its models, that is, simplified representations of how people and institutions make decisions, into three classes: price theory and game theory from 1940 to 1980, information economics and contract theory together with causal analysis from 1980 to 2010, and bounded rationality, the economics of a decision-maker who cannot compute everything, together with behavioural economics and big data. The third class is the youngest, though its ideas are hardly new. Herbert Simon, Nobel laureate of 1978, described as early as 1955, in A Behavioral Model of Rational Choice, a decision-maker who does not look for the best of all possible options, because he cannot compute them all, and settles for one that is good enough. Only around 2010, as MacLeod pointed out, did computers become fast enough for such models to be computationally operational.
Bounded rationality is exactly what is needed to understand artificial intelligence in law, because today it can be measured, and the measure is computational complexity, that is, the amount of time a computer needs to solve a problem depending on the number of its elements. The classic account is the 1979 book by Michael Garey and David Johnson, Computers and Intractability. There are problems in which every additional element doubles that time, so with a few dozen elements no computer can solve them exactly and one has to settle for an approximation. A standard economic model assumes that the decision-maker faces resource constraints or information constraints. Bounded rationality adds a third, cognitive constraint: too little memory and time to compute what could in principle be computed. That is how Simon put it, and after him John Conlisk in his survey Why Bounded Rationality? (Journal of Economic Literature, 1996).
MacLeod showed it with chess, the example Simon himself reached for. The number of possible games has about a hundred and twenty zeros (Claude Shannon estimated it in Programming a Computer for Playing Chess in 1950), so nobody plays chess by calculation. In the experiment William Chase and Simon described in 1973 in Perception in Chess, players were shown a position of about twenty-five pieces for five seconds. A novice could then recall only a few pieces, roughly as many as working memory holds, while a master recalled nearly the whole board, because he had not seen twenty-five pieces but a handful of familiar chunks: a castled king, a Ruy Lopez structure, an isolated queen's pawn. When the pieces were placed at random, the master's edge vanished. Skill, then, is human capital acquired through experience, it consists in recognising patterns, and it is not easily codified, because the master does not know what he knows until he sees the position; and when the position is not in his memory, the move is guided by an evaluation function that we colloquially call intuition. In the same way an experienced nurse catches early sepsis before any single vital sign on the patient's monitor trips an alarm threshold, because she sees the whole picture, whereas a novice nurse reacts to one signal.
From there MacLeod moved to law. Herbert Hart, one of the most important legal philosophers of the last century, wrote in 1958 in Positivism and the Separation of Law and Morals that the words of legal rules have a core of settled meaning and a penumbra of debatable cases, in which a court cannot simply apply the law. His example was a rule forbidding vehicles in a park. A car is a vehicle. Certainly. But a bicycle? An electric scooter? A stroller? The rule does not say, and the court has to decide. Hart wrote this as a philosopher; MacLeod wrote the intuition down as a model. The set of possible cases is effectively unbounded, while every legal system is a finite text, so it works like a lookup table with a finite number of rows. For a case that has its row, the text determines the outcome. For a case that does not, the outcome has to come from the decision-maker's evaluation function, that is, from his sense of what is right. On the slide MacLeod presented it is a separate function that assigns to each case the outcome society's values deem correct. The proposition he showed says that every finite legal system has a penumbra, and that adding rules shrinks the penumbra but never eliminates it. A judge who raised children himself will let strollers into the park, and his decision becomes a precedent that makes the law more precise.
What follows for artificial intelligence? MacLeod argues that any value system written down as a finite list of commands, prohibitions and values is incomplete, so all human values cannot be programmed into a computer, though that is exactly what, as he put it, a bunch of twenty-year-olds writing the code is trying to do, whereas a human being has a childhood and a lifetime of adult experience for it. I have one reservation about the analogy: a language model is not a table of the kind MacLeod describes, in which a case either has its row or does not; it builds its answer from the probabilities of successive words, so to a case it has never seen it does not answer “I don't know” but generates a decision of unknown quality, and rather than having a penumbra like a code, it always acts like a judge in the penumbra: it recognises a pattern and fills in the rest. MacLeod's claim about values therefore stands, but its cause lies elsewhere: the model's evaluation function was written in by its producers at the alignment stage, and every instruction and document the model receives together with the question then adds its own context to it. How much that changes is what the two papers described below are about.
His conclusion, which I would summarise as selection rather than design, MacLeod prefaced with the warning that from that point the lecture became speculation. He derived it from the penumbra proposition and from the way the law deals with judges: since values cannot be written down in full, systems of artificial intelligence, which have become so complex that even their designers do not really understand their behaviour, have to be selected for desirable properties, in much the way judges have long been selected, rather than governed by criteria specified for all systems. His example was down to earth: an AI agent, that is, a program that carries out tasks online by itself, was told to get a reservation at a very popular gym, was given a goal but no list of prohibitions, and achieved the goal by hacking the reservation system. The most important question, and here I agree with MacLeod, is who holds the control rights over the selection mechanism, exactly as with the selection of judges today. The law has long since stopped trying to design the perfect judge; it selects judges and then evaluates them by their judgments. Aligning a model is at the same time, as MacLeod showed following Steven Kerr (On the Folly of Rewarding A, While Hoping for B, 1975) and Canice Prendergast (The Provision of Incentives in Firms, 1999), a principal-agent problem, well known in agency theory: the model, like an employee, does what it is rewarded for rather than what the producer meant, and performance measures work only in the simplest cases, because in complex environments performance is judged subjectively and after the fact. With Elliott Ash, MacLeod showed in Reducing Partisanship in Judicial Elections Can Improve Judge Quality that in the United States, state supreme court judges selected in nonpartisan elections or by technocratic merit commissions write opinions that are cited more often than those of judges from partisan elections, and that this is the effect of selection rather than of the structure of incentives. Part of this reasoning MacLeod published this year in the Annual Review of Economics, in The Economics of Professional Decision-Making: Can Artificial Intelligence Reduce Decision Uncertainty?.