Human rationality and artificial intelligence
Computational capacity is the scarce resource of the mind. Held in common, economics and AI struggle to discern rationality from intelligence.
Is rationality the same thing as intelligence? Is ‘rational’ AI the same as ‘intelligent’ AI? The limited explanatory power of neoclassical economics suggests they are not equivalent. Simultaneously, the rationality achieved with the artificial intelligence of today suggests ways economics can grow beyond its narrow definition of rationality as utility maximization.
Like AI, economics is a field that attracts interest of a varied polarity. In a 1978 essay titled, “Rationality as process and as product of thought”, Herbert Simon noted that the interest in the relationship between economics and other social sciences was in part a reflection of the field’s hubris with respect to their societal contributions. Observing the academic climate in 2024 — little has changed. Current interest in the economics-social sciences relation still appears to be driven by economic hubris, notwithstanding the field’s very real contributions to society. Economic hubris is subject to a certain irony, however, as the field has remained a step behind other social fields, rigorously rederiving their theories, all while the maximization assumption fails to produce new behavioral predictions (Simon, 1978; pp. 5).
Economics, irony aside and pun intended, has monopolized rationality. For its export, economics offers the social sciences a narrow definition of a rational agent as a utility maximizer. On imports, blissfully ensconced in a cocoon of tortured math, economics as a whole does not fully appreciate that all other social sciences make assumptions of rationality (e.g., sociology, psychology, anthropology etc). Indeed, one is able to perform functional analysis without “dressing it in the garb of marginalism” and still manage to identify basic causative processes (Simon, 1978; pp. 4). In fact, Simon suggested economists be circumspect in their endeavor to import economic analysis into other social fields — the commodity they are offering may well be in generous supply (Simon, 1988; pp. 64).
What then, does economics have to offer AI? Presently, not much, as it happens. The current state of economics is a cautionary tale of a stagnating discipline; an illustration (caricature or provocation?) of what occurs when a field stubbornly fixates on a narrow definition of rationality and mistakenly equates the concept with intelligence. AI in its current state, conversely, has a lot to offer economics. Consider the basic problem of searching for an optimal choice in a complex, dynamic system; how exactly the search is carried out may vary, but the relevant question is when to terminate the search. On one hand, an optimizing model has a correct point of termination based on the marginal cost of search. When defining rationality, this is where most economists end. On the other hand, given computational constraints, a satisficing model terminates search based on an offer exceeding an aspirational level. Arguably, both optimization and satisficing processes are necessary aspects of human rationality and artificial intelligence.
To contrast optimization with satisficing is to acknowledge that the scarce resource of the mind is computational capacity. Normative and positive economics alike tends to be limited to the study of the ‘rational’ allocation of scarce resources. In all fairness, at present not all economists are myopically limiting their focus to the results of choice i.e., the outcomes of decision making. Behavioral economists have escaped the clutches of substantive rationality to explore the process of choice, wandering into the realm of procedural rationality to study the processes behind choosing an action. The former concerns optimization, the latter is a question of satisficing. Behavioral economics, afraid of rocking the boat, however, treats satisficing as irrationality — mental acrobatics must be needed to view pragmatic action in the face of resource constraints as irrational. Psychologists, conversely, comprehend pragmatism. Through adversarial collaboration, and while maintaining their disagreements, Daniel Kahneman and Gary Klein reveal that despite predictable error, intuitive expertise can be extremely rational (e.g., chess players, firefighters etc; Kahneman and Klein, 2009). More bluntly, behavioral economics has yet to recognize that operating with heuristics and biases is as rational as the utility maximization of neoclassical economics.
When it comes to rationality, economics has a handle on optimization, on the matter of satisficing as rationality, Simon turns to artificial intelligence. Drawing on integer programming as an example, Simon explains that comparing the efficiency of competing search procedures is necessary to evaluate tractability. He argued that any theory of rationality must give an account of “problem solving in the face of complexity” (Simon, 1978; pp. 12). Suffice to say, Simon believed that theories of computational complexity had many implications for procedural rationality. One such theory, common to both AI and psychology, the theory of heuristic search, is concerned with search procedures that will let systems of limited computational capacity make difficult decisions in complex settings. One can rely on heuristics for terminating searches. When searching for a good solution, for instance, we can assign upper and lower bounds to the values of solutions (Simon, 1978; pp. 12). Parsimony, I would argue, is accepting that human rationality is a mix of optimization and satisficing.
If we allow for the fact that almost all human behavior is rational, just not only in the narrow sense of maximization, then it follows that the computational constraints of the mind require rational processes to survive with said constraints. Then, the take away for economics should be: when it comes to human rationality, either optimization or satisficing will suffice. Consider the flexibility, for instance, this expansion of human rationality adds to functional analysis. Combining economics with AI, functional arguments describe the movement of systems towards stable self-maintaining equilibria; issues arise in complex systems within dynamic environments where at any moment the system’s momentary position may not be near equilibrium, global or local (Simon, 1978; pp. 4). Through functional analysis, however, one can show that certain behavioral patterns are sufficient for explaining a function, but it does not show that an alternate behavioral pattern would not also suffice. Thus, satisficing solutions remain important for AI, and should hold equal importance for economics.
Satisficing requires asking qualitative and structural questions. As Simon noted, functional analysis in economics is primarily quantitative, this often neglects the qualitative and structural inquiries also made possible by this type of analysis. Economists studying institutions, however, know that compared to market equilibria, maximization assumptions matter little in comparisons of institutions because the relevant question is of satisficing, not optimization. When explaining institutional structures, Simon argues, there is no need to insist that costs and returns are being equated at the margin (Simon, 1988; pp. 63). Moreover, attention is a scarce resource, and as Simon explains “[w]e cannot attend to information simply because it is there.” (Simon, 1988; pp. 73). Borrowing an example of his, imagine governments as parallel computers — it is attention that is scarce, not information. When problems in government become interrelated: “there is that constant danger that attention directed to a single facet of the web will spawn solutions that disregard vital consequences for the other facets.” (Simon, 1988; pp. 73)
Attention scarcity, whether in individuals or organizations, is a result of the limited computational capacity of the mind. Hence, as parsimony lovers, economists need to accept satisficing as a form of human rationality. Indeed, procedural rationality was revealed to be distinct from substantive rationality because of problems of uncertainty and (mutual) expectations. In a futile and slightly pointless exercise, economists have searched for “substantive criteria broad enough to extend the concept of rationality beyond the boundaries of static optimization under certainty” (Simon, 1978; pp. 10). Game theory, fortunately, is a neat solution for many of these problems, but it too needs to expand its view of rationality to include procedural rationality. That is to say, game theory should include both optimizing and satisficing behavior to be useful. Recent (and inspiring) work on simple mechanism design makes it clear that the computational capacity of the mind is a scarce resource; hence, simplicity in mechanism design is essential.
Helpfully, as fields, both economics and AI are full of optimistic optimizers, so there is much common ground to explore in the study of procedural rationality. As a starting point, when it comes to economic theory, instead of just improving substantive criteria, progress will be made by including procedural criteria. To quote Simon, economics can learn from AI through “the study of procedural rationality in circumstances where attention is scarce, where problems are immensely complex, and where crucial information is absent” (Simon, 1978; pp. 14). For AI, the field should note with caution the limitations of neoclassical economics (optimization) and continue to pursue procedural rationality (satisficing). Intelligence and rationality remain distinct. Defining human rationality, moreover, as a combination of optimizing and satisficing is a way for both economics and AI research to accelerate progress. Perhaps rational AI means finding a way to do more with less — and since efficiency matters, understanding human bounded rationality could be the key to artificial intelligence.
