Organizers: Peter M. Todd, Laura Martignon and Kathryn Blackmond Laskey
SUMMARY
Ward Edwards has declared the 21st century to be the "Century of Bayes." The identification of the rational ideal with the sort of probabilistic reasoning that Bayes championed was first articulated during the Enlightenment, a time of great enthusiasm for reason's potential to liberate humankind from the shackles of dogma and superstition. This view of probabilistic rationality gradually fell out of favor, so that by the beginning of this century probability theory had become just another mathematical tool of the natural sciences, used to model chance phenomena in the physical and social sciences but no longer thought to be the calculus of enlightened human reason. But the late 20th century has seen a resurgence of interest in probability as a model for subjective degrees of belief. After initial skepticism, this view is now flourishing in artificial intelligence and machine learning, and many psychologists have returned to using probabilistic theories as normative standards of rationality. Supporting this trend, the field of decision analysis has focused on developing cognitive tools to help people become better probabilists.
But at the same time, a number of psychologists and cognitive scientists have come to reject the notion that logic and probability theory should be viewed as normative ideals for human rationality. Instead, these researchers propose that humans use simple evolved heuristics to draw domain-specific inferences with incomplete knowledge, limited time, and bounded computational power. Because these cognitive resources are constrained, human reasoning must rely on a toolbox of "ecologically rational" fast and frugal decision-making strategies adapted to the structure of information in the decision environment. (See e.g. Gigerenzer, Todd, and the ABC Research Group, "Simple heuristics that make us smart," Oxford University Press, in press.) In a similar vein, many researchers in artificial intelligence and machine learning argue that the most effective route to machine intelligence is to design more or less simple, "boundedly rational" heuristic algorithms that make no attempt at decision theoretic optimality.
This workshop brings together people in cognitive science, decision theory, and machine learning to consider issues arising from the disagreement between these two very different views of the nature of rationality. We will discuss questions about the rationality and usefulness of simple vs. complex decision-making strategies for humans and machines, including the following:
We will structure the one-day workshop as two three-hour sessions, each containing six short talks and a longer panel discussion involving all the speakers and audience members. The morning session will concentrate on simple and complex models of human decision-making, while the afternoon session will focus on simple and complex machine learning models. In this way, we hope to get those on both sides of the simplicity/complexity fence talking with each other in each session.
If you are interested in participating in this workshop, either with a presentation or just joining in the discussion, please contact one of the organizers to help us schedule the presentations and discussions accordingly.
MORNING SESSION: "Psychological mechanisms"
CHAIR OF THE SESSION PETER M. TODD
| Time | Theme | Speaker |
| 7:45 a.m. | Introduction/Simple heuristics for reasonable decisions | Peter M. Todd |
| 8:15 a.m. | The 'Cognitive Toolbox' Image as the Basis of a Theory of Judgment | Reid Hastie |
| 8:45 a.m. | The Linear Model: Frisky, fallacious, formidable | Ken Hammond |
| 9:15 a.m. | Simple heuristics for Bayesian inference in concept learning | Josh Tenenbaum |
| 9:45 a.m. | Situatedness, Search, and Satisficing | Michael A. Goodrich, Erwin R. Boer, and Wynn C. Stirling |
| 10:15 a.m. | Commentary and discussion | Ward Edwards |
AFTERNOON SESSION: "Normative mechanisms"
CHAIR OF SESSION: N.N.
| Time | Theme | Speaker |
| 4 p.m. | Introduction/To be or not to be a Bayesian...a matter of information formats | Laura Martignon |
| 4:30 p.m. | TBA | TBA |
| 5 p.m. | On the Irrationality of All Statistical Methods | Malcolm Forster |
| 5:30 p.m. | On the Generalizability of Decision Models | Terry Connolly |
| 6 p.m. | Rationality and intelligence | Stuart Russell |
| 6:30 p.m. | Commentary and discussion/When is a Bayesian not a Bayesian? A Decision Theoretic View of Satisficing Heuristics |
Kathryn Blackmond Laskey |
____________________________________________________________________________
Simple heuristics for reasonable decisions
PETER TODD
Decision-making often seems like a difficult, extended process of gathering a lot of information and combining and considering it in various ways. But evolution would not have shaped our minds to use slow and complex mental mechanisms if simpler, "fast and frugal" heuristics could do about as well on important tasks. In our research program on ecological rationality, we have been exploring just how well simple decision-making heuristics--for instance, using name recognition to decide which of two cities is bigger--can do when they are applied in environments with particular appropriate information structure--for instance, environments where bigger things are more often recognized. The key feature of these new heuristics is that they limit the search for further information (often employing "one-reason decision making") or for further objects to consider (through Simon's satisficing aspiration levels) when making a choice. Despite not using all available information or checking many options, these heuristics still compare very favorably with more traditionally rational decision mechanisms in specific domains. In this talk, I will outline our research program, subliminally plug our new book ("Simple Heuristics that Make us Smart", Gigerenzer, Todd, and the ABC Research Group, Oxford U. Press, in press), and present two examples of this approach in two-object choice and sequential mate search.
____________________________________________________________________________
The 'Cognitive Toolbox' Image as the Basis of a Theory of Judgment
REID HASTIE
Elementary, automatized cognitive processes like frequency estimation, identity recognition, similarity perception, and causal inference rules, as well as even more basic elementary information processes are candidate components for the construction of cognitive strategies. Which heuristics and algorithms are likely to be in our 'cognitive toolboxes,' when are they used, and what are they good for?
____________________________________________________________________________
The Linear Model: Frisky, fallacious, formidable
KEN HAMMOND
Topics will include: similarities and differences between the traditional linear model and the "fast and frugal" algorithm, issues to resolve.
____________________________________________________________________________
Simple heuristics for Bayesian inference in concept learning
JOSH TENENBAUM
I will describe a new theoretical framework for concept learning, based on Bayesian inference. Given examples of a new concept C, the Bayesian learner constructs a hypothesis space of all possible extensions of C and assigns each of these hypotheses h a probability of being the true extension of C, based on the likelihood of the observed examples under h. In order to decide whether to generalize C to a new object y, the learner computes the probability that y is an instance of C by integrating the predictions of all hypothetical extensions, weighted by their probability. This Bayesian theory explains a central phenomena that lies beyond the scope of conventional learning theories: how people are often able to generalize a concept after seeing only a very few positive examples.
Carrying out these Bayesian computations directly would be intractable even in very simple domains, because the relevant hypothesis spaces are typically huge or even infinite. However, I show that for some important special cases, such as concepts corresponding to conjunctions of independent binary features or conjunctions of independent intervals in a continuous multidimensional space, the Bayes-optimal generalization procedure can be expressed exactly in closed form, without the need for the learner to compute any messy integrals. Moreover, the resulting generalization procedures provide a rationale for several well-known cognitive heuristics, such as the dependence of similarity on a contrast between common and distinctive features (Tversky, 1977), and the dependence of generalization range on exemplar variance (Fried & Holyoak, 1984; Rips, 1989). I will also show some empirical patterns of concept generalization from human subjects, which can be fit quite closely by the Bayesian theory with only one free parameter.
In sum, I propose that Bayesian inference may provide a competence theory for human concept learning, and that simple, cognitively plausible heuristics may come remarkably close to achieving this competence in practice.
____________________________________________________________________________
Situatedness, Search, and Satisficing
MICHAEL A. GOODRICH, ERWIN R. BOER AND WYNN C. STIRLING
Conventional decision theories require (1) a performance standard, (2) an ability to evaluate consequences (the results of acting given uncertain state of nature), and (3) the ability to generate and search through a space of possible actions. Such a top-down model of decision making has been criticized because it is neither perceptually plausible, cognitively computable, nor consistent with observations. Perceptual plausibility requires that agents (human and machine) be _situated_ in a real world context with the ability to effect desirable percepts without burdensome representations. Cognitive computability requires that the requirement for _search_ be minimized in the interest of speed, workload, and limitations on working memory. Observations suggest that not only do people systematically deviate from normative predictions of optimal action, but also exhibit a wide range of _satisficing_ behaviors.
We will report our perspective on the interaction between simple decision heuristics and complex decision theories, and give examples from our research in both modeling human automobile driver behavior as well as designing intelligent agents. Based on our work, our hypothesis is that much of decision-making and behavior generation is based on simple satisficing mechanisms learned from experience and adapted to situated existence. We speculate that these mechanisms can be modulated by higher level knowledge and processes that may emulate normative standards of rationality.
____________________________________________________________________________
To be or not to be a Bayesian...a matter of information formats
LAURA MARTIGNON
The famous statement "In his evaluation of evidence man is not a Bayesian at all" (Tversky & Kahnemann, 1972) has been gradually replaced by a far more biblical attitude: There is a time to be a Bayesian and a time to use simple, fast and frugal heuristics. Empirical evidence gathered by cognitive psychologists during the last five years indicates that when information is presented in frequency formats, humans can be Bayesians (Gigerenzer & Hoffrage, 1995). This is true even in tasks in which inferences are to be made based on more than just one cue (Krauss, Martignon, & Hoffrage, 1998; Waldmann & Martignon, 1998). Yet the formats in which information is presented matters greatly: There are formats-- including probabilities, and percentages--that do not foster Bayesian reasoning. There are also tasks, like inferences based on several cues, where the formats that favor Bayesian strategies require sequential presentation and sequential treatment and demand excessive decision time and storage capacity. Thus for multi-cue tasks, humans making decisions under conditions of limited time, knowledge, and memory may instead use fast and frugal strategies that are good approximations of the Bayesian benchmarks. The fact that there are no large systematic differences between the perfomance of simple heuristics used by humans and the Bayesian alternatives for the same tasks can be seen as an argument favoring the thesis that humans are approximate Bayesians, after all.
____________________________________________________________________________
On the Irrationality of All Statistical Methods
MALCOLM FORSTER
The advantage of fast and frugal heuristics, when they work, is that they are fast and frugal. They are "bounded", whereas Bayesian methods are "unbounded." But is that their only advantage?
It may seem that Bayesianism continues to have no competition within the realm of unbounded rationality. One such arena concerns the task of comparing models in science, where scientists use statistical methods in their various manifestations, including Bayes method, classical hypothesis testing, minimum description length criteria, cross-validation and Akaike's information criterion. In this arena, the advent of affordable high-speed computers is reducing the computational expense, so that speed and frugality is less of an issue.
Nevertheless, computer simulations suggest that all of these standard methods are less than optimal in maximizing predictive accuracy when novel predictions are at stake. Such examples include cases of extrapolation in which the domain of prediction extends well beyond the observed data. Other examples include causal modeling, in which it is important to predict what will happen when some novel kind of intervention or manipulation takes place.
If this is correct, then the fast and frugal heuristics used by scientists to compare scientific models may be important not only because they are faster and more frugal than formal statistical methods, but because they are more accurate. This is not an argument against all statistical methods, but it is a criticism of all the ones that are in common use at the present time.
____________________________________________________________________________
On the Generalizability of Decision Models
TERRY CONNOLLY
I will cover the spectrum from highly local decision models (naturalistic, recognition-primed, image theory, etc) to the highly global (SEU, Bayes, etc), discuss the evidence favoring each, and describe how this leads inevitably to a contingency model of some sort.
____________________________________________________________________________
Rationality and Intelligence
STUART RUSSELL
The long-term goal of artificial intelligence (AI) research is the creation and understanding of intelligence. Productive research in AI, both practical and theoretical, benefits from a notion of intelligence that is precise enough to allow the cumulative development of robust systems and general results, yet general enough to avoid defining away the interesting aspects of the problem. In this talk I outline a gradual evolution in our formal conception of intelligence that brings it closer to our informal conception and simultaneously reduces the gap between theory and practice. I also describe some tools for creating intelligence in this formal sense.
____________________________________________________________________________
When is a Bayesian not a Bayesian? A Decision Theoretic View of Satisficing Heuristics
KATHRYN BLACKMOND LASKEY
(no abstract)
____________________________________________________________________________