From Grant McCracken’s blog piece entitled “Why there will always be an anthropology”

In the Wall Street Journal today, the book review opens this way.

Consider Linda, a 31-year-old woman, single and bright.  As a student, she was deeply concerned with discrimination and social justice and also participated in antinuclear protests.  Which is more probable?  (a) Linda is today a bank teller; (b) Linda is a bank teller and active in the feminist movement.

[Psychologists Daniel Kahneman and Amos Tversky determined] that most respondents picked "b," even though this was the narrower choice and hence the less likely one. 

Shaywitz, the reviewer, says that Kahneman and colleagues have

reshap[ed] the study of economics by challenging the assumption that a person, when faced with a choice, can be counted on to make a rational decision.

I would argue that "b" is the rational decision.  It shows us the respondent working with what he knows.  We have given him a little information and he is working this information into an intelligent choice. 

That's a completely ridiculous conclusion to draw from the experiment - even statistically. You could replace 'bank teller' with 'bartender' or 'lawyer' and the result would be the same. Having primed somebody with the suggestion that Linda is socially conscious, it is actually more likely for them to think that she's active in the feminist movement than that she's a bank teller or school bus driver, because, statistically, someone likely to be be concerned with discrimination is also likely to be concerned with feminism. (This is the 'prior information' that steve postrel talks about).

Consider this re-wording of the question:

"Li Shen is 19 and earns passing grades in college. As a child, he used to draw with crayons, and played with his classmates. Which is more probable? (a) Li is studying law; (b) Li is studying law and has friends"

This is a question that is disconnected from the premise supplied (okay, that is debatable, but it is still a tenuous connection). Given that the intent of the experiment is to determine whether people will pick the more probable (read larger) set of two, it must be demonstrable that (a) the conclusion does not follow from the premise; and (b) that one of the sets is larger than the other (presumably there are law students somewhere without any friends - but this requires interpretation). Any other experimental design induces interpretation on the behalf of the subject. Let's try again:

"Which is more probable? (a) Li is studying law; (b) Li is studying law and has brown hair"

Better. But completely pointless. In fact, the experiment could have been written with the choices like this:

"Which is more probable? (a) Linda is active in the feminist movement and a bank teller; (b) Linda is a bank teller"

or even:

"Which is more probable? (a) Linda is active in the feminist movement; (b) Linda is active in the feminist movement and has red hair"

I'm willing to bet that with the last formulation, the subjects would have picked option (a) - the correct, 'rational' answer. One does not have to resort to a discussion of the context of definitions to see that this is an experiment that proves absolutely nothing.

Do Kahneman & Tversky have any excuse for having wasted perfectly good grant money on such a badly designed experiment?