The results of two actual runs of the game described here are in. The experiment was performed with a Mensa group and a non-Mensa group. Each group knew of their own status (ie the Mensa members knew they were playing against other Mensa members). The results:

Non-Mensa. Mean=26.9 Winning number 18.
Mensa Mean=21.18 Winning number 14.

The experimenter's theory is that this difference reflects the different IQ's of the two groups. The smarter group is closer to the game theoretic result. Rob, however, made the excellent point that it could be entirely due to a difference in expectations. If you expect others to be more rational, you will adjust your guess downward. It might be that with two subsets of the same group, we'd get this same phenomenon merely from telling one half they were playing against average intelligence and the other that they were playing against high intelligence.

In fact, the guy's model specifically says "In general as the fraction of irrational players increases from 0 to 100% the optimum submission moves from 1 to 33." And "I made it clear that each group was playing against their peers. Mensa understood their peer group to be much more intelligent than average but the same did not hold for banking staff." Based on these two ideas, my default assumption would be that the difference was due to expectations of opponent intelligence, not individual intelligence. If you told Mensa members they were playing against bank tellers, they might well have the same average as bank tellers playing against bank tellers (or at least close).

It would be fascinating to re-run the experiment in this fashion. Alternatively (or additionally), choose two groups with different IQ's, but tell them the same thing about what sort of opponents they have. This would tell us whether the variation is due to intelligence or expectations.

-----Original Message-----
From: Zietsman, Garth GC [mailto:Garth.Zietsman@standardbank.co.za]
Sent: Tuesday, August 17, 2004 15:18
To: 'Neil Emerick'
Subject: FW: Results and explanation of the number submission competition.

This is the feedback I promised - along with my explanation. 

What's that? You don't care! OK then move on to my next email. 

The challenge in the competition was to submit an integer, from 1 to 100 inclusive, to a competition, on the understanding that the winning number would be the one closest to 2/3 (two thirds) of the value of the mean number submitted. The problem for a competitor was to deduce/infer/guess what 2/3 of the other competitor's 2/3 guess would be.

[By the way there were questions about my use of the word mean instead of average. Mean has a precise meaning whereas the meaning of average is ambiguous. Mean is the sum of all entries divided by the number of entries. This is one of the three forms of average. The others are the median and mode. The median is the 50th %ile or point at which the sample is divided into halves. The mode is the most popular submission and there can be more than one of these, all of them widely divergent. These three types of average need not coincide.]

There is a rational answer to the challenge. The definition of rational is the maximisation of payoff given the information available. In broader terms, rational is whatever decision/action optimises your 'utility function' (best overall value to you, considering your personal wants) given the information at your disposal.

The rational (or best) answer is 1. Why?

Well consider the situation with only two players. Obviously the mean is halfway between the two entries and 2/3 of that value must be closest to the lowest submission. The only solution that can't be beaten is to make sure the other fellow cannot submit a number lower than yours. So you have to pick 1 if you have any brains at all.

Suppose there are many players and they are all rational. Obviously no number > 67 could possibly win and so no rational person would choose a number greater than 67. But if 67 is the maximum possible number for rational people it follows that no rational person would choose a number > 2/3 of that or > 44. If that becomes the maximum acceptable number for rational players then it wouldn't be rational to choose a number > 2/3 of that i.e. > 29, etc, etc, etc. One can immediately see that IF ALL THE PLAYERS ARE RATIONAL the only possible answer is the number closest to 100*(2/3 to the power of infinity) i.e. the only possible rational answer is 1.

But suppose all the players aren't rational. The moment that is true, 1 is no longer the answer most likely to win. Now you (a rational person) have to make a judgement on how rational the other contestants are. 

Suppose you are sure that all the other players are completely irrational. In that case, the optimum assumption would be that they would select numbers at random i.e. the submitted numbers will be uniformly distributed with a mean of 50. 2/3 of 50 is 33. So you would submit 33. 

Easy enough but suppose some fraction of the players are irrational and the rest rational. You now have to think about how the rational player will respond to the rational and irrational players respectively and how they will combine them. The mean of the irrational fraction will be 50 as before. In the absence of irrational players, the rational players would all submit 1. So if the rational and irrational groups combined (assuming equal size) the lowest mean submission should be midway between them i.e. 25.5 and 2/3 of that is 17. But the irrational players aren't absent and the rational players would have to submit a number > or = 17 and the winning submission would move to 22, etc, etc - and ideal submission would go on creeping up. On the other hand no rational person would give a submission higher than 33. If they all submitted 33 (and the irrationals 50 on average) the winning submission would be 28. Knowing that, the rational players would aim at 28 thereby making the winning number 26, etc, etc. The two trends - from 17 up and 28 down - would converge on 25. So if exactly half the entrants are totally irrational and the others rational the rational entry would be 25.

If you assume a different proportion of irrational players this process will converge on a different optimum. In general as the fraction of irrational players increases from 0 to 100% the optimum submission moves from 1 to 33.  

How do we estimate the overall degree of rationality? Until now I have assumed either complete irrationality or complete rationality in each player. Of course players are rarely one or the other and have a certain degree of rationality. However to deal with this one simply needs to sum the degree of irrationality in each player to obtain the expected degree of irrationality in the sample. Then that is treated as the proportion of totally irrational players and the remainder are assumed to be completely rational. But the problem of calculating the degree of irrationality in each player remains.

It seems as though we use ourselves as reference points and estimate the proportion of people less or more rational than ourselves. Imagine that you playing the game against numerous clones of yourself. Any time you second guessed yourself your other selves would be on to you and you would have to second guess again. You can't possibly outwit yourself or assume that you are dumber than you know yourself to be. So you wouldn't be able to stop second guessing yourself and would rapidly settle on 1. In other words when you know (for sure) the other competitors can understand exactly what you understand you will submit the rational answer. It's only when you think other competitors don't understand what you do, that you think you can predict their number without them predicting yours. [Actually I have found that in practice when people are asked to entertain this assumption they still submit answers > 1. Quite amazing really.]

It's possible that people use intelligence as a proxy for rationality when they compare other competitors to themselves. Indeed I was commonly asked about the intelligence of the other competitors when I conducted this study. The higher the average IQ, the less likely people are to judge themselves more rational than their competitors. Also hardly anyone thinks they are less intelligent than everyone else.   

I would predict therefore that in a real game the value of the winning number would be above 1 and that it would decrease as the average IQ of the competitors increased.

What did I find?

Well I held two separate competitions. One for ordinary bank staff and one for Mensa. Actually the mean IQ of the bank staff sampled was well above 100 but still well short of 133 - the Mensa minimum. I made it clear that each group was playing against their peers. Mensa understood their peer group to be much more intelligent than average but the same did not hold for banking staff. The results were as follows.

Non-Mensa.                                                                                                Mean=26.9 Winning number 18.

Mensa (if exclude those who submitted 69 - outliers & possibly as a joke.) Mean=16.4 Winning number 11.

Mensa (if include the 69 submissions.) Mean=21.18 Winning number 14.

My expectations were confirmed. The entries of both groups are markedly different from the rational answer and the Mensa entries are very reliably lower on average than the non-Mensa entries - according to a t-test (statistical significance way beyond the 0.01% level). 

People do submit answers that are partially shifted toward the rational answer (at least for simple problems like this one) and the more intelligent the average competitor the greater this shift. However the shift is never complete. Why?

Obviously people aren't perfectly intelligent or rational. The fact that most people submit a number above the winning number - a necessity if the mean submission isn't 1 - implies that people are prone to underestimating the proportion of people more intelligent than themselves. Humans are fallible but almost always overestimate their ability. 

The road to more inclusive rationality i.e. a form that doesn't assume everyone else is rational, is to estimate the proportion of irrationality among the other players and then correct that figure downward. For example, if the average individual Mensan had taken one more second guessing step than they were comfortable with he or she would be more likely to have to moved closer to the winning answer than away from it. If you're not playing against others but trying to estimate the outcome of a complex process then downgrade your estimate of your own ability. It also pays to look for alternative answers even when you think you've found the answer.