SD 142 Solution

Information Theory

2 marks

There is a game called "Twenty Questions" in which a person must determine what another person is thinking of by asking questions which can be answered "Yes" or "No" within 20 questions. Calculate the information content in bits of the answer to one question in this game, assuming the complete 20 questions are asked.

In one question there are two possible events, Yes or No (1) so the answer to the

question gives you 1 bit of information(1). Although there are 20 questions asked, that does not change the information content of one question - that there are 20 questions in this case is a "red herring". Be sure that you read questions carefully.

Hint: The most common error on this question is to try and calculate the information content in bits of "the answer" to what the person is thinking of. Using information theory you cannot calculate this information. Just because there are 20 questions does not mean that there are 20 possibilities for what the person is thinking of, that cannot be determined from the information in this question. Thinking carefully about this direction quickly reveals that the only possible solution to the question is the solution above. A tricky question? Yes, but not if you really understand information theory!