Each member of a random sample of 1,000 adult males from the United States was asked a number of questions, including questions about height and annual income. When the responses were analyzed, it was determined that taller men had greater incomes than shorter men, on average, and the difference was statistically significant. Which of the following conclusions would be most appropriate based on these results?
(A) The study establishes that being tall causes men to have greater incomes, on average, and this conclusion can be generalized to all men in the United States.
(B) The study establishes that being tall causes men to have greater incomes, on average, but this result only applies to the men in the sample.
(C) The study establishes that taller men tend to have greater incomes, on average, than shorter men, and this conclusion can be generalized to all men in the United States.
(D) The study establishes that taller men tend to have greater incomes, on average, than shorter men, but this result only applies to the men in the sample.
Standards
S-IC.3: Recognize the purposes of and differences among sample surveys, experiments, and observational studies; explain how randomization relates to each.
Correct answer and commentary
Student performance
The correct answer to this question is Option (C). The question asks what kinds of generalizations are appropriate based on a survey given to a random sample of adults. Randomly selecting individuals from the population of adult males in the United States creates a sample that tends to be representative of that population. Thus, the study’s conclusions about the relationship between height and income can be generalized to all men in the United States. 49% of students answered this question correctly.
Option (D), selected by 38% of students, was the most popular incorrect answer. Limiting the study’s conclusions to just the sample of men who answered the survey questions demonstrates a misunderstanding of random selection. The purpose of random selection is to allow for generalizations about the population in cases like this where it is not feasible to collect data on every individual.
Options (A) and (B) both conclude that being tall causes men to have greater incomes. Clearly, men’s heights are not randomly assigned, so men who are tall may tend to differ from shorter men on other qualities that affect their income. Therefore, it is not appropriate to draw conclusions about cause-and-effect. The reason for the relationship described in Option (C) cannot be determined from this study. While it is possible that taller men in the aggregate are granted certain privileges which correspond to earning greater incomes than shorter men, it is also possible that other explanations exist. For example, some height differences among men may reflect differences in nutrition and health while growing up, which may be related to family income and educational opportunities, which in turn affect job opportunities.