Question: 

In a random sample of high school seniors, the proportion who use text messaging was 0.88. In a random sample of high school freshmen, this proportion was 0.68. Researchers found the difference in proportions to be statistically significant and obtained one of the following numbers for the p-value. Which is it?

(A) 0.03
(B) 0.20
(C) 0.78
(D) 1.56

Level: 
Advanced

Standards

S-IC.5: Use data from a randomized experiment to compare two treatments; use simulations to decide if di"erences between parameters are significant.

Correct answer and commentary

The correct answer to this question is Option (A). When researchers say the difference in proportions is statistically significant, they mean that a difference like the one they found would be unlikely to occur due to sampling variability alone if the true proportions who use text messaging were the same for seniors and freshmen.

Imagine that the true proportions were the same for the two populations. If you took a random sample of seniors and a random sample of freshmen, the proportion who text in the two samples would likely be different, but how different would you expect them to be? If you simulated sampling from these two hypothetical populations many times, calculating the difference in the sample proportions each time, you could create a distribution of differences to use for reference. The p-value tells us how often (as a probability/proportion) a difference of 0.88 – 0.68 = 0.2 or greater would occur in the distribution of simulated differences, created under the assumption that there is no true difference in the two populations. Since the researchers found that the difference was statistically significant, we know that these results would rarely occur due to sampling variability alone, so the p-value must be small. The p-value in Option (A), the correct response, indicates that a difference of 0.2 or greater would only occur about 3% of the time by chance alone if the proportion who text were the same in the population of seniors and the population of freshmen. This is consistent with saying that the difference in proportions is statistically significant.

Since Option (B) and Option (C) are between 0 and 1, they are both possible values for a p-value, but they are not consistent with the researchers’ statement that the difference in proportions is statistically significant. If sample results have a 0.20 or 0.78 probability of occurring by chance alone under the assumption that there is no difference in the populations, then those results do not provide convincing evidence of a difference in the proportion of freshmen and seniors who text. Option (B) was the most popular incorrect choice. Many students probably chose this answer, because 0.2 is the difference between the two sample proportions given in the problem. Option (D) is larger than 1, so it is not a possible value for a p-value.

Student performance