2.12 Sampling and Randomization Distributions
The Values Are Not Always the Same Kind of Thing
Consider a small model population whose values are 1, 3, 5 and 7, each equally likely. The population mean is 4. Draw two values independently with replacement and calculate their sample mean. One sample might be (1, 7): its two observations are 1 and 7, and its mean is 4.
The population distribution describes individual values 1,3,5,7. The sample distribution describes the observations in one particular sample. A sampling distribution instead describes a statistic over all possible samples of a fixed size from the same population. Here its values are two-observation means. It is not a histogram of the original observations, and it is not a distribution of sample sizes.
All 16 ordered samples are equally likely. Organizing the mean for each possible pair gives the exact sampling distribution:
| First draw / second draw | 1 | 3 | 5 | 7 |
|---|---|---|---|---|
| 1 | 1 | 2 | 3 | 4 |
| 3 | 2 | 3 | 4 | 5 |
| 5 | 3 | 4 | 5 | 6 |
| 7 | 4 | 5 | 6 | 7 |
| Mean | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
|---|---|---|---|---|---|---|---|
| Probability | 1/16 | 2/16 | 3/16 | 4/16 | 3/16 | 2/16 | 1/16 |
The means cluster around 4 even though the individual population is uniform on four separated values. Some mean values, such as 2, are not individual population values. Fixing n=2 is essential: changing n changes the statistic’s sampling distribution.
Simulate the Statistic, Not Just the Observations
Exact enumeration becomes impractical for large populations. A simulation can approximate the distribution by repeatedly generating random samples from a population model with known parameter values. For this toy model, generate two independent uniform integers 1–4 and map them to 1,3,5,7. Calculate and record one mean per sample. Repeat many times, then tabulate or plot those means. The number of observations in each sample remains 2; the number of plotted statistic values equals the number of repetitions.
A finite run need not reproduce the exact 1:2:3:4:3:2:1 pattern. Its recorded statistic distribution approximates the all-possible-samples distribution. A single mean or a screenshot of a simulator is insufficient: record the repeated samples and their calculated statistics so the approximation can be checked.
Keep Responses Fixed; Randomize Their Labels
For a different experiment, suppose four puzzle-completion times are 2,4,6 and 8 seconds. Two participants received method A and two received method B. Under a no-effect model, the observed response values can be reassigned to the two treatment groups: choose two distinct participants uniformly for A; the other two form B. Keep the responses fixed and calculate for each reassignment.
| Values assigned to A | Values assigned to B | D: seconds |
|---|---|---|
| 2,4 | 6,8 | −4 |
| 2,6 | 4,8 | −2 |
| 2,8 | 4,6 | 0 |
| 4,6 | 2,8 | 0 |
| 4,8 | 2,6 | 2 |
| 6,8 | 2,4 | 4 |
Repeated random reallocation or reassignment, with the statistic recorded each time, produces a randomization distribution. It approximates the statistic’s reference/sampling behavior under the repeated-assignment model. The mechanism differs from drawing fresh population observations: no new response values are created, and both group sizes stay fixed. This small complete enumeration is an exact reference; a finite random subset of reassignments estimates it.
What Changes as Sample Size Grows?
The central limit theoremEditorial qualification: The captured n ≥ 30 statement is not a universal guarantee. Under the CLT’s assumptions, the accuracy of a Normal approximation depends on the population and sample size. explains why random-sample means can have approximately normal shapes even when the population values are not normal. With independent sampling and finite population variance, larger samples improve the normal approximation. Averages also become more concentrated around the population mean. Increasing the number of repetitions improves our view of a fixed sampling distribution; increasing the size of each sample changes that distribution itself.
For a strongly skewed population, much larger samples may be needed; n=30 is not a universal guarantee. A bounded four-value population is simpler than an extremely skewed one. Nor does the CLT say every statistic or every four-person reassignment distribution is normal. Identify the statistic and sampling mechanism before applying it, and describe the actual recorded distribution rather than claiming a perfect shape from a small simulation.
Try It Yourself
The following original practice applies the ideas above to a fresh setting. Show the reasoning requested in each part. Answers and task-specific rubrics are on the separate solutions page.
Original generated practice for the Fall 2026 course.
Glossary and deeper reading · 12 refreshers
These editorial refreshers supplement the reading. Definitions and examples are added guidance; original source wording and recorded answers remain in their original context.
-
Population
A population is the full set of individuals or measurements a study aims to describe. It must be defined clearly before deciding whether a sample supports generalization. For example, all students enrolled in a particular school this year form a population.
-
Mean
The mean is a measure of center often called an average. For a finite data set, the arithmetic mean is the sum of the values divided by their number; for a random variable, the mean is its probability-weighted expected value when that value exists. Means can be pulled toward extreme values and are not generally resistant. For example, the arithmetic mean of 2, 4, and 9 is 5.
-
Sample
A sample is the subset of a population actually studied. How it is selected matters for the conclusions that can be drawn about the population. For example, 60 selected students may be a sample from a school’s full enrollment.
-
Observation
An observation is a recorded value or set of values for an individual or occasion. The individual may be a person, object, or event rather than a human subject. For example, one plant’s measured height is an observation in a plant-growth study.
-
Sampling distribution
A sampling distribution describes how a statistic varies across all possible samples drawn under a specified sampling design and sample size. It is a distribution of statistics rather than of individual observations. For example, the proportions from repeated random samples of 60 students form a sampling distribution.
Read more: Sampling distribution in the captured Statistics reading →
-
Sample statistic
A statistic is a numerical summary calculated from sample data. Its value can change when a different sample is selected from the same population. For example, the proportion of walkers among 60 sampled students is a statistic.
Read more: Sample statistic in the captured Statistics reading →
-
Simulation
A simulation imitates a chance process using a specified model. Repeated simulated outcomes can approximate probabilities when the model faithfully represents the situation. For example, random digits can model repeated draws from a bag with known color proportions.
-
Random sampling
Random sampling uses a specified chance process to select individuals from a population. It supports population inference when the sampling frame and data collection are appropriate, but does not guarantee perfect balance or eliminate every source of bias. For example, a random-number generator can select student identifiers from an enrollment list.
Read more: Random sampling in the captured Statistics reading →
-
Experiment
An experiment deliberately imposes treatments and measures responses. Random assignment, comparison, and replication help distinguish treatment effects from other variation. For example, assigning plants to fertilizer treatments is an experiment.
-
Treatment
A treatment is a specific condition imposed on an experimental unit. In a multifactor experiment, a treatment is a combination of factor levels. For example, low fertilizer with eight hours of light is one treatment combination.
-
Central limit theorem
Editorial qualification: The captured n ≥ 30 statement is not a universal guarantee. Under the CLT’s assumptions, the accuracy of a Normal approximation depends on the population and sample size.
For independent identically distributed observations with finite variance, the central limit theorem explains why the standardized sample mean approaches a Normal distribution as sample size increases. There is no universal sample size that makes every population sufficiently close to Normal; skewness and extreme tails affect the approximation. For example, averaging many independent waiting times can produce a less skewed distribution of averages than the original waiting-time distribution.
Read more: Central limit theorem in the captured Statistics reading →
-
Statistical independence
Statistical independence means that learning one outcome does not change the probability distribution of another. For two events, it implies that the probability of both occurring equals the product of their probabilities. For example, independent draws with replacement keep the second draw’s color probabilities unchanged by the first.
Read more: Statistical independence in the captured Statistics reading →