Investigative Questions and Data Collection
A good question does more than name a variable. It tells us what information to collect, what kind of analysis will answer the question, and what conclusions the design can support. These connections let us recognize when an impressive calculation is answering the wrong question.
Three Components of a Question
The first component guides data collection and is phrased in terms of the variable or variables of interest. “How long is a student’s usual trip to school, in minutes?” tells us what response to record. A consistent definition of “usual trip” helps keep responses comparable.
The second component guides the choice of analysis. An estimation question might ask for a range of plausible values for a population proportion or mean. A hypothesis-test question must identify the parameter and the direction of the alternative: is it different from a reference value, greater than it, or less than it? For two categorical variables, the question may concern association, equivalently a departure from independence. These are different purposes, even when they begin with the same data sheet.
| Investigative purpose | Example question | Analysis direction |
|---|---|---|
| Not equal | Is the population mean trip time different from 20 minutes? | Either direction from 20 |
| Greater than | Is the population mean trip time greater than 20 minutes? | Above 20 |
| Less than | Is the population proportion of late trips below 0.10? | Below 0.10 |
| Association / not independent | Are transport mode and arriving late associated in this population? | Departure from independence |
| Confidence interval | What range plausibly estimates the population mean trip time? | Estimate a parameter with a range |
The third component identifies the type of conclusion and the population to which it applies. A randomly selected sample can support an inference to its population. An experiment with random assignment of treatments can support a cause-and-effect conclusion about the treatment effect under the study conditions. Naming a population does not by itself make the sample representative, and naming a cause does not turn an observational comparison into an experiment.
Worked Example: Three Connected Decisions
Urbanopolis randomly selects 80 of its 2,400 students to record usual school-trip times. The team asks whether the population mean exceeds 20 minutes. Collection records time in minutes for each selected student; analysis concerns the population mean with an above-20 alternative; the intended inference concerns currently enrolled Urbanopolis students. Because the team did not assign transport methods, it cannot use this design to claim that one transport method causes longer travel.
If the purpose is instead to estimate a plausible range for the mean, the second component changes to a confidence-interval goal. The measured variable and population can stay the same. The distinction lies in what the investigation is trying to learn, not whether one calculation looks more impressive.
Census, Experiment, or Observation?
A census records information from every item or individual in the population. Recording the travel times of all 2,400 students would be a census of that student population. Recording 80 students is not a census merely because every selected student responds. A complete count of a different, smaller group can be a census of that smaller group; always identify the population.
An experiment imposes conditions or treatments. An observational study records variables without imposing treatments. A survey is an observational study that collects human responses using a standard set of questions. “Observational” does not mean that researchers only watch from a distance: interviewing people or using records can also be observational.
Experiment Components
Imagine testing two methods for teaching vocabulary. A researcher assigns students to flashcards or a practice game, then measures the number of words each student recalls one week later. The student is the experimental unit, the observational unit assigned a treatment. With people, we often say subject or participant. The explanatory variable, also called a factor, is practice method. Its categories, or levels, are flashcards and game; these are the two treatments. The response variable is recalled-word count measured after the treatment.
With more than one factor, a treatment is a combination of levels. If practice method has two levels and session length has two levels, 10 and 20 minutes, there are four treatments: flashcards/10, flashcards/20, game/10, and game/20. A treatment is the assigned condition, not the resulting test score. The test score is the response.
| Factor | Levels | Combination example |
|---|---|---|
| Practice method | Flashcards, game | Game |
| Session length | 10 minutes, 20 minutes | 20 minutes |
| Treatment | One level of each factor | Game for 20 minutes |
| Response | Words recalled after treatment | 17 words |
Observational Studies Through Time
In an observational study, researchers record explanatory and response variables of interest without imposing treatments. A prospective study selects observational units at a point in time and gathers information at that time and into the future. Select commuters now, record their current trip information, and track later absences. A retrospective study selects units at a point in time and gathers information from the past: select current commuters and examine their previous month’s trip logs. The key distinction is when the information being gathered occurred relative to selection, not when the researcher finishes typing the report.
Observational relationships can have alternative explanations. A confounding variable is associated with both the explanatory and response variables and can help explain their observed relationship. Suppose students who take the bus have longer trips and also tend to live farther from school. Distance is associated with transport mode and with travel time. It could explain some of the observed bus-versus-walk difference, reducing our ability to interpret that difference as a causal bus effect. A third variable is not automatically a confounder merely because it is recorded; its connection to both variables matters.
Worked Example: Reading Rather Than Assigning
A team surveys 100 residents with the same questions about bicycle use and health, then compares their responses. This is an observational survey. It did not assign bicycle use. A resident’s age might be associated with both bicycle use and health, making it a possible confounder. If instead the team selected residents now and followed their future cycling and health records, it would be prospective; if it examined the selected residents’ past records, it would be retrospective. None of these time labels supplies random assignment.
What Generalizes?
Random selection uses a chance mechanism, such as a random-number generator, to select all sampled units from the population. When observational units or experimental units are randomly selected from a population, generalizations can concern that population, subject to the usual quality of the study and responses. Choosing the nearest people or asking volunteers does not give random selection. A volunteer sample describes the participants and may inform a population of similar individuals, but it does not justify an unrestricted claim about an entire city simply because the sample is large.
Keep random selection and random assignment separate. Random selection links a sample to a population. Random assignment links a treatment comparison to a cause-and-effect argument by reducing systematic differences between treatment groups. An experiment may have random assignment among volunteers and support a treatment-effect conclusion for participants or similar experimental units, while lacking a basis for generalizing to everyone in the population. A randomly selected observational study may support population descriptions while still lacking a causal treatment comparison.
| Selection of units | Assignment of treatments | Main design support |
|---|---|---|
| Random selection | No assigned treatments | Population inference; observational association |
| Volunteers | Random assignment | Treatment-effect comparison; limited generalization |
| Random selection | Random assignment | Population inference and treatment-effect comparison |
| Convenience selection | No assigned treatments | Description of observed participants; limited broader inference |
Independent Practice
Use the information in each question to make your own calculation, representation, or argument. Explain what your result means in its setting.
Original generated practice for the Fall 2026 course.
Glossary and deeper reading · 14 refreshers
These editorial refreshers supplement the reading. Definitions and examples are added guidance; original source wording and recorded answers remain in their original context.
-
Population proportion
A population proportion is the fraction of the entire population with a specified characteristic. It is a parameter, commonly written p, that a sample proportion can estimate. For example, the fraction of all enrolled students who walk to school is the population proportion of walkers.
Read more: Population proportion in the captured Statistics reading →
-
Mean
The mean is a measure of center often called an average. For a finite data set, the arithmetic mean is the sum of the values divided by their number; for a random variable, the mean is its probability-weighted expected value when that value exists. Means can be pulled toward extreme values and are not generally resistant. For example, the arithmetic mean of 2, 4, and 9 is 5.
-
Population
A population is the full set of individuals or measurements a study aims to describe. It must be defined clearly before deciding whether a sample supports generalization. For example, all students enrolled in a particular school this year form a population.
-
Sample
A sample is the subset of a population actually studied. How it is selected matters for the conclusions that can be drawn about the population. For example, 60 selected students may be a sample from a school’s full enrollment.
-
Census
A census attempts to collect data from every individual in the defined population. It avoids sampling variability from selecting only a subset, but can still have coverage, response, and measurement errors. For example, asking every enrolled student about transportation is a census of that enrollment.
-
Treatment
A treatment is a specific condition imposed on an experimental unit. In a multifactor experiment, a treatment is a combination of factor levels. For example, low fertilizer with eight hours of light is one treatment combination.
-
Observational study
An observational study measures individuals without assigning the treatments or conditions being compared. It can reveal associations, but confounding can prevent a causal conclusion. For example, recording students’ existing sleep habits and scores is observational when researchers do not assign sleep schedules.
Read more: Observational study in the captured Statistics reading →
-
Experimental unit
An experimental unit is the smallest unit independently assigned a treatment. A human experimental unit is often called a subject or participant. For example, if fertilizer is assigned to an entire planter, the planter rather than each leaf is the experimental unit.
Read more: Experimental unit in the captured Statistics reading →
-
Explanatory variable
An explanatory variable is used to predict or explain variation in a response variable. In an experiment, researchers deliberately assign values or treatments related to an explanatory variable. For example, assigned study time could be an explanatory variable when test score is the response.
Read more: Explanatory variable in the captured Statistics reading →
-
Response variable
A response variable records the outcome being studied. Its relationship to an explanatory variable can be investigated without assuming that the explanatory variable causes it. For example, plant height is a response when researchers compare fertilizer treatments.
Read more: Response variable in the captured Statistics reading →
-
Confounding
Confounding occurs when the effects of different variables cannot be distinguished in the observed comparison. A potential explanation is that another variable is related to both the treatment or explanatory variable and the response. For example, comparing fertilizer A outdoors with fertilizer B indoors mixes fertilizer and growing-location effects.
-
Random sampling
Random sampling uses a specified chance process to select individuals from a population. It supports population inference when the sampling frame and data collection are appropriate, but does not guarantee perfect balance or eliminate every source of bias. For example, a random-number generator can select student identifiers from an enrollment list.
Read more: Random sampling in the captured Statistics reading →
-
Random assignment
Random assignment uses chance to allocate experimental units to treatment conditions. It tends to balance other influences across groups and supports causal inference in a well-conducted experiment, without guaranteeing identical groups. For example, shuffled labels can assign plants to two fertilizer treatments.
Read more: Random assignment in the captured Statistics reading →
-
Experiment
An experiment deliberately imposes treatments and measures responses. Random assignment, comparison, and replication help distinguish treatment effects from other variation. For example, assigning plants to fertilizer treatments is an experiment.