Bias: Identify What Can Go Wrong — Worked Solutions
Worked Solutions
Compare your reasoning with the worked solutions after attempting the questions. A different valid representation or explanation can earn credit when it satisfies the stated criteria.
Original generated practice for the Fall 2026 course. Return to the lesson and independent checks.
Variation or bias?: worked answer
A bathroom scale adds 2 kg to every true weight. A random sample mean also differs from its population mean simply because the selected people vary.
Part a
Identify which difference is systematic bias and which can be ordinary sampling variation. Connect bias to the estimated parameter.
The scale systematically inflates recorded weights, so means computed from it tend to be larger than the true population mean. A randomly selected sample can differ from the population merely by chance; that difference alone does not establish systematic bias.
| Evidence for credit | Points |
|---|---|
| Systematic upward measurement error identified. | 1 |
| Chance difference distinguished from consistent parameter overestimation. | 1 |
Who decides to answer?: worked answer
A neighborhood website invites anyone to click and rate the proposed parking rules. It reports 10,000 responses. A second interviewer asks only people standing near the office entrance.
Part a
Identify the selection concern in each method and explain why the large response count does not fix it.
The website is voluntary response, with a sample entirely made of self-selected volunteers who may have unusually strong opinions. The entrance method is convenience sampling, selecting accessible people. Neither uses random chance to select neighborhood residents, so a large number does not remove the systematic selection concern.
| Evidence for credit | Points |
|---|---|
| Identifies voluntary and convenience mechanisms. | 1 |
| Explains nonrandom selection and why size is not a repair. | 1 |
Not listed or not responding?: worked answer
An app-only list omits residents without smartphones. Among listed residents randomly selected, some never reply.
Part a
Identify the two distinct bias concerns and explain when each enters the study.
Omitting non-smartphone residents is undercoverage: part of the population has no selection opportunity. Missing replies from selected residents is nonresponse: those people were selected but supplied no answer. Either can be important if affected residents differ on the studied variable.
| Evidence for credit | Points |
|---|---|
| Correct two names. | 1 |
| Distinguishes selection opportunity from missing selected response. | 1 |
| Connects differences to study variable. | 1 |
What might the answer conceal?: worked answer
A questionnaire asks “Do you support our fair and necessary fee?” and then asks participants to report their exercise time while the fitness coach watches.
Part a
Identify two response-bias mechanisms and give one improvement for each.
The first question uses leading favorable wording. Replace it with a neutral question about support for the specified fee. The second self-report may overstate exercise to please the coach; confidential responses and a clear time period may reduce that pressure. Both concern recorded answers differing systematically from true opinions or behavior.
| Evidence for credit | Points |
|---|---|
| Names leading wording and pressured self-report. | 1 |
| Relevant improvement for each. | 1 |
| Connects to systematic recorded-response error. | 1 |
Audit the bus-feedback study: worked answer
A city wants the proportion of all bus riders satisfied with a new timetable. It posts an open poll in its phone app, asks “Do you agree our excellent timetable makes travel easier and cheaper?”, and receives 6,000 replies. Riders without smartphones cannot see the poll. A separate follow-up randomly selects 200 riders from a complete roster, but 130 do not respond; late-night riders are especially hard to contact during daytime calls. The city knows late-night riders face more cancelled trips.
Part a
Identify the voluntary-response mechanism and explain a plausible satisfaction distortion.
All app-poll respondents choose whether to participate, making it voluntary response. Riders with especially favorable or unfavorable experiences may be more motivated than indifferent riders, distorting satisfaction. The information does not establish one definite direction solely from self-selection.
| Evidence for credit | Points |
|---|---|
| Entirely self-selected response mechanism. | 1 |
| Specific motivation difference and cautious direction statement. | 1 |
Part b
Identify undercoverage, including both outright omission and a way a group could be less likely to be selected.
Riders without smartphones are omitted from app invitations. Even among smartphone owners, riders who rarely open the app are less likely to see and join the poll. Both concern selection opportunity for parts of the target population.
| Evidence for credit | Points |
|---|---|
| Identifies omitted group. | 1 |
| Identifies reduced-opportunity group with actual mechanism. | 1 |
Part c
Identify the response-bias problems in the poll question. Rewrite it to collect one clearly defined, neutral response.
“Excellent” leads respondents toward approval, and “easier and cheaper” combines two judgments that may differ. One neutral question is “Overall, how satisfied are you with the new bus timetable?” with clearly ordered response options. Separate questions about journey time and cost are also valid.
| Evidence for credit | Points |
|---|---|
| Identifies leading and confusing combined wording. | 1 |
| Neutral single-quantity/judgment revision. | 1 |
Part d
Identify the follow-up’s nonresponse concern. Use the stated facts to explain a plausible direction of bias in its satisfaction estimate.
Selected riders who do not respond create nonresponse. Late-night riders are especially missing and face more cancellations, so they may be less satisfied than respondents. Their missing responses could make the estimated satisfaction proportion consistently too high relative to the all-rider parameter.
| Evidence for credit | Points |
|---|---|
| Selected nonresponders identified. | 1 |
| Relevant respondent/nonrespondent difference and plausible upward direction. | 1 |
Part e
Explain why increasing the open-poll replies to 60,000 would not automatically repair the design. Name another nonrandom method the city should avoid.
More self-selected replies can preserve the same volunteer and app-coverage distortions. Size does not introduce random chance or include omitted groups. A convenience survey of riders at the stop nearest city hall would also select easy-to-reach riders rather than the population by chance.
| Evidence for credit | Points |
|---|---|
| Explains persistent systematic mechanism despite more observations. | 1 |
| Concrete convenience method and absence of random selection. | 1 |
Part f
Suppose riders report fewer missed rides because they are embarrassed about arriving late. Identify this additional bias source and explain what differs from what.
This is response bias in self-reported behavior: recorded missed-ride counts tend to be lower than true missed-ride counts. It concerns inaccurate answers, not the choice of sampled riders or missing selected responses.
| Evidence for credit | Points |
|---|---|
| Identifies self-report response bias. | 1 |
| Recorded-versus-true downward difference and distinction from selection. | 1 |
Part g
Propose one repair for each of coverage, nonresponse, and wording. Explain why none guarantees that all bias is gone.
Use a complete rider roster with non-app contact methods to address coverage. Contact selected people at varied times and follow up to reduce nonresponse. Use neutral, separate questions to address wording. These target different mechanisms; remaining missing replies, mistaken records, or inaccurate self-reports can still affect results.
| Evidence for credit | Points |
|---|---|
| Three mechanism-specific repairs. | 1 |
| Explains remaining possible errors rather than automatic elimination. | 1 |