Access this content offline
Omniguides by NOX
Skip to the reading

4.2 Inference as a Decision

Section Overview

  • Figure 4.3.1
  • Gallery 4.3.1 - Ways to Change Power
  • Example 4.3.1
  • Interactive 4.3.1 - Example 4.3.1
  • Pop Quiz 4.3.1
Source gap (S0357): Source numbering note: the overview and prose use 4.3.x, while this section’s gallery, interactive, and quiz widgets use 4.2.x. The captured labels are retained.

Two Types of Error

Although significance tests are probabilistic, decisions are not - you either reject H0H_0 and accept HaH_a or vice versa. It is still convenient for us to call the two hypotheses H0H_0 and HaH_a, but H0H_0 no longer has the special status (the statement we try to find evidence against) that it had before when we were conducting tests of significance.

There are two types of incorrect decisions. We can commit a Type I Error, where we reject H0H_0 (accept HaH_a) when in fact H0H_0 is true, or we can commit a Type II Error, where we accept H0H_0 (reject HaH_a) when in fact HaH_a is true. All possible decisions with respect to H0H_0 or HaH_a being true can be found Figure 4.3.1

Figure 4.3.1

Decision grid. Columns are truth about the population: H zero true and H a true. Rows are decision based on sample: reject H zero and accept H zero. Rejecting a true H zero is an orange Type I Error; accepting it is a green Correct Decision. Rejecting H zero when H a is true is a green Correct Decision; accepting it is an orange Type II Error. Open full-size image S0358
Truth about the population and decision based on sample
Decision based on sampleH0H_0 trueHaH_a true
Reject H0H_0Type I ErrorCorrect Decision
Accept H0H_0Correct DecisionType II Error

Measuring α and β

The probability of detecting a Type I error is simply the significance level α\alpha, or

α=P(Type I Error)\alpha=P(\text{Type I Error})

The probability to detect a Type II error is measured by the power of the test. The power of any statistical test is:

Power=1−β,where\text{Power}=1-\beta,\quad\text{where}
β=P(Type II Error)\beta=P(\text{Type II Error})

Suppose we have performed a power calculation and found that the power is too small, meaning ther is a greater probability to make a Type II Error. So what can we do to increase the power and reduce Type II Error? Here are a few ways:

  • Increasing α\alpha - A significance test at α=0.05\alpha=0.05 will have a higher probability of committing a Typerejecting the alternative hypothesis in favor of the null than a significance test at α=0.01\alpha=0.01. 5% test of significance will have a greater chance of rejecting the alternative hypothesis than a 1% test because the strength of evidence required for rejection is less.
  • Increasing the sample size - The more data we have, the more we know about the sample statistic, so we have a higher probability of distinguishing the correct parameter values.
  • Decreasing σ\sigma - Has a similar effect to increasing the sample size.
  • Increasing the distance between the alternative statistic and the null statistic. Values that lie within the sampling distribution where HaH_a is true but also lie close to the null value are harder to detect than alternative values that are further away from the null values.
Source gap (S0359): Source wording note: the screenshot says “ther” and contains the joined fragment “Typerejecting” in the first bullet. It also describes rejecting the alternative hypothesis in that bullet. These visible words are retained; missing wording has not been supplied.

The captured centers are μ0\mu_0 and μA\mu_A. The second caption changes α\alpha; the third changes the distance between μ0\mu_0 and μA\mu_A.

Source gap (S0360): The legend is cut off along the left side in all three gallery captures. Its missing text and color keys cannot be recovered from these images.

Example 4.3.1

Example 4.3.1

You are a Quality Control Manager for Fesqueeze, a company who makes squeezable air freshener products. You have to decide whether or not to approve a outbound shipment of product. A key mechanical component in the product is Normally distributed, with a standardized width of μ=18.000\mu=18.000 mm and a standard deviation of σ=0.010\sigma=0.010 mm. Out of this shipment, you take an SRS of n=5n=5. You will decide to reject the shipment if the mean width of the sample differs at an α=0.05\alpha=0.05 level.

a) Identify Type I and Type II error

b) You decide that a shipment should be rejected if their mean width is 0.015 away from the industry standard. From this information, compare the percentage of the time you make both types of errors.

Recorded answer to Example 4.3.1

a) H0H_0 is the claim that the shipment of Fesqueeze products is good and meets the industry standard of 18.000 mm, or H0:μ=18.000H_0:\mu=18.000

HaH_a is the claim that the shipment of Fesqueeze is bad and doesn’t meet the industry standard of 18.000 mm, or Ha:μ≠18.000H_a:\mu\ne18.000

In this case, the Type I error would be rejecting a shipment, although the shipment was good and the Type II error would be approving a shipment, although the shipment was bad.

b) The probability of making a Type I error is the significance level, or 5% of the time.

Finding the probability of making a Type II error will be a bit more difficult.

Looking at Gallery 4.3.1, we see that the probability of making a Type II error is represented by the solid blue shading, which is part of the sampling distribution that assumes the alternative hypothesis to be true. However, if we look back to Chapter 4.2, significance testing calculates the probability of obtaining a sample statistic under the assumption that the null hypothesis is true, instead.

This means that we first must find the corresponding sample mean xˉ\bar x for a of 1.96, the critical z-value for a two-sided test based on the z-table.

z=xˉ−μ0σ/n=xˉ−18.0000.010/5≥1.96z=\frac{\bar x-\mu_0}{\sigma/n}=\frac{\bar x-18.000}{0.010/5}\ge1.96
xˉ≥1.960.0105+18.000\bar x\ge1.96\frac{0.010}{5}+18.000
xˉ≥18.004\bar x\ge18.004

Then, we find the probability that this sample mean is as at least as extreme as the value obtained if the alternative hypothesis were true, where μa=18.015\mu_a=18.015:

P(xˉ−μaσ/n≥18.004−18.0150.01/5)P\left(\frac{\bar x-\mu_a}{\sigma/n}\ge\frac{18.004-18.015}{0.01/5}\right)
P(z≥−1.41)P(z\ge-1.41)

Our z-score is negative, meaning that we are dealing with area in the left tail, so we would reverse the inequality to ensure we were finding the correct probability:

P(z≤−1.41)=0.08P(z\le-1.41)=0.08

So, which error would be more likely to commit? Well, we have a 5% chance of rejecting a good lot of bearings, and we have an 8% chance of accepting a bad lot of bearings. In this case, it’s more likely that a Type II error will be committed.

Source gap (S0363): Source calculation note: the recorded work prints denominators σ/n and 0.010/5, gives x̄ ≥ 18.004 and then z ≥ −1.41, and reverses that inequality in the following line. Those formulas and reasoning are retained as captured. The phrase “for a of 1.96” and the later reference to bearings also appear in the original.

Mnemonic: Remembering Type I and Type II. A type I error occurs when a statistician is being cone-servative with their results. Type II errors occur when a statistician is two reckless!! (You may hate me right now, but you will always remember this).

Interactive 4.2.1 Example 4.3.1

Captured annotation overview of two Normal curves. The left curve is centered at μ zero = 18.000 and the right at μ A = 18.015. The regions across the top are Reject H zero, Accept H zero, and Reject H zero. Lavender α shading marks two rejection tails of the left curve; green β shading lies to the left of the right cutoff under the right curve. The cutoff is labelled x bar = 18.004, with z = 1.96 partly clipped below. Callouts label Rejection Region 1, Beta, Rejection Region 2, Critical Value, and Alpha. Open full-size image S0365

Captured callout labels: Rejection Region 1; Beta; Rejection Region 2; Critical Value; Alpha.

Captured values: μ0=18.000,xˉ=18.004,μA=18.015,z=1.96\mu_0=18.000,\quad\bar x=18.004,\quad\mu_A=18.015,\quad z=1.96. The legend labels green β\beta and lavender α\alpha.

Source gap (S0365): Only the annotation overview is captured. Expanded callout explanations are absent, and the bottom of the z = 1.96 label is clipped in the original image.

Pop Quiz 4.2.1

Question 1 of 5

When a Type I error occurs:

Question 2 of 5

When a Type II error occurs:

Question 3 of 5

In a two-tail test for the population proportion, if the null hypothesis is rejected when the alternative is true, then:

Question 4 of 5

What level of significance would be most appropriate to choose in a hypothesis test, considering that making a Type I error would be more costly than making a Type II error?

Question 5 of 5

A profit-hungry electronics manufacturer would make the choice to:

Source gap (S0366): All five quiz prompts and choices are captured in S0366–S0370. None of these captures displays a checked answer or answer feedback.
Glossary and deeper reading · 10 refreshers

These editorial refreshers supplement the reading. Definitions and examples are added guidance; original source wording and recorded answers remain in their original context.

  • Significance test

    A significance test assesses how compatible observed data are with a specified null model. It compares a test statistic or p-value with a decision rule chosen for the study. For example, a test can assess whether a sample provides evidence that a population walking proportion differs from a stated value.

    Read more: Significance test in the captured Statistics reading →

  • Type I error

    A Type I error occurs when a test rejects a true null hypothesis. The test’s significance level controls its probability under the null model. For example, a study might report a treatment difference even though the population treatment effects are equal.

  • Type II error

    A Type II error occurs when a test fails to reject a false null hypothesis. Its probability depends on a specified true alternative and the test’s design. For example, a study might fail to detect a real treatment difference when the sample is too small.

  • Probability

    Probability quantifies how likely an event is under a specified chance model. It lies between 0 and 1, with larger values representing greater likelihood. For example, drawing a red card from a well-shuffled standard deck has probability one half.

    Read more: Probability in the captured Statistics reading →

  • Significance level

    A significance level is a threshold selected before testing to limit Type I errors under the null model. A p-value at or below that threshold leads to rejection under the stated rule, while a larger p-value leads to failure to reject. For example, an alpha level of 0.01 demands stronger incompatibility with the null model than a level of 0.05.

    Read more: Significance level in the captured Statistics reading →

  • Alternative hypothesis

    An alternative hypothesis specifies the population difference or direction for which a test seeks evidence. It should be chosen from the research question before inspecting results. For example, an alternative can state that the population mean travel time is greater than 15 minutes.

    Read more: Alternative hypothesis in the captured Statistics reading →

  • Sample

    A sample is the subset of a population actually studied. How it is selected matters for the conclusions that can be drawn about the population. For example, 60 selected students may be a sample from a school’s full enrollment.

    Read more: Sample in the captured Statistics reading →

  • Sample statistic

    A statistic is a numerical summary calculated from sample data. Its value can change when a different sample is selected from the same population. For example, the proportion of walkers among 60 sampled students is a statistic.

    Read more: Sample statistic in the captured Statistics reading →

  • Sampling distribution

    A sampling distribution describes how a statistic varies across all possible samples drawn under a specified sampling design and sample size. It is a distribution of statistics rather than of individual observations. For example, the proportions from repeated random samples of 60 students form a sampling distribution.

    Read more: Sampling distribution in the captured Statistics reading →

  • Statistical power

    Editorial qualification: The captured text describes power as detecting a Type II error. Power is the probability of correctly rejecting the null hypothesis for a specified true alternative, equal to 1 − β.

    Statistical power is the probability of rejecting the null hypothesis for a specified true alternative. It equals 1 minus the Type II error probability for that alternative and can increase with a larger sample or a larger true effect. For example, a study with power 0.8 detects its specified alternative in about 80% of repeated studies under the model.

Search lessons, terms and problems