Access this content offline
Omniguides by NOX
Skip to the reading

1.5 Correlation and Linear Regression

Section Overview

  • Gallery 1.5.1 - Scatterplots & Correlations
  • Example 1.5.1
  • Gallery 1.5.2- Scatterplot & Reg on Calc
  • Gallery 1.5.2 - Lin. Reg. & 2 Vars on Calc
  • Interactive 1.5.1 - Example 1.5.1
  • Pop Quiz 1.5.1

Scatterplots

We’ve already discussed using two categorical variables together, visually expressed in the form of two way tables. In this section we will discuss using two quantitative variables together. If you have taken any sort of math or physical science class before, you are probably already familiar with the scatterplot, a graph that displays the relationship between the explanatory variable (represented by the x-coordinate and graphed on the x-axis) and the response variable (represented by the y-coordinate and graphed on the y-axis). An observation’s explanatory variable is named as such because we want it to predict, explain or cause a response in the observation’s response variable.

Scatterplots have three primary attributes – their direction, strength, and form:

Direction – If the x-values increase as the y-values increase on the scatterplot, then the two variables have a positive association. Conversely, if the x-values increase as the y-values decrease on the scatterplot, then the two variables have a negative association. If neither of these scenarios occurs, then we have no association.

Strength – Strength defines how closely a scatterplot follows a clear form. We call scatterplots with a very clear linear form strong and scatterplots with a less clear linear form weak.

Form – We can usually describe a scatterplot’s form in terms of being linear or non-linear. One thing that occurs in both linear and non-linear scatterplots that we can’t directly account for in either the direction or strength of a scatterplot is any clustering that occurs in certain sections of the graph.

Correlation

We can use a measure to also describe the relationship’s direction and strength. This measure is known as the correlation coefficient, represented by the variable rr.

  • For relationships with a positive association, r>0r > 0.
  • For relationships with a negative association, r<0r < 0.
  • The range of values the correlation coefficient can take is −1≤r≤1-1 \le r \le 1. The closer rr is to ±1\pm1, the greater the stronger the relationship is between the two variables.
  • Similar to the mean and standard deviation, the correlation coefficient is not a resistant measure because it is strongly affected by outliers.
  • The correlation stays the same regardless of which of your quantitative variables is your explanatory variable and which is your response variable. It’s up to you to determine which variable is which!

What can the correlation tell us about the relationship’s form, though? Well, we can calculate the correlation for any scatterplot, but rr is best at measuring linear relationships. This means that even if the scatterplot has an rr that is close to 0, the scatterplot could still follow some form, such as a curve. See Gallery 1.5.1 for some examples of scatterplots and their estimated correlation values.

Regression Line

Our brain loves to interpret data by creating models. That might explain why you feel the urge to draw a “line of best fit” through the points on a scatterplot, formally called a least-squares regression line. It is named as such because it minimizes the squares of the vertical distances between the points on the scatterplot and the line. The regression line is represented by the equation y^=a+bx\hat y=a+bx, where

  • aa = intercept, or the y-value of the line where x=0x=0
  • bb = slope, or the value that tells you by how many units yy has increased when xx has increased by 1 unit
  • xx = value of explanatory variable
  • y^\hat y = predicted value of response variable

The slope bb can also be obtained through the formula:

b=r(sysx)b=r\left(\frac{s_y}{s_x}\right)

where

  • rr = correlation coefficient
  • sxs_x = standard deviation of x
  • sys_y = standard deviation of y

Example 1.5.1

Example 1.5.1

A high school Statistics teacher wants to see if backpack weight is correlated with grade point average. The table on the following page shows the weights and the cumulative GPA of his 21 students.

a) Make a scatterplot of the data and interpret its direction, form, and strength.

b) Calculate the correlation, intercept, and slope, of the least squares regression line.

Backback Weight (kg)Grade Point Average (%)
0.5679
0.6780
0.8679
1.0984
1.5482
1.8983
2.1686
2.4882
2.5372
2.8388
3.1083
3.3385
3.5989
3.8987
3.9185
4.1386
4.5690
5.1181
5.3292
6.2787
8.3081

The equation displayed in frame 10 is Y1=81.097392965234+0.85077433543894XY_1=81.097392965234+0.85077433543894X.

The visible summary values in frames 8 and 9 are transcribed below. The table associates them with the recognizable calculator statistics; the original overlapping labels remain available in the images.

StatisticBackpack weightsGrade Point Average
Mean3.24380952483.85714286
Sum68.121761
Sum of squares298.3348148079
Sample standard deviation1.9668057264.508721707
Population standard deviation1.9194058514.400061842
Number of observations (visible in weight display)21

The GPA display also shows ∑xy=5778.17\sum xy=5778.17.

Example 1.5.1 — recorded answer

a) See Gallery 1.5.2 on how to create scatterplots on your calculator. You can also create one by hand, but we recommend entering the data into your calculator since we will be using it to make other calculations. Upon sketching/graphing the scatterplot, we see that a very weak, positive linear correlation exists between the two variables: backpack weight and GPA due to the two noticeable outliers in the data, the one where the student has an unusually low GPA of 72% and the one where the student has an unusually high backpack weight of 8.3 kilograms.

b) See Gallery 1.5.3 on how to find the correlation, intercept, and slope for a least squares regression line.

For the correlation, we get r=0.371r=0.371, which confirms our earlier statement of a weak, positive correlation.

For the intercept, we get a=81.097a=81.097.

For the slope, we get b=0.851b=0.851. We can also confirm this value by using our slope formula:

b=r(sysx)b=r\left(\frac{s_y}{s_x}\right)

We already know the correlation, and the standard deviations are calculated to be: sx=1.967s_x=1.967 and sy=4.509s_y=4.509. See Gallery 1.5.3 on how to find these values. Plugging them into our formula we get:

0.3711⋅(4.5091.967)=0.8510.3711\cdot\left(\frac{4.509}{1.967}\right)=\mathbf{0.851}

Interactive 1.5.1 Example 1.5.1

Captured annotated scatterplot for Example 1.5.1. Horizontal axis: Backpack Weight (kg), with labels 1 through 7. Black points and an increasing blue regression line are shown. An Intercept callout points toward the line at the left; a Slope and Correlation callout points to the line near the middle. The vertical axis labels are absent from this source view. Open full-size image S0104
Source gap (S0104): The screenshot records the “Intercept” and “Slope and Correlation” callouts but no activated callout text or other interaction states. The vertical-axis labels are absent in the captured view.

Pop Quiz 1.5.1

Question 1 of 5

Out of the following 4 pairs of variables, which pair isn’t the explanatory and response variable respectively?

Question 2 of 5

Using the least-squares regression line y^=7.6−0.25x\hat y=7.6-0.25x, how much does yy increase for each single unit increase in xx?

Question 3 of 5

From question 2, we can tell that the correlation between xx and yy is

Question 4 of 5

For the following regression line, y^=16+0.97x\hat y=16+0.97x we can conclude that the scatterplot is

Source gap (S0108): Question 4 displays an empty option D. Its text is absent from the screenshot and is not supplied here.
Question 5 of 5

A Least-squares regression line:

Source gap: The five captured quiz states S0105–S0109 display no checked answers, grading, or feedback. No answer key was recorded.
Glossary and deeper reading · 11 refreshers

These editorial refreshers supplement the reading. Definitions and examples are added guidance; original source wording and recorded answers remain in their original context.

  • Scatterplot

    A scatterplot displays paired values of two quantitative variables as points. The horizontal coordinate represents one variable and the vertical coordinate represents the other. For example, plotting hours studied against test scores helps reveal a pattern without establishing causation.

  • Categorical variable

    A categorical variable records a group or label for each individual. Numeric labels still count as categories when arithmetic on them has no meaningful interpretation. For example, favorite school subject is categorical, even if each subject receives a numbered code.

    Read more: Categorical variable in the captured Statistics reading →

  • Two-way table

    A two-way table organizes counts for combinations of two categorical variables. The cells show combinations, while row and column totals summarize each variable separately. For example, a table can classify students by grade level and whether they walk to school.

    Read more: Two-way table in the captured Statistics reading →

  • Positive association

    Two variables have a positive association when larger values of one tend to accompany larger values of the other. Individual observations need not follow the tendency perfectly. For example, taller trees may tend to have wider trunks.

  • Negative association

    Two variables have a negative association when larger values of one tend to accompany smaller values of the other. The tendency can be linear or curved and does not by itself show cause and effect. For example, higher fuel efficiency may accompany lower fuel use per mile.

  • Mean

    The mean is a measure of center often called an average. For a finite data set, the arithmetic mean is the sum of the values divided by their number; for a random variable, the mean is its probability-weighted expected value when that value exists. Means can be pulled toward extreme values and are not generally resistant. For example, the arithmetic mean of 2, 4, and 9 is 5.

    Read more: Mean in the captured Statistics reading →

  • Standard deviation

    Standard deviation measures spread around the mean using squared deviations and a square root. A sample standard deviation uses the sum of squared deviations divided by one less than the sample size before taking the square root. For example, a larger standard deviation of travel times indicates greater variability, in the original time units.

    Read more: Standard deviation in the captured Statistics reading →

  • Quantitative variable

    A quantitative variable records a numerical count or measurement for each individual. Differences between its values have a meaningful numerical interpretation. For example, the number of books borrowed and the mass of a backpack are quantitative variables.

    Read more: Quantitative variable in the captured Statistics reading →

  • Explanatory variable

    An explanatory variable is used to predict or explain variation in a response variable. In an experiment, researchers deliberately assign values or treatments related to an explanatory variable. For example, assigned study time could be an explanatory variable when test score is the response.

  • Regression intercept

    A regression intercept is the line’s predicted response when the explanatory variable is zero. It may lack a useful contextual interpretation when zero lies outside the relevant data range. For example, an intercept predicts height at age zero even if measurements were collected only for mature plants.

  • Response variable

    A response variable records the outcome being studied. Its relationship to an explanatory variable can be investigated without assuming that the explanatory variable causes it. For example, plant height is a response when researchers compare fertilizer treatments.

Search lessons, terms and problems