Linear Regression
A least-squares regression line predicts a response from a predictor by minimizing the sum of squared vertical residuals.
Read the question and data before calculating
A slope of $2.4$ means the predicted response increases by $2.4$ response units for each one-unit increase in the predictor, on average.
- Step 1 - Write the paired data: hours studied x = 1, 2, 3, 4, 5 and score y = 2, 4, 5, 4, 5
- Step 2 - Find both means: x-bar = (1+2+3+4+5)/5 = 15/5 = 3 and y-bar = (2+4+5+4+5)/5 = 20/5 = 4
- Step 3 - Multiply the paired deviations and add: (-2)(-2) + (-1)(0) + (0)(1) + (1)(0) + (2)(1) = 4 + 0 + 0 + 0 + 2 = 6
- Step 4 - Add the squared x deviations: (-2)^2 + (-1)^2 + 0^2 + 1^2 + 2^2 = 4 + 1 + 0 + 1 + 4 = 10
- Step 5 - Divide to get the slope: slope b = 6 / 10 = 0.6 points per hour of study
- Step 6 - Use the means to get the intercept: a = 4 - 0.6 x 3 = 4 - 1.8 = 2.2, so the line is y-hat = 2.2 + 0.6x
- Step 7 - Predict and subtract to get a residual: at x = 3, y-hat = 2.2 + 0.6(3) = 4.0, and the real score was 5, so the residual is 5 - 4.0 = 1.0
- Step 8 - Measure how much of the variation the line explains: squared residuals 0.64 + 0.36 + 1.00 + 0.36 + 0.04 = 2.40, total squared spread in y = 6, so r-squared = 1 - 2.40/6 = 0.60
Interpret a changed data situation
Create two data sets with the same positive slope but different residual spread, then compare their prediction reliability.
Show the worked answer after attempting it
Use x = 1, 2, 3, 4, 5 for both sets. Set A: y = 3, 5, 7, 9, 11. Every point lies on y = 2x + 1, so every residual is 0, the sum of squared residuals is 0, and r = 1 with r-squared = 1.00. Set B: y = 5, 3, 5, 11, 11.
Its mean y is 35/5 = 7, the paired deviation products add to (-2)(-2) + (-1)(-4) + (0)(-2) + (1)(4) + (2)(4) = 4 + 4 + 0 + 4 + 8 = 20, and the squared x deviations add to 10, so the slope is 20/10 = 2 and the intercept is 7 - 2(3) = 1. Set B fits the very same line y = 2x + 1.
But its residuals are 2, -2, -2, 2, 0, giving a sum of squared residuals of 4 + 4 + 4 + 4 + 0 = 16 against a total squared spread of 56, so r-squared = 1 - 16/56 = 0.714 and r is about 0.845. Both models predict 13 when x = 6. Set A's prediction is worth trusting to the decimal, while Set B's typical miss is about 2 units, so its prediction should be quoted as roughly 13 give or take 2. Identical slopes, very different reliability.
Conditions that make the summary meaningful
Know scatterplots, mean and standard deviation, slope, residuals, and the distinction between association and causation. Continue through Mathematical Statistics when you need the complete sequence.
Calculate, then return to the context
- Plot the paired data and check form, direction, unusual points, and whether a linear model is plausible.
- Fit or calculate the line, then interpret slope and intercept with variables and units.
- Inspect residuals for pattern and use the model only within a defensible range.
Challenge the numerical conclusion
A result is useful only when it survives the original conditions and a second line of reasoning. Check the completed work before studying the error pattern below.
Check linear regression by a second route
For every observation compute residual $y-\hat y$; a useful linear fit has residuals scattered around zero without systematic curvature.
Reference and notation for linear regression
Reading a correlation coefficient: what each value of r means for the scatterplot and for prediction
| Correlation r | r-squared | What the scatterplot looks like | Example pair of variables |
|---|---|---|---|
| r = 1.00 | 1.00 | Every point sits exactly on one rising line | Inches and centimeters of the same lengths |
| r = 0.80 | 0.64 | A tight rising cloud around the line | Adult height and shoe size |
| r = 0.30 | 0.09 | A wide rising cloud, easy to mistake for noise | Hours of TV watched and test score |
| r = 0.00 | 0.00 | A shapeless blob with no tilt | Birth month and math grade |
| r = -0.60 | 0.36 | A moderately tight falling cloud | Outdoor temperature and hot chocolate sales |
| r = -1.00 | 1.00 | Every point sits exactly on one falling line | Fuel left in the tank and miles driven at a steady rate |
Language you need to use precisely
- Least-squares regression line
- The least-squares regression line makes the squared vertical distances as small as possible. Those distances run from the data points to the line.
- Residual
- A residual is the observed y value minus the value predicted for that x. A positive residual means the point sits above the line.
- Slope
- The slope is the predicted change in y for each one-unit increase in x. Its unit is the unit of y divided by the unit of x.
- Correlation coefficient
- The correlation coefficient, written r, is a number from -1 to 1. It measures how tightly points hug a straight line and which way they tilt.
- Coefficient of determination
- The coefficient of determination is written r-squared. It is the fraction of the up-and-down variation in y that the line explains. A value of 0.60 means the line explains 60%.
- Extrapolation
- Extrapolation uses the line to predict outside the x values you collected. It is risky because the data does not prove that the pattern continues there.
Use linear regression beyond one example
Farming
An agronomist compares corn yield with inches of rainfall to predict harvest size. The slope means bushels gained per extra inch of rain.
Retail planning
A shop fits a line between weekly sales and advertising dollars. The slope estimates the sales linked to each extra $100 of ads.
Pediatric growth charts
Doctors compare a child's height with a fitted trend for age. A large negative residual can be a signal worth investigating.
Connected ideas worth studying next
- Solving linear equations: A regression line is still just y = mx + b, so predicting a y from an x, or solving backward for the x that gives a target y, is ordinary linear-equation work.
- Mean, median, and mode: The regression line always passes through the point built from the mean of x and the mean of y, which is why both means show up in the first step.
- Standard deviation: The slope equals r times the standard deviation of y divided by the standard deviation of x, so spread in both variables sets the steepness.
Ask for help with the decision, not only the answer
Bring your complete linear regression problem and the first step you cannot justify to the AI math solver. Ask it to explain that decision, then reproduce the method and check it against the original conditions.
Questions students ask about linear regression
What is the difference between correlation and regression?
Correlation gives you one number, r, that measures how tightly the points follow a line, and it is the same whether you call x the predictor or y the predictor. Regression gives you an actual equation for predicting one specific variable from the other, and swapping which variable you predict generally changes the line.
What does r-squared = 0.60 mean in plain words?
It means the line accounts for 60% of the up-and-down variation in the y values, and the remaining 40% is scatter the line cannot explain. In the worked example the total squared spread in scores was 6 and the leftover squared residuals were 2.40, so 3.60 out of 6, or 60%, was explained.
Why is it called least squares?
Because among all possible lines, this one makes the sum of the squared vertical residuals smallest. Squaring is used so that a miss of +3 and a miss of -3 both count as bad, and so that one huge miss counts far more than several tiny ones.
Can I use the line to predict outside the range of my data?
That is extrapolation and it is unreliable. A line fit to 1 through 5 hours of study says a 20-hour student would score 2.2 + 0.6(20) = 14.2 on a test out of 10, which is impossible. Real relationships flatten out or bend, and your data has no evidence about what happens beyond where you looked.