Statistics and Regression, lesson 7
What regression assumes
Errors, outliers and the assumptions that quietly break.
Least squares always gives you a line, even when a line is the wrong idea. This lesson measures how sure you can be of a slope, then shows three ways that measure quietly breaks: a bend, uneven noise and rows that repeat. You need the least-squares line from the last lesson and t-statistics from Tests and p-values.
By the end, you can:
Part 1. How sure the slope is
Compute a slope's standard error and t-statistic.
The five weeks of GULL gave a slope of 0.8. Five other weeks would give another slope, as five other days give another average. The slope's standard error says how far it typically moves.
Example 1.1
The slope's standard error over five weeks
The line GULL = 0.2 + 0.8 × market misses the five weeks by −1, −0.4, −0.6, 0.8 and 1.2. The market's squared distances from its average add to 10. How sure is the slope?
- Add the squared missesΣe² = 1 + 0.16 + 0.36 + 0.64 + 1.44 = 3.6
e stands for a residual.
- Divide by n − 23.6 / 3 = 1.2
The line fitted two numbers, a and b, so its misses run a little small. n − 2 makes up for both, as n − 1 did for an average.
- Divide by the market's spread, then take the root√(1.2 / 10) = 0.35
The leftover spread, over how spread out the market's weeks are.
- Count standard errors from zerot = 0.8 / 0.35 = 2.31
As with an average's t-statistic.
AnswerStandard error 0.35, so t = 2.31. For a slope from five points the 95% bar is 3.18, not 1.96, so p ≈ 0.10: five weeks can't rule out a slope of 0.
Read the formula in two halves. The top, √1.2 ≈ 1.1%, is the typical miss around the line: more noise, more wobble. The bottom is the market's spread. Points far apart pin the tilt, like a plank held at both ends. Bunched points let it swing.
Standard error of the slope
How far a fitted slope would move if you drew the data again: the leftover spread around the line, divided by the spread of x. The slope's t-statistic is b over it, read against the t number for n − 2: 3.18 for five points, close to 1.96 for a few hundred.
SE(b) = √(Σe² / (n − 2)) / √Σ(x − x̄)²
The formula takes three things on trust: a straight line is the right shape, every miss has the same spread, and no miss says anything about another. Any of them can fail with the slope unmoved, and then the standard error is wrong, along with every p-value built on it.
Your turn
A regression over 27 weeks gives a slope of 0.5. Its squared residuals add to 50, and Σ(x − x̄)² = 50. What are the slope's standard error and t?
Part 2. What the residuals show
Read a residual plot for a bend, a fan or a far point.
Keep going with Pro
You've done the first part. Pro opens the rest of this lesson, and every other one.
- What the residuals show
- Copies aren't evidence
Statistics and Regression overviewSpotted a mistake? Tell us.