What would you like to understand?

Popular destinationsCalculatorsCoursesPracticeFormulas

Unlock Premium Help

Explore every tutor and study workspace for seven days, with no daily AI cap during the trial.

Choose a billing period
Ask a parent to pay

Nothing is charged during the first seven days. Unless you cancel, the plan you selected begins when the trial ends and renews on its stated schedule. Manage cancellation from your plan page. Stripe handles the payment details.

Normal equations formula: what it means and how to use it

Find a vector whose model prediction is closest in squared distance to data. This least squares relationship belongs to linear algebra. Read the symbols against the problem, confirm the required conditions, and estimate the scale before calculating.

\[A^TA\widehat{\mathbf x}=A^T\mathbf b\]

Check dimensions before using Normal equations

Write the size and role of every vector, matrix, space, and transformation. An expression can look algebraically familiar while being dimensionally undefined. Rebuild missing background in the linear algebra subject guide, then use linear algebra practice to retrieve the relationship without being told its name.

Follow one object through Normal equations

Track both the coordinates and the geometric or structural meaning:

\[The residual $\mathbf b-A\widehat{\mathbf x}$ is orthogonal to every column of $A$.\]

Carry out the transformation in a valid order

  1. Name the unknown, and the unit the answer has to come out in.
  2. Match the symbols to your values. $\widehat{\mathbf x}$ is the least-squares solution.
  3. Check the conditions before substituting. A unique solution is guaranteed when columns of $A$ are linearly independent; QR is often numerically safer.
  4. Substitute, keep exact values to the last line, then test the sign, size, and unit against a rough estimate - The check that catches most linear algebra slips.

An order or dimension error to avoid

What goes wrong: Students see Aᵀ on both sides and cancel it, dropping back to Ax = b.

Why it fails: You can only cancel a factor that has an inverse, and Aᵀ is usually rectangular with no inverse at all. Cancelling it asks for an exact solution, which is the thing that does not exist.

Repair: Multiply out both sides and solve the smaller square system instead. Fitting a line through (1, 1), (2, 2), and (3, 2) gives AᵀA = [[3, 6], [6, 14]] and Aᵀb = (5, 11), whose solution is intercept 0.667 and slope 0.5.

Multiply back or test a simple vector

After using Normal equations, repeat the calculation with units attached and test the result against a boundary case or a quick numerical estimate.

When the transformation statement holds

If two columns of A are multiples of each other, AᵀA is singular and there is no unique answer, only a whole family of equally good fits. Even with independent columns the step of forming AᵀA squares the conditioning, so near-duplicate columns lose accuracy fast and a QR factorization is the safer route.

Questions about Normal equations

Why can I not just solve Ax = b directly?

Because with more data points than unknowns there is usually no line passing through every point. The system is inconsistent, so you switch the goal from exact to closest.

Should the residual come out as zero?

Only in the lucky case where the data really does fit exactly. Normally the residual is nonzero, and what makes the answer the best one is that this leftover is perpendicular to every column of A.

What does AᵀA being singular tell me about my data?

That your predictors carry duplicate information, such as one column being a constant multiple of another or a sum of two others. Drop or combine the redundant column and the fit becomes unique.

How big is the system I end up solving?

AᵀA is square with one row and column per unknown, no matter how many data points you have. Fitting a straight line to a million points still leaves you with just a 2 by 2 system.

Make it yours

Change one input, predict whether the output should rise, fall, or stay fixed, and only then recalculate.