Using Normal Errors for Line Fits

  One can also derive a "characteristic" equation for the fit of a line starting with the following figure to set up the problem. An arbitrary point, r, can be written as the sum of three vectors linking r with the origin. We need an arbitrary reference point, r₀, on the line and the part of r that remains can be split into a part along the line, ρe, and a perpendicular deviation from the line, δ.

For the point on the line we can use the average position of the data points so we set r₀=<r>. Angle brackets are used here to indicate an average. Using the Calculus of Variations we can solve for the unknown parameter ρ and simplify the expressions for δ and the variance, V. P-bar is a projection operator that eliminates the part of a vector in the direction of the line, e. Again the expression for the variance has to be modified to Φ to allow arbitrary variations of e.

To find the change of Φ corresponding to a change in e we need to express e in terms of the unit vectors in the directions of the axes and the direction cosines of e. This allows us to determine the change in P-bar and the constraint that requires the e be a unit vector. The only way to get dΦ=0 for arbitrary changes in the directions cosines is to set the expression in square brackets equal to zero. The result is another eigenvector equation. This may be what is referred to as "the characteristic equation." Eigenvectors are sometimes referred to as "characteristic vectors" which may have come from 19th century terminology.

The expression for the matrix is rather complicated involving the data points and matrices for which the i,jth component is equal to unity and the rest are zero. Expressing the data points in in terms of the unit vectors allows us to simplify the expressions for the components of M.

We get the same result that we found for fitting planes and their nD analogues. But for the nD line giving the best fit we have to choose the largest eigenvalue and the corresponding eigenvector for e since the data points should be spread out most along the line.