More on the Least Squares Fit Systematic Errors

  I decided to do a diagnostic to determine the source of the systematic errors in ordinary least squares and transverse least squares fits and came to the conclusion they are due to changes in the spread of the data caused by the random errors. We know from data analysis that the rms error for two independent sources of error is computed using the Pythagorean theorem so the square of the spread of the values of x, σ_x, will be sum of the squares of a constant term, σ_x0, and the rms error, δx, for x. The same is true for the spread of y.

The diagnostic gave the following results for various values of δx. The initial line had a slope s=1.25, intercept y₀=0.5, and rms error in y δy=0.03.

The numerator for the slope for ordinary least squares is essentially a constant. Only the square of σ_x showed a quadratic dependence on δx.

This seems to present the opportunity for an adjustment of the systematic errors associated with these least squares fits.