Transverse Least Squares Fits Show Better Convergence for Mean Values

  If one repeats a least squares fit a large number of times one would expect the means of the slope and intercept to converge to a particular value and so we should be able to determine if ordinary least squares and transverse least squares give different results. The average values for a batch of 20 trial fits of 25 data points for a given line with random errors couldn't resolve the issue. However the process can be repeated by generating a new set of 1000 normal random for each batch of 20 lines, the values for the fit averages saved and a cumulative average can be computed as the number of batches increases. Transverse least squares appears to show a better convergence to the true values.

The estimate given by ordinary least squares for the slope and intercept of the line appears to be biased.

Some Special Cases in Transverse Least Squares

  The previous results ignored some special cases for the minima of the variance. They are degenerate problems and for completeness they are given here.

Transverse Least Squares Gives Comparable Results To Those of Ordinary Least Squares

  I did some data analysis to compare transverse least square with ordinary least squares and got comparable results. But we start by taking a closer look at some formulas for the transverse least square for the z-values ξ and η.

It turns out that the expected values ⟨ξ⟩ and ⟨η⟩ both equal 0, ⟨ξ²⟩ and ⟨η²⟩ both equal 1 and the slope, σ,  of the fitted line in the ξ,η-plane is ±1. In the x,y-plane the following formulas are useful.

The data analysis did fits of 25 lines with 20 data points per lines having small random normal errors in the values of the coordinates. Each of the two fit methods gave comparable results with the uncertainties in the mean values of the slope, s, and y-intercept, y₀, indicated by the standard deviation, sd, greater than the deviations from the original values as these two samples show.

In the first example ordinary least squares gave the better fit while the opposite was true in the second example. The two methods give independent fits of the lines.

The positions of the data points along the lines ranged from approximately 0 to 2 units from the y-intercept of the original line.

Generating Random Numbers with a Normal Distribution in Excel

  I wanted to test the transverse least squares fit method with some normally distributed errors but Excel doesn't have a random number generating function that will do this. What it does have is the function NORM.INV which will compute the inverse of the normal cumulative distribution function and the RAND function which will generate uniformly distributed random numbers between 0 and 1 and so one can use NORM.INV(RAND(),μ,σ) to generate a set of normally distributed random numbers with mean μ and standard deviation σ. But the result sometimes looks a little strange.

This is because the resulting random numbers produced by the RAND function are subject to statistical fluctuations and its histogram will not have uniform counts.

In the example above the expected value of the count in each partition is about 25. One can get more uniform RAND counts by generating the same number of random numbers for each interval of the histogram.

The resulting set of random numbers is closer to a normal distribution.

An array with 25 rows and 40 columns was used to contain the random numbers above. If p=40 is the number of partitions and k=0 to 39 is an index specifying the columns the formula used for each cell in the array is [k+rand()]/p. One can use the OFFSET function to stack all the cells in a single column but the order is not random and one has to use RAND again to place another random number next to each of the numbers. Both columns can be copied and their numerical values pasted in two new columns and which are then sorted by the second ordinal column. This changes the order of the set of random numbers like one would do in shuffling a deck of cards.

Supplemental (May 26): I've been trying to come up with a descriptive name for the modified probability distribution and "tempered" in the sense of "duly proportioned" seems to be a good choice. Here are some of the steps showing the transfer the table of random numbers into a single column, the addition of a ordinal number and the copy, paste and sort for the set of tempered normal random numbers, rnums.

A Fix for the Rescaling Problem Using Transverse Least Squares

 One can eliminate the dependency of the transverse least squares fit on units of measurement by using z-scores defined as the deviation of the variables from mean values rescaled by the standard deviations, a measure of the spread of the data.

Here's some data to compare this modified method with standard least squares.

The fits can now be done using the z-scores ξ and η instead of x and y.

The comparison shows that the transverse least squares appears to give a better value for the slope of the line.

One can then use the formulas above to convert the fit lines back into the x,y plane.

A Comparison of Linear Least Squares With Transverse Least Squares

  The most common method for fitting data uses linear least squares which minimizes the sum of the squares of the vertical errors or deviations from a straight line. Instead of the sum of squares one can use the expected value, an average, of the square of the errors for the objective function, V, to determine the vertical intercept, y₀, and the slope, s, of the line.

An alternative is to use transverse errors, those normal to the line, instead of the vertical error. The following figure enables one to deduce the formula for the square of the transverse deviation from the line.

Expected values can also be used for an objective function and obtain formulas for the fit solution. One partial derivative gives a formula for the y-intercept of the line in terms of the slope s. Substituting this expression into the second partial derivative gives a quadratic formula for s. For the minimum value of the objective function both partial derivatives are set equal to zero.

To compare the two least squares methods we can generate randoms errors for points on a given line as well as expected values for the data.

Using the formulas for the two fits one can calculated the intercepts and slopes of the lines of best fit.

The two fitted lines tend to coincide as the magnitude of the errors decreases. For the data above the relative error of the coefficients of the lines, y₀ and s, from the original line is smallest for the transverse errors.

Supplemental (May 19): What happens if one rescales the axes? For ordinary least square rescaling the y-axis rescales the coefficients by the same factor. The two axes can be rescaled independently so the axes can have different units of measure. With the transverse least squares there is a tacit assumption that the axes have the same units so one cannot rescale them independently. In this case the fit would not be independent of the coordinate system.

Correction (May 21): The values given for the relative errors for fits are the sum of the square of the errors. The correct values are 0.0926 (fit1) and 0.0932 (fit2).