Some Special Cases in Transverse Least Squares

  The previous results ignored some special cases for the minima of the variance. They are degenerate problems and for completeness they are given here.

Transverse Least Squares Gives Comparable Results To Those of Ordinary Least Squares

  I did some data analysis to compare transverse least square with ordinary least squares and got comparable results. But we start by taking a closer look at some formulas for the transverse least square for the z-values ξ and η.

It turns out that the expected values ⟨ξ⟩ and ⟨η⟩ both equal 0, ⟨ξ²⟩ and ⟨η²⟩ both equal 1 and the slope, σ,  of the fitted line in the ξ,η-plane is ±1. In the x,y-plane the following formulas are useful.

The data analysis did fits of 25 lines with 20 data points per lines having small random normal errors in the values of the coordinates. Each of the two fit methods gave comparable results with the uncertainties in the mean values of the slope, s, and y-intercept, y₀, indicated by the standard deviation, sd, greater than the deviations from the original values as these two samples show.

In the first example ordinary least squares gave the better fit while the opposite was true in the second example. The two methods give independent fits of the lines.

The positions of the data points along the lines ranged from approximately 0 to 2 units from the y-intercept of the original line.

Generating Random Numbers with a Normal Distribution in Excel

  I wanted to test the transverse least squares fit method with some normally distributed errors but Excel doesn't have a random number generating function that will do this. What it does have is the function NORM.INV which will compute the inverse of the normal cumulative distribution function and the RAND function which will generate uniformly distributed random numbers between 0 and 1 and so one can use NORM.INV(RAND(),μ,σ) to generate a set of normally distributed random numbers with mean μ and standard deviation σ. But the result sometimes looks a little strange.

This is because the resulting random numbers produced by the RAND function are subject to statistical fluctuations and its histogram will not have uniform counts.

In the example above the expected value of the count in each partition is about 25. One can get more uniform RAND counts by generating the same number of random numbers for each interval of the histogram.

The resulting set of random numbers is closer to a normal distribution.

An array with 25 rows and 40 columns was used to contain the random numbers above. If p=40 is the number of partitions and k=0 to 39 is an index specifying the columns the formula used for each cell in the array is [k+rand()]/p. One can use the OFFSET function to stack all the cells in a single column but the order is not random and one has to use RAND again to place another random number next to each of the numbers. Both columns can be copied and their numerical values pasted in two new columns and which are then sorted by the second ordinal column. This changes the order of the set of random numbers like one would do in shuffling a deck of cards.

Supplemental (May 26): I've been trying to come up with a descriptive name for the modified probability distribution and "tempered" in the sense of "duly proportioned" seems to be a good choice. Here are some of the steps showing the transfer the table of random numbers into a single column, the addition of a ordinal number and the copy, paste and sort for the set of tempered normal random numbers, rnums.

A Fix for the Rescaling Problem Using Transverse Least Squares

 One can eliminate the dependency of the transverse least squares fit on units of measurement by using z-scores defined as the deviation of the variables from mean values rescaled by the standard deviations, a measure of the spread of the data.

Here's some data to compare this modified method with standard least squares.

The fits can now be done using the z-scores ξ and η instead of x and y.

The comparison shows that the transverse least squares appears to give a better value for the slope of the line.

One can then use the formulas above to convert the fit lines back into the x,y plane.

A Comparison of Linear Least Squares With Transverse Least Squares

  The most common method for fitting data uses linear least squares which minimizes the sum of the squares of the vertical errors or deviations from a straight line. Instead of the sum of squares one can use the expected value, an average, of the square of the errors for the objective function, V, to determine the vertical intercept, y₀, and the slope, s, of the line.

An alternative is to use transverse errors, those normal to the line, instead of the vertical error. The following figure enables one to deduce the formula for the square of the transverse deviation from the line.

Expected values can also be used for an objective function and obtain formulas for the fit solution. One partial derivative gives a formula for the y-intercept of the line in terms of the slope s. Substituting this expression into the second partial derivative gives a quadratic formula for s. For the minimum value of the objective function both partial derivatives are set equal to zero.

To compare the two least squares methods we can generate randoms errors for points on a given line as well as expected values for the data.

Using the formulas for the two fits one can calculated the intercepts and slopes of the lines of best fit.

The two fitted lines tend to coincide as the magnitude of the errors decreases. For the data above the relative error of the coefficients of the lines, y₀ and s, from the original line is smallest for the transverse errors.

Supplemental (May 19): What happens if one rescales the axes? For ordinary least square rescaling the y-axis rescales the coefficients by the same factor. The two axes can be rescaled independently so the axes can have different units of measure. With the transverse least squares there is a tacit assumption that the axes have the same units so one cannot rescale them independently. In this case the fit would not be independent of the coordinate system.

Correction (May 21): The values given for the relative errors for fits are the sum of the square of the errors. The correct values are 0.0926 (fit1) and 0.0932 (fit2).

Stefan's Fit of the Dulong and Petit Data for the Velocity of Cooling in a Vacuum

  In March 1879 Stephan pointed out that the velocity of cooling data obtained by Dulong and Petit could be calculated with an radiation function involving the 4th power of the absolute temperatures. How might he have discovered this? We start as he did with the velocities of cooling for an ambient temperature of 0 °C. In a manner similar to that previously used we can assume the difference of the two rates is Δv=A+Be^λT where T is the absolute temperature of thermometer in the vacuum given in Kelvins. If we assume a value for λ we can use linear least squares to find the remaining coefficients and search for a value of λ which will minimize the error.

Next we subtract A from Δv to get an estimate of the rate for emission of heat radiation. The data appears to be fairly linear in a log-log plot so we look for another set of coefficients for a second fit. A little math allows us to convert the constant term into a factor.

This fit is fairly good but noticing the value of B we are tempted to replace it with an integer, n, to see what we get for a third fit. Assuming n=4 the only unknown is the coefficient a and a search can be used to obtain the best fit. The search works better than averaging ratios of Δv/(T⁴-T₀⁴) to find a.

One gets calculated values and errors very close to those of Stefan.

Supplemental (Apr 1): The averaging of the ratios procedure shows that using some other integer n to compute the coefficient a verifies that n=4 gives the least error.

Edit (Apr 1): Caught an error in the first fit so had to redo it. The fit assumed the formula shown above rather than the original formula shown, Δv=Ae^λT-B. This affected the second fit also since the coefficients were confused and B was added to Δv while A should have been subtracted. Using a log-log fit can can bias the errors somewhat so it is best to work with the original data. The second fit above suggests a 3rd power law and one finds mention of it in some publications. The mistake didn't affect the last fit or the comparison of power laws.

Supplemental (Apr 1): Curvature in a semi-log plot might have suggested the quadratic emission power law mentioned by Dulong and Petit. The slight curvature in the plot of the data in the second suggests the presence of a systematic error in the log of the velocity and even greater fit errors in the velocities themselves.

Absolute Zero of Temperature

  To arrive at the Dulong and Petit empirical formula for the velocity of cooling one does not need to know the absolute temperatures, just changes in temperature relative to the zero of the temperature scale one is using. They did however assume that the rates of emission and absorption of heat were the same function of the absolute temperature. Their use appears to have been influenced by Dalton's discussion of the existence of an absolute temperature in 1808. Dalton states the difference from absolute zero may be approximately 1500 °F below the zero of the Fahrenheit scale but one can interpret this as the heat content of a body expressed in terms of the standard unit of heat based on heat capacity which is not constant for all materials. In the Dulong and Petit empirical formula with the equivalence of emission and absorption the absolute temperature of the thermometer scale used is a common factor that can be removed from the exponential factor and included in the common coefficient.

In 1848 Thomson (Kelvin) proposed an absolute temperature scale based on Carnot's work on steam engines. Joule and Thomson worked on this and a few years later arrived at a value of -273.7 °C for the temperature of absolute zero.

Supplemental (Apr 1): In 1854 Thomson and Joule used the coefficient of expansion of air for an estimate of the absolute zero of temperature equal to 272.85 °C and also indicated where the 273.7 °C value came from. A value for the thermal coefficient of expansion for air, α, equal to 0.0036623/°C had been published prior to this by Regnault in 1842. The formula for the relative expansion is 1 + αΔT so for a temperature change of 100 °C a change of 100α is observed.