Wednesday, May 1, 2013
Working with Big Numbers
As was mentioned in previous blogs the maximum precision available in Mathcad 11 is about 15 decimal places. The limit is somewhat fuzzy and the roundoff errors can accumulate with the sum of a large number of terms. If the roundoff error is δ=0.5x1015 for a number then the error for the sum of n numbers can be as much as n·δ so one needs to take the necessary care to avoid errors of this kind. I wrote some utilities to work with "big numbers" in Mathcad and did a check on the Mathcad's performance.
Note that Mathcad made a roundoff error on the sum of u and v above. The correlation matrix for the curve fit procedure involves the sum of squares and other even powers of xk which can bias the result. The expected value for the error of the sum is zero but the standard deviation of this sum is proportional to the square root of the number of data points.