Faulty Decisions

  Perhaps this might be a good time to review Bayesian inference and its application to decision making. Suppose we have two methods to assess the quality of items in a given sample. The first method is more exact but is also more difficult than the second. Let's call the results of the first test good (G) and bad (B) and those of the second pass (P) and fail (F). How might we go about comparing them?

We could assume that the probabilities of the two sets of outcomes are linearly related by matrix transforming the first set of probabilities to the second. Since the results are either G or B in the first case and P or F in the second only two components of the matrix are independent so we choose the variables to be the cross terms.

So given p, q, b and c one one can determine p' and q'. Alternatively one can take a statistical approach to the problem. In the example below ten sets of trials with N₀=100 were averaged to get estimates of the values for N₁, N₂, N₃, N₄, N₁₃, etc. Three stochastic variables s_p, s_a, s_d equal to 1 or 0 based on whether or not a random number between 0 and 1 inclusive was less than or equal to the probability associated with the variable were used.

The following results were obtained.

The conditional probabilities are p(i|j) or "the probability of outcome i given j" where i and j are the number of the cells or alternatively G, B, P and F. It was assumed that there was a 10% chance that a G item would test as F and a 20% chance that a B item would test as P. There was only a 1% chance that a passed item would be bad but a 27% chance that a failed item would be bad.

The probability of passing a bad item ends up about twice the probability of failing a good item but the chance of encountering a bad item is relatively rare.

Supplemental (Sep 23): More on conditional probability can be found in Parzen, Modern Probability Theory and Its Applications. The paper cited is Bayes, An Essay towards Solving a Problem in the Doctrine of Chances.

Supplemental (Sep 23): The values above for p' and q' are the estimated values. The calculated values are p'=0.865 and q'=0.135.