Uncertainty in the Cross Terms for the Comparison of the Two Quality Assessments

  When comparing the two processes for determining quality of items using the more accurate estimates good (G) and bad (B) and the less accurate estimates pass (P) and fail (F) the uncertainty  in the cross terms can be quite large. The conditional probabilities were determined as follows.

We can recalculate the Excel worksheet that generated the 10 sets of 100 random estimates of the quality of G, B, P, and F items to get a new set of averages and save the numerical values. With 100 of these trials the average of b and c were determined and as well as the root mean square deviations from this average.

One can see that there is quite a bit of variation in the cross terms even though the average turns out to be fairly accurate. A large number of tests of a given set of items is needed to get good estimates of the cross terms b and c in order to check theoretical results.