Technical Report No. 6-85
Improved Monte Carlo estimation of statistical significance for tests of trend in rates or proportions
Kopecky KJ, Preston DLSummary
Asymptotic significance levels of tests for monotone trends in rates or proportions can be profoundly anticonservative when applied to small numbers of events and when distributions of exposure to risk are highly skewed. In such cases Monte Carlo (MC) estimation of observed levels of significance (“p-values”) can be very useful. We describe a simple technique of importance sampling (IS) which can greatly improve the efficiency of MC estimation in this setting. Implementation of the IS technique is described, and the variance of the IS estimator is derived. It is shown that, in many situations likely to occur in practice, the variance is substantially less than that of a simple MC estimator proposed earlier. Generalizations beyond the case of survival data without ties are described, and the use of IS is illustrated with data regarding mortality among atomic bomb survivors.
Asymptotic significance levels of tests for monotone trends in rates or proportions can be profoundly anticonservative when applied to small numbers of events and when distributions of exposure to risk are highly skewed. In such cases Monte Carlo (MC) estimation of observed levels of significance (“p-values”) can be very useful. We describe a simple technique of importance sampling (IS) which can greatly improve the efficiency of MC estimation in this setting. Implementation of the IS technique is described, and the variance of the IS estimator is derived. It is shown that, in many situations likely to occur in practice, the variance is substantially less than that of a simple MC estimator proposed earlier. Generalizations beyond the case of survival data without ties are described, and the use of IS is illustrated with data regarding mortality among atomic bomb survivors.