Critique of DR. Sanders' Presentation (Preliminary Analysis of the Hanford Data)

Page 1

In presenting this critique of the preliminary analysis of the705502 Hanford data,I wish to make i t clear at the outset that it isnot my purpose t o demolish the validity of the basic analytic technique Dr. Sanders has developed. Indeed, I consider itto be a n ingenious and useful technique, capable of giving valid answers t o some of the questions of interest. I want to focus here on certain details that leave room for improvement,in the hope that constructive criticism on my part at this time will propitious ly affect future analyses of these data. control groups with which t o compare the Hanford employees; given an adequate discussion of this, and has carefully pointed out that many biases may be involved. Nor shall I discuss the estimation of the differences Is hall not discuss the selection of in mortality experience between’Hanford employees and a control group; Dr. Sanders has himself noted how this can, and will, be refined in future analyses. Rather I want to discuss how the presence and absence of differences found so fly ~~~ t n 1-0 attributed to the presence and absence of occupational hazards. I f we perform a test of statistical significance,we are assuming the existence of some populations from which we have random samples. particular case, these populations must be hypothetical ones--the hypothetical infinite population of all conceivable employees a t Hanford, and analogous populations for the control groups. W e assume that our data samples are random samples from these populations, and, on the basis of then,test whether the true populations differ.A valid test of significance js one such that, if the null hypothesis (that the populations do not differ)is true,the probability of finding then to be significantly different is equal to the significance level

Page 2

of the test. All the sign tests that Dr. Sanders has performed are aboslute ly valid ; however, I am unable t o see why any of the other tests, i n particular those in section6 of his presentation, should be valid.(It is possible that they are in fact approximately valid--but I cannot as yet see any logical reason why they should be). The variance (provided it exists) of any random variable x is always estimated unbiasedly by the usual formula In the particular case that x is binomially distributed,this formula i s successes tt.Dr. Sanders uses this result to calculate a standard error of11 the difference between two percentages of post death years ; but,since the distribution of post death years is not binomial, it is irrelevant. It would seem t o me preferable to calculate the percentage of post death years for each individual, and, letting this be x calculates2 by the general formula (1). If the result is s2forn employees and s2for n controls, then the standard error for the difference between the two mean percentages of post death years (x and x ) is simply For large samples such as we have here,the distribution of x approximately normal, regardless of the distribution of percentage post death years(for each individual). Thus w e can use the standard error(2) and the

Page 3

normal distribution to obtain an approximately valid significance test. It should be noted that Iam not suggesting that the difference between the percentage post death years in the two groups be estimated by f1 - x2; but that wet estr or the significance of this difference by comparing f 1 2- x with its i n terms of original units, and yet,for mathematical convenience, transform (This is a bit like saying that we may wish t o estimate means the data before performing atest of significance). A statistical test may be valid, and yet of little use to us.This occurs if the test is not powerful; i.e., hasa low probability of giving a significant result when the alternative hypothesis that we are interested in is true.Atest can be powerful against some alternative hypotheses and yet not powerful against others.Dr. Sanders has noted many significant d if - ference s in mortality experiences between Hanford employees and controls,but i n each case, on examining the pattern of mortality experiences,hehas concluded that the significance is not due to the particular alternative hypothesis :.,:crested in, namely that there is a difference in occupational hazard. We must conclude therefore,ifhe is right,that the tests he has applied have a t least a certain amount of power against hypotheses which w e are not interested in. The question then arises, are his tests powerful against the particular alternative hypothesis we are interested in ? Although I cannot answer this question,I have two specific suggestions that would help detect any difference in mortality experience due to occupational hazard,if in fact such a difference exists. rank correlation between the(signed)difference in mortality experience and the cohort number (year of entering employment); the fact that the pattern.is indistinct,ort5atit shows many reversals of sign,is not sufficient to The first is to actually calculate and test the

Page 4

indicate that there is no underlying trend. ference in mortality experience be correlated,in a similar manner, tothe Second, I suggest that the d if - -_number of years of employment a t Hanford--it seems tome that this would provide a more critical test than merely considering the time of entry into employment. Of course, these two variables are themselves correlated ; and, if this corre la - tion between the number of years of employment and the year of entering employment a t Hanford were perfect,there would be no difference between the If we consider now simply the power of any test to detect a sign if - i cant difference between the Hanford employees and a control group, regardless of what that difference might b e due to, much more can be said.In view of the fact that there have apparently been differences of opinion on this subject inthe past--to the extent that some have considered that this study would necessarily be inconclusive--1 think it is pertinent forme to give the line of reasoning I would have followed, i f asked to express an opinion before this work was begun. Suppose w e knew to start with that there would be about 32,000 employees and 13,000 siblings, and that the mortality (up tothe cutoff date) would be about 0.2 (20%). Then the standard error of the difference between the mortal i ties of the two groups would be about and so a difference of about twice this, .00813 or over 4% of 0.2, would b e required to detect a difference in mortality at the 5% level.Of course, if w e compare the siblings with only those 10,000 odd employees who have siblings

Page 5

in the sibling group, a much larger difference would be required. On the other hand, the fact that each group is heterogeneous (due t o sex, cohort differences,etc.) means that,if the heterogeneity is appropriately eli mi - nated in the analysis,overall a somewhat smaller difference would be suf - fic i ent ; however, i t can b e shown that this effect is extremely small compared with the effect of changing the sample size. It would therefore be true tosay, given these sample sizes,that the observed difference in mortality would have to be atleast about 4% to reach significance ; but this says nothing about length of life. Now letus assume that the variance of length of life,for in di - vi dual s who live to be atleast sixteen,is 120 bears) ' . only used as an illustration,but it is certainly of the right order of This figure is magnitude. If there is only a2 @ %m or tal it y,we can know the length of life of only 20% of our sample, i.e. of about 6,400 employees and 2,600 siblings. Thus, the standard error of the difference between the two mean lengths of life wouid b e about and so a difference of about 6 months would be necessary to detect a difference atthe 5% significance level if information on the mortality is ignored. (This assuines only that the nean length of life is approximately normally distributed, not that length of life itself is). Once again, t>is result depends critically upon the sample sizes assumed; and correct elimination of heterogeneity will improve the situation somewhat. To illustrate the latter point,letus suppose an analysis were performed i n which as much genetic variation as possible were eliminated. To

Page 6

do this each employee would have t o be compared with his own sibling, vhereas inDr. Sanders' analysis a cohort of employees i s compared witha corresponding cohort of siblings. estimates of the heritability of longevity,but let us be generous and guess it t o be around 60%. Then, since sib s have on an average only half their genes i n common, ina more refined analysis the variance would b e reduced by 30%, i.e. i t would be0.7times as large. The standard error would thus be I do not know if there are any reliable 4.7 = 84% of what it would otherwise be; in other words, for the sample sizes assumed, a difference of about 5 months, rather than six months, would b e necessary to detect a difference at the 5% level. Now it can b e proved that if two independent statistics are each significant at the 22% level,the combined significance level is 5%. Thus, to reach a combined significance level of 5%, it is sufficient if the difference in mortality and the difference in length of life are each significant at the 22% level. Assuming the standard errors given by (3) and (4) above, adif - .Lalit y of .00416 x 1.23 = .00512, o r 2.6% of 0.2, would be significant at the 22% level ; and a difference in mean length of life of 0.255 x1.23 = 0.314 years,or a little under four months, would be significant atthe 22% level. So an observed difference of four months in length of life, together with a corresponding difference of 2.6% ir, mortality, would be sufficient to reach significance at the 5% level. It is interesting now to consider the Table 13 presented by Dr. Sanders and compare his results with these theoretical predictions that,asI have indicated,could have been made before the study was begun. To derive the standard deviations in Table 13, Dr. Sanders has calculated the variance of

Page 7

\ post death years observed, over an average life span of about 17years,using the general formula (1) above. Thus the standard deviation for all employees is 3.4443 years, and that for all employees with siblings is 2.4798 years. Dr. Sanders does not present the standard deviation for siblings,soI shall take it as being equal to the latter figure. error of the mean difference in post death years(again assuming 32,000 employees It follows that the standard Thus, an observed difference of twice this, .0574 years or21days, would be significant at the 5% level. amount of heterogeneity,a somewhat smaller observed difference would still (In fact,since the analysis eliminates a certain i be significant at the 5% level). Now the standard error of the mean difference in post death years must be equal to the standard error of the mean difference in years lived, and s o we have a discrepancy: we predicted that a mean dif - ference of about 4 months in length of life would be necessary t o be significant (in combination witha corresponding difference in mortality), and yet when the data are actually analyzed, ina manner that combines mortality with length of life,it appears that a difference of only 2 1 days is necessary.I believe the reason for this lies to a large extent in the fact that the predictions were appropriate for mean life expectancy, whereas Dr. Sanders' analysis considered post death years(or,equivalently,years lived) up toa cut-off point.In other words i t seems tome very possible that,ifDr. Sanders found by his methods a mean post death difference of 2 1 days, and then we waited untl l the last person died in order to estimate the mean difference in

Page 8

/ life expectancy, w e should find this difference to be nearer four months. Using the distribution of length of life as given in life tables for the U.S. population, and the distribution of ages inthe two samples atthe time of entering employment, it would be possible to deduce mathematically just what, i n terms of life expectancy, a difference of post death years of 2 1 days corresponds to.In fact a very approximate calculation,using an abridged life table for the whole U.S. population for 1960, indicates that w e could have predicted (knowing that the mean age of entering employment a t Hanford is 35) that Dr. Sanders would have found the standard deviation of post death years to be about 5; since his analysis results in a standard deviation of about three years,,we can conclude that we are dealing with a more homogeneous population than the whole U.S. population,with a standard deviation about three fifths as large. This would mean that a difference in mean length of life of about 2% - 3 -1onths would be necessary t o be sign if - Ln summary, I do not think we need seriously doubt what the facts of this matter are; previous differences of opinion have resulted from viewing the same facts from different viewpoints, without a clear distinction being made between these vie v points.If we are interested in the difference between Hanford employees and a control group up to the present time, thc.n this is what w e should study.If, however, w e are more interested in the difference in life expectancy between the two groups, then this can also be estimated. Any such estimate will necessarily involve extrapolation,since the majority of both samples are still alive; to obtain it we need t o know the forn! of the d is tri - but ion for age of death, but this is obtainable from published life tables.

Page 9

Thus a maximum likelihood estimate of this difference,in life expectancy, together with its standard error, could b e obtained, though with a fair amount of computer programming and time. what estimated difference in life expectancy would lead to a significant difference,just as w e can calculate from the results in Dr. Sanders' Table 13 the analogous difference in mean post death years. then predict the corresponding differences for larger sample sizes,orthe W e should then know with more accuracy In either case we could necessary sample sizes to make the difference as small as any desired quantity. It is not within my personal realm of competence, however, to suggest how small the desired difference should beto make any study worthwhile. A final point I must make concerns what real difference we could expect to detect ; this is not the same as what observed difference we could expect to detect. To take a specific example, suppose w e are interested in the mean difference in post death years, which w e can assume t o …