TESTING THE EFFICIENCY OF DEODORANTS 113 The experimental work, however, is of such poor caliber from a scientific point of view that it is hardly worthy of the serious attention that has been forced upon it by the promotional work of advertising men. All in all, it suffers from the following failings, shortcomings, and errors, among others, each of which is not to be found in each article examined, but every paper contains at least one, and usually many of these mistakes: 1. The use of extremely few number of subjects. Findings have even been reported in which only one subject was used! 2. The use of very few judges, usually one judge. 3. The failure to verify this judge's findings by a retest, by himself or others. 4. The failure to present the raw statistical data on which the findings are based. 5. Completely incorrect interpretations of the statistics that are pre- sented. 6. Failure to disclose the nature of the materials tested--both qualita- tive and quantitative. It seems too elementary for words that rm scien- tific periodical with any standing has any right to publicize the claims for a product whose composition is unknown. This position, taken in a cate- gorical statement by the American Medical Association, requires reafFirma- tion. There is good reason to believe that some of the work on chlorophyll was made on deodorants which contained aluminum salts! 7. Failure to exclude judges from a knowledge of the test materials, permitting psychological prejudice to run rampant in the tests. 8. Failure to use any controls in some instances, or adequate controls in others. 9. Failure to use placebos in most instances. 10. Automatic transfer of results obtained with one product to another material of different composition. This is particularly true of serious quan- titative changes in the formulas. It would be possible to choose almost any piece of work reported in this literature and show that it is hardly worthy of acceptance. To illustrate this point, permit me to cite only one article. It appeared, not in some popular journal where everything is slipshod and exaggerated, but in Post- graduate Medicine, written by Drs. Royal M. Montgomery and Henry B. Nachtigal] (2). As the report of an experiment, it is neither better nor worse than most of the writings on chlorophyll. "To further prove the effectiveness of this specially prepared chloro- phyllin, the basketball players were divided into three groups for a two- day test," write these authors. "Group 1 was given 200 rag. daily of raw chlorophyll Group 2 was given 200 mg. daily of a copper salt of chloro- phyll, and Group 3 was given nothing." The authors then present the following table:

114 JOURNAL OF THE SOCIETY OF COSMETIC CHEMISTS TABLE 1.•Os•oscovic REAt•rCS AFTER 48-HouP. TEs•r Before After Exercise Exercise Group 1 400 mg. untreated chlorophyll (5 students) 1.13 1.3 Group 2 400 rag. copper chlorophyll (5 students) 0.5 1.0 Group 3 No chlorophyll (10 students) 0.8 1.13 Now, let us examine this chart. What is wrong with this work? 1. The number of subjects used is pitifully small. There were 20 in all, of whom only five used the chlorophyllin salt. 2. There is no indication of what the variability around each average might be and what the standard deviation for each figure is. The average 1.13 can be obtained from an infinite set of data, and without access to such data the averages are meaningless. 3. It is not even stated whether the computations are means or medians. 4. There is no application of the test of significance. The fact that one figure is lower than another does not indicate, even by this poverty-stricken method of testing, that one product proved more effective than the other. This could only be shown by applying further statistical evaluation to deter- mine whether these results were significant that is, whether they could be obtained by chance alone or are attributable to factors other than chance. For the determination of significance, one should have access to the standard deviations, the standard errors of the means, and the value of t. It is not even disclosed that these determinations were made by or for the authors. 5. There is no evidence that the subjects were comparable to each other before the test and that they gave the same odor score at that time. In a previous part of the same paper, several readings are given for the 20 ath- letes, but they all consist of averages, and nowhere is it explained how the people were divided into 'the three groups and what variations within the groups and from one group to another could be found before this 48-hour test began. 6. There is no indication of the margin of error on this instrument, and there is certainly good reason to doubt that it is smaller than the differences found among the products. In fact, since the instrument records its find- ings only as whole integers, the margin of error must be at least q-1, unless it is contended that it is 7. It is stated that the judgments were taken by physicians. There is no indication as to whether there was one judge or several used for each athlete, whether the same judge was used for all athletes, whether the judges separately tested the same athletes without access to each other's findings, and whether there was any agreement among the judges, if more than one was used.