SUNSCREEN TESTING BIAS 359 sunscreen having an expected SPF of 75. A subject with a MED of 45 mJ/cm2 will receive exposure for approximately 103 min on a test site applied with the same sunscreen. Thus, the subject with the higher MED will dramatically reduce the throughput of the testing facility, thereby reducing profi tability. There is much pressure on testing facilities to commit selection bias, in confl ict with GCP (1,2). Selection bias for subjects with his- torically low MED values for whatever reason would result in a higher SPF of a sunscreen determined by one testing facility versus another testing facility that adheres to a GCP compliant recruitment procedure. SUBVERSION BI AS Additional an alysis presented herein reveals that a subset of subjects consistently presented with values either well above or well below the average SPF value of the dataset (15.6 ± 1.2). This subset of subjects may be instrumental in affecting the SPF value of a sunscreen in a clinical trial. For example, the difference between the aver- age SPF by the subject exhibiting the highest average SPF for P2, 19.8 ± 0.9, and the subject exhibiting the lowest average SPF for P2, 12.3 ± 2.6, is 7.5 SPF units, or 61%. In addition, although this difference in SPF value is the maximum found within this dataset on P2, this difference in SPF value may be even greater within a dataset on a sunscreen having a higher SPF value (4). This subset of subj ects appears to be separate and independent of the relationship between MED and SPF, which causes selection bias. The subject exhibiting the highest average SPF for P2 had an average unprotected MED of 14.9 mJ/cm2, and the subject exhibiting the second highest average SPF for P2 had an average unprotected MED of 18.1 mJ/cm2. The subject exhibiting the lowest average SPF for P2 had an average unprotected MED of 40.2 mJ/cm2. A testing facility might be more likely to commit subversion bias near the end of a SPF test. At the end of an SPF test, one more or two more high SPF values might be needed to obtain the expected SPF. However, the bias would be much more dramatic if conducted throughout all 10 subjects in an SPF test. Exploiting this difference among subjects (subversion bias) would provide another reason for variation in re- sults among testing facilities. Subversion bias for subjects with historically high SPF values, either consciously or subconsciously, would result in a higher SPF of a sun- screen for one testing facility versus another testing facility that adheres to a GCP compliant recruitment procedure. Table II Subjects (n = 6) with All SPF Values below the Average SPF of P2 Subject # Lowest SPF value Highest SPF value Sample size Counts below 15.6 4703 10.440 15.000 3 3 31524 12.836 14.368 4 4 61224 9.061 14.387 4 4 66837 13.064 14.384 3 3 71172 11.400 14.369 3 3 71862 11.583 14.983 3 3

JOURNAL OF COSMETIC SCIENCE 360 CONCLUSION Freeing sunscreen t esting from selection bias and from subversion bias would be a worth- while goal in enhancing the validity of SPF testing results. Currently, the FDA method (9) invalidates an SPF test if all subjects were of the same Fitzpatrick skin phototype. The correlation between Fitzpatrick skin phototype and MED is poor. A test could easily in- corporate SPF test values from nine values from Fitzpatrick skin phototype 1 and one value from Fitzpatrick skin phototype 2 while committing selection bias and/or subver- sion bias. External validity w ill be improved by requiring subjects across all MED values as sug- gested by Alejandria et al (5). They suggested that each valid SPF test includes at least three subjects with a MED of d15 mJ/cm2, at least three subjects with a MED between 15 mJ/cm2 and 40 mJ/cm2, and at least three subjects with a MED of ≥40 mJ/cm2. This would minimize the selection bias reported by Kawanda et al. (3), Damien et al. (4), and Alejandria et al. (5). External validity wil l also be improved by restricting the use of individual subjects. Using a subject, no more than six times per year and no more than once every 60 days would minimize, but not eliminate, the potential for subversion bias. Until selection bias and subversion bias are eliminated, variations in SPF values from different testing facilities will continue. REFERENCES (1) J.A. Lewis. Statistical principles for clinical trials (ICH e9): an introductory note on an international guideline. Statist. Med., 18, 1903–1942 (1999). (2) WHO, Handbook for Good Clinical Research Practice (GCP) (World Health Organization, Geneva, Switzerland, 2002), p. 30. (3) A. Kawada, T. Noda, M. Hiruma, A. Ishibashi, and S. Arai, The relationship of sun protection factor to minimal e rythema dose, Japanese skin type, and skin colour, J. Dermatol., 20, 514–516 (1993). (4) D. L. Damian, G. M. Halliday, and R. S. Barnetson, Sun protection factor measurement of sunscreens is dependent on minimal erythema dose, Br. J. Dermatol., 141, 502–507 (1999). (5) M. Alejandria, A. Marra, G. Roberts, and M. Caswell, Disparate SPF testing methodologies generate similar SPFs. II. Analysis of P2 standard control SPF data, J. Cosmet. Sci., 70, 181–196 (2019). (6) J. A. Hartigan and M. A. Wong, Algorithm AS 136: a K-means clustering algorithm, J. Roy. Stat. Soc. C Appl. Sta t ., 28, 100–108 (1979). (7) P. J. Rousseeuw, Silhouettes: a graphical aid to the interpretation and validation of cluster analysis. J. Compu t. Appl. Math. 20, 53–65 (1987). (8) R Core Team, R: A Language and Environment for Statistical Computing (R Foundation for Statistical Com- puting, Vienna, Austria, 2019), accessed January 21, 2020, www.R-project.org. (9) Department of Health and Human Services, Food and Drug Administration, Sunscreen drug products for over-the-counter human use proposed amendment of fi nal monograph proposed rule, Fed. Regist., 72, 49070–49122 (2007).