j. Soc. Cosmet. Chem., 48, 283-288 (November/December 1997) Meta-analysis is a cost-effective tool for estimating mildness differences P. B. NEUMANN, K. D. ERTEL, B. H. KESWICK, and G. Y. RAINS, The Procter & Gamble Company, Sharon Woo& Technical Center, 11511 Reed Hartman Highway, Cincinnati, OH 45241. Accepted for publication December 1, 1997. Synopsis Meta-analysis is a quantitative method of combining results from independent studies to form an overall conclusion based on all available data. The objective of this research was to validate the application of meta-analysis as a tool to estimate treatment differences over a series of small, randomized, blinded, pilot studies that followed the same protocol. Three forearm controlled application technique (FCAT) screening studies were conducted to compare dryness and lBS capacitance differences induced on forearms treated with two personal cleansing bars. In each study, subjects' forearms were washed twice daily for five days on randomly assigned fixed volar sites according to a procedure that simulates normal consumer use of personal cleansing bars. Visual dryness and lBS capacitance data were collected at baseline and after five days. Both weighted and unweighted meta- analyses were performed to estimate the difference between the two treatments. An FCAT study that enrolled 105 subjects was then conducted to compare the same two cleansing bars. Meta-analyses estimates of treatment differences derived from the screening study data closely paralleled the estimates obtained in the larger study. The visual dryness difference was estimated to be 0.167 + 0.079 by the weighted meta-analysis and 0.168 + 0.080 by the unweighted meta-analysis. P-values were 0.035 and 0.036, respectively. The visual dryness difference was estimated to be 0.194 + 0.040 by the larger study. For IBS capacitance, the weighted meta-analysis estimated the difference between treatments to be -0.039 + 0.013 (p-value = 0.004) and -0.042 + 0.015 (p-value = 0.006) for the weighted and unweighted meta-analysis, respectively. The larger study estimated the treatment difference to be -0.023 + 0.004 (p-value 0.0001). This example demonstrates that estimates obtained from studies pooled by meta- analysis can adequately predict the results obtained from a single large-base-sized trial. INTRODUCTION The purpose of this article is to demonstrate with a practical example how closely estimates of treatment differences derived by pooling data from several small clinical trials correspond to results obtained from a larger clinical trial when the studies are all conducted according to the same protocol. Meta-analysis is a statistical method for systematically pooling data from a series of studies to obtain a more robust estimate for a treatment effect than could be obtained from an individual study. Meta-analysis techniques to combine studies in a statistically 283

284 JOURNAL OF THE SOCIETY OF COSMETIC CHEMISTS legitimate manner were first published by Tippett in 1931. These techniques are widely recognized today in the statistical and clinical literature, even though the actual term "meta-analysis" was introduced by Glass in 1976 in a study of psychotherapy (1-11). Chalmers estimated in 1991 that in excess of 150 meta-analyses of randomized clinical trials had been published in the English language, and that new ones were appearing at an increase of over 15 % per year (12). Meta-analysis has been applied to a diverse number of problems ranging from agriculture to medicine, psychology, education, and even cloud seeding (12). It is a particularly powerful technique for pooling data from several studies conducted under a common protocol with common treatment comparisons. In developing a new product, numerous small screening studies are normally conducted to guide final formulation. These screening studies represent a significant time and capital investment for a company, and can be an important resource for estimating treatment effects. For example, multiple small studies to investigate formulation and/or processing variables provide the researcher with information about the range of factors that can be altered without affecting product performance. Screening studies are often considered individually when making decisions at a particu- lar stage of a project, with a larger study conducted at the end of development to support claimed benefits prior to marketing the product. For example, suppose that a researcher has tested numerous formulation or processing variables in the development of the final product. Each of these small studies provides the researcher with information about the range of factors that can be altered without affecting product performance, even though the studies may not be large enough to show statistically significant differences versus a benchmark control. However, combining data from a number of these small studies can produce an estimate of treatment differences that is more reliable and more generally applicable than data from a single large study. The reason for this is that a single large study is usually conducted under a limited breadth of factors that could impact perfor- mance. Examples of these types of factors are weather conditions, season of the year, geographical location, local population, and product batches. By combining data from smaller studies, a more realistic estimate of the true treatment effect can be made. Due to the wide application and popularity of meta-analysis techniques, there is a growing number of publications providing cautions and guidelines for the appropriate use of meta-analysis. These focus on issues such as establishing criteria for the inclusion of studies in meta-analysis, evaluating the quality and weighting of individual trials, and minimizing publication bias (3,8,13-15). As summarized by Olkin in Science, "Doing a meta-analysis is easy. Doing one well is hard" (16). METHODS Three forearm controlled application technique (FCAT) screening studies were con- ducted from April 1994 to February 1995, comparing the clinical mildness of a new mild cleansing bar with a marketed mild cleansing bar as a benchmark (17). Visual dryness and lBS capacitance data from these studies were pooled by meta-analysis techniques to estimate the difference in mildness between these two mild bars. In March 1996, an FCAT study enrolling 105 subjects was conducted to compare treatment difference estimates provided by the meta-analysis.