CONSUMER TESTING A GUIDE TO PRODUCT DEVELOPMENT 185 The repeated pair test measures the amount of vacillation which occurs in the testing of any given pair of samples and, therefore, allows the re- searcher to estimate the size of the relatively small core of people who ex- hibit a real ability to discern a difference and choose between the samples. MULTIPLE PAIR TEST INCLUDING DEGREE OF PREFERENCE While the multiple pair test increases the information which can be ob- tained from a given number of judgments, a still greater efficiency can be gained if the consumer is asked, in a paired preference, to express her degree of preference for one sample over the other rather than simply stating which she prefers. This technique (7), in effect, provides a weighted preference which can be used to evaluate more precisely and sensitively the importance of product attributes on consumer acceptance. By including an expression by the consumer of how much she prefers one sample over the other, enough additional information is made available so that complex interrelationships between variables can be measured. Using the example of the lipstick, it may be required to select not only a perfume but also a color and a degree of gloss, each of which factors is represented by several different possibilities or levels. If one variable were to be evaluated at a time the researcher would necessarily overlook the possibility that the preferred gloss may depend on the color selected which in turn may depend on the type or intensity of perfume. By this technique, it is now possible to design tests wherein several variables, each at several levels or degrees, can be evaluated simultaneously. Frequently the range of variables and levels can be selected in advance so that when the results are completed and the data analyzed, the optimum combination of factors has been included in the ranges covered and can be specifically and precisely defined. The required number of consumers to run such a test is not so great as might be assumed from the large number of factors being evaluated. The designs used permit fractional replicadons, and generally, not many responses are necessary on each of the pairs of var- iables tested. This type of test design is extremely informative and valuable where a larger number of variables are involved. Its effective use requires careful planning in advance and its design must generally be worked out for the specific case in hand. Many different experimental samples must be prepared representing each of the variables to be tested. Even after the test is completed, the analysis of the data and interpretation of results is time consuming. Capable statistical service is almost indispensable for full appreciation of the possible advantages. Once these conditions are met, however, this technique provides a maximum amount of information through efficient use of consumer responses.

186 JOURNAL OF THE SOCIETY OF COSMETIC CHEMISTS CONCLUSION Obviously, selecting the right test is not simply a matter of picking up a book and deciding just what kind of test should be run. Consumer test- ing is a valuable tool and, as such, requires considerable thought and plan- ning. It has great value in its ability to give the technical researcher an unbiased measure of the all-important effect of product qualities on con- sumer acceptance. This is what every sincere technical man wants from his tests, and not a glossing over or approval which can only result in dis- appointment when a new product is finally marketed. In some cases it may be desirable for an organization to establish its own consumer test functions. In other businesses this may be too expensive and a better choice may be to consult established agencies who provide con- sumer testing of various sorts as a service. These will have available in varying degrees the necessary facilities for contacting consumers, designing tests, analyzing data and interpreting results. If these services are to be purchased outside of the organization, it may be well for the research de- partment to maintain a small staff or at least one man who is familiar with testing techniques, their capabilities and limitations and who can assist in selecting, designing and interpreting test results. In any case, a little understanding of the value to be derived and the opportunities which exist in this rapidly improving field cannot fail to be useful. The ultimate objective is better products with better market and profit potentials. Consumer testing, properly used, will help to achieve this objective. REFERENCES (1) Bradley, R. A., and Terry, M. E., Biometrika, 39, 324 (1952). (2) Carroll, M. B., "Consumer Product Testing Statistics," New York, Reinhold Publishing Co. (1958). (3) Dykstra, O., Jr., Biometrics Bull., 12, 301 (1956). (4) Dykstra, O., Jr., "Rank Analysis of Incomplete Block Designs: Unequal Numbers of Repetitions of a Method of Paired Comparisons," Master of Science Thesis, Rutgers University (1957). (5) Ferris, G. E., Biometrics, 14, 39 (1958). (6) Ishler, N.H., Laue, E. A., and Janisch, A. J., Food Technology, 8, 389 (1954). (7) Scheffe, H., •..din. Statist./lssoc., 47, 381 (1952).