CONSUMER TESTING A GUIDE TO PRODUCT DEVELOPMENT 183 SINGLE SAMPLE EVALUATION TESTS These tests are set up so that each respondent receives only one sample. She is asked to rate the sample on some sort of a hedonic scale. She may rate several attributes of the one sample at the same time, but obviously she cannot compare the set of attributes she sees with any other possibili- ties. That is, she cannot be expected to recognize that she would find the sample more acceptable if it possessed a smoother texture, more oil, higher perfume levels, a different color, etc. In fact, her comments about her likes and dislikes are apt to be quite misleading. In the food field, for instance, bitterness is rarely described as such by the tester. Comments of too sour, too acid or too strong are much more common. However, other samples which represent other possible variations can be sent to other consumers and the results compared with each other to decide which makes the best product. This type of test has one major appeal in that the consumer is evaluating the sample under more or less normal circumstances that is, she is not apt to make direct comparisons with other samples in her normal consuming pattern, and this test does not force such a comparison upon her. Here the statistician can design tests by factorial techniques (2) so that combinations of attributes can be evaluated by groups of consumers who see only one type of sample each, and the best choice is worked out when the data are compiled and analyzed. PAIRED COMPARISON TESTS The paired comparison test is one of the most commonly used consumer test techniques. It is used, for example, to compare experimental samples with established products, with competition, with alternate samples which may offer some advantages to consumer, and particularly when it is desired to measure the influence of some specific characteristic. This characteristic may be present in one sample of the pair and not in the other or may be present in some lesser or greater degree. These tests result in a percentage preference expressed for each sample plus some percentage who are unable to express a preference. Commonly, the no preference vote is equally divided between the two samples so thast results are reported as 55-45 percent, 61-39 per cent, etc. Consumers can also be asked for reasons for their preference, and much corollary informa- tion can be gained by this means. MULTIPLE PAIRED COMPARISONS While a simple paired comparison is extremely useful, the researcher is frequently faced with a more complicated situation. Suppose the perfume for a lipstick has been narrowed down to four pos-

184 JOURNAL OF THE SOCIETY OF COSMETIC CHEMISTS sible choices. A selection could be made by testing perfume A against B, C and D in turn, using, say, 100 judgments for each comparison. An inference can be drawn from this test as to the relative preference for A, B, C and D by the direct comparison with A. By the method of multiple paired comparisons (1), the same total number of judgments can be made to yield much more information. All six pos- sible pairs are compared (AB, AC, AD, and BC, BD, CD) using only 50 judgments per pair or the same total number of judgments (300) as before. Information of A rs. B is obtained from the 50 judgments by the con- sumers receiving the pair AB however, information on A rs. B is obtained also by comparing the panel results of A against C with the results of B against C and of A against D with B against D. Each of these 50 indirect comparisons adds about half as much as a direct comparison, or 25 plus 25 in addition to the 50 direct comparisons, making the equivalent of 100 judgments on AB. This, of course, is true for each of the six pairs. The results of this type of test, where all possible pairs receive the same degree of consumer evaluation, provide a much better basis for selecting the best sample of the four variables tested. Furthermore all triangles are automatically closed which provides justification for using the indirect comparisons as well as direct comparisons in evaluating each sample. Methods for analyzing the data from such a test have been published (3, 4). REPEATED PAIR TESTINC} A new technique which provides information about the consistency of consumer's stated preferences has recently been described (5). Each respondent is given the same pair of samples twice to provide a measure of whether or not the stated preference is something firmly established or merely a "passing fancy." Different blind codes are assigned to the pair each time it is tested so that the consumer does not recognize that the second pair of samples is identical with the first. Preferences are usually reported as percentages preferring one sample or the other, or no preference for either. It is usually assumed that some of those people who express a preference may actually be quite insensitive to the difference involved, or may have been influenced by extraneous factors such as which sample was tried first, what code may have been used, etc. However, in an ordinary paired preference test, the researcher has no means of estimating whether respondents are merely being polite in trying to express a preference for one sample or the other, or whether they really see a difference and will continue to recognize this difference. The repeated pair test shows the number of people who originally preferred A but later switched to a preference for B or to a "no preference." By analyzing all the changes which have occurred it is possible to predict the expected con- sistency if pairs were to be repeated with the same consumers many times.