CONSUMER PRODUCT TESTING 7{39 Methods of Obtaining Data There are only two basic methods for obtaining responses from con- sumer panel members. The first is by questionnaire and the second by face-to-face or telephone interview. Even the interviews are commonly conducted by questionnaire. Usually these are structured question- naires, whereby all interviewers use the same words, the same questions, the same procedure. Interviews, understandably, are more expensive than questionnaires mailed to and from the consumer panel members. Interviews provide several advantages. First, they assure a response, whereas a mailed-in questionnaire can be totally ignored. Seocond, they assure early re- sponses. A mailed-in questionnaire can be put aside until the respond- ent has forgotten details about the test product or until after the deadline for compiling and reporting results has passed. Third, more questions can be asked by an interviewer. Less patient consumers will not spend the time to fill out carefully a long and complicated questionnaire a carelessly answered questionnaire is worthless. In using the interview technique, considerable effort should be made to obtain reliable, honest interviewers and then to train them so as not to allow them to introduce a bias. In general, most testing agencies do not identify the client or the underlying reasons for testing the product. This eliminates another source of interviewer bias. A part of the inter- viewer training period is devoted to pre-testing the questionnaire, and if the interviewers are intelligent people, they may be able to point out ambiguous words, better sequences of questions, and suggest where struc- tured probes could be inserted. Questionnaires to be filled in by the consumer panel members also should be pre-tested. With experience the composer of questionnaires learns how to ask questions so that any literate person will understand them. But the danger of questionnaires lies in not knowing the meaning of terms, either those asked by the questionnaire or those used by the consumers. For instance, checklists in questionnaires have asked for preference on mildness. Does this refer to feeling sensations, or to flavor, or to level of flavoring? When a consumer says a toothpaste leaves his mouth clean, is he referring to cleansing ability or refreshment due to the flavoring ? When he says there is too much carbonation in a beverage, does he mean just that or is he saying that the weak flavoring is subjugated by the carbonation ? Not knowing the meaning of terms results from not knowing the

770 JOURNAL OF THE SOCIETY OF COSMETIC CHEMISTS product's properties. People who really know the product should participate in developing the questionnaire and in editing the responses before compilations are made. In fact, a great deal can be learned by reading through all answers from each respondent. This provides a perspective of the over-all issues before the specifics are examined, as well as an understanding of what each panelist is trying to say. REQUISITES OF PILOT CONSUMER PRODUCT TESTING The foregoing discussion of the component parts of consumer product testing was intended to orient the many facets of product testing from a technical person's point of view. The topography of consumer product testing is very similar to that of conducting a laboratory analysis. First, define the objective: What information is needed and with what precision? Second, select the measuring device and analytical pro- cedure: What precautions and controls will be needed ? Third, conduct the analysis and obtain the findings. Fourth, relate the findings to the objective. This approach to product testing is always used in A.D.L.'s Food and Flavor Laboratory. During these studies several important principles have been developed which experience has shown are the most useful for product development problems. In the following discussion, it will become evident that the primary principle in applying consumer panels to product testing is to understand what is to be tested and why. The Discriminating- Communicating Panel Usually a random or quota sample is not used. Because of the desire to have accurate information that can, if necessary, be translated into technical terms, a panel of consumers is selected who have these particu- lar qualifications: First, they are interested in testing the particular product. This means that they will probably complete the test, taking care to respond carefully to the questions and giving the product a fair trial. While aware that they are performing a special favor, the testers also feel they are influencing the design of products that they, as con- sumers, may someday see in the marketplace. Second, this consumer panel is intelligent. This means that they will follow our instructions, try to avoid confusing product identities or codes, and likely will be able to express themselves adequately. Their ability to communicate their observations is of paramount importance. Third, they observe or distinguish a product's properties accurately. This ability to discrim- inate has been demonstrated by pre-testing.