$8•t JOURNAL OF THE SOCIETY OF COSMETIC CHEMISTS learned that: (a) the local panel produced fewer code-confusers (either because the local panel was a "better" panel or because their alternate, rather than weekly, use of the toothpastes sharpened their memories of the products) and (b) most of the code-confusers had voted for Toothpaste W instead of Toothpaste R, which they had de- scribed as W. This meant, not that the code-confusers balanced out, but rather that the preference percentages increased slightly for Tooth- paste R and decreased slightly for Toothpaste W. Role of Consu•ers Who Do A ot Cast a Preference Vote Also observed in Table I is the increase in percentages of panelists who had no preference, who preferred neither, and who failed to respond to the preference question. In the last column, such persons in the na- tional panel who did not cast a preference vote were more numerous than those who voted in favor of Toothpaste W i.e., 25.3% voted for W and 26.1% did not vote for either of the test products. The nonvoters in a paired preference test pose several questions: Did they really have no preference? Didn't they really test the prod- ucts? Did they refuse to commit themselves ? Who are these people? What level of nonvoters is negligible and at what level do they become important in interpreting the test findings? Why did 20% or more of these consumer testers have no preference: Consumer-test operators and users of consumer-test data must never forget these fundamental questions of methodology. If they do, the status of product testing through consumer panels will remain as it is now: an approach of unknown reliability for decision making. Of the nonvoters in this study, more children than adults declared no preference and children under six years of age were found to be mainly responsible for the children's relatively high no-preference percentage (2). Checking these nonvoters' response sheets showed that the children gave no information and the adults found no difference between the toothpastes. A slightly higher percentage of nonvoters was found in the national panel: 26.1% national 20.8% local. In fact, the percent- age of national nonvoters was a modicum higher than the national pre- ferrers of Toothpaste W (25.3%). Thus far, the local panel had fewer code-confusers and fewer non- voters. In other words, more persons (by per cent) in the local panel provided information. The remainder of this vertical analysis examines the quantity and quality of the information given by preference voters.

COMPARISON OF CONSUMER PANELS IN PAIRED PREFERENCE TEST. II 385 Key Word Categories Past experience with consumer responses to written or oral open-end questions about foods and beverages demonstrated that the various terms consumers use fall into fewer than ten categories. In the case of carbonated beverages, mention will be made of the amount of carbona- tion, the level of sweetness, the type of flavoring, color and sediment (if any), size of the container, and like or dislike (3). For dried soup, the mother will comment on ease of use, homemade flavor or richness, salt level, texture of solids, and generalities (4). The categories of com- ments about foods and beverages can usually be segregated as follows: general comments of comparison or like or preference nonflavor com- ments on use properties, texture, appearance and flavor comments on the type of flavor, the level of flavor, and specific flavor integers (e.g., off-taste, sweetness) or flavor impressions (e.g., filling, not satisfying). Similarly, consumer responses in this toothpaste test fell into one general, one nonflavor, and five flavor categories which are presented and defined in Table II. General comments are not descriptive of the products. Nonflavor comments refer to physical properties observed during use of the tooth- pastes. The flavor comments, it will be noted, show some overlap. For example, sensory impact (in No. 4) and hot factor (No. 5) could be the same integers aftertaste (No. 6) and clean-not clean (No. 7) cer- tainly are related. But in order to keep the data intact, for first-round interpretation at least, the coding procedure avoided overlaps. Tone of Key Word Comments In reading the response sheets from both panels it became evident that the R-preferrers saw the issues between the two toothpastes differ- ently from the W-preferrers, as is often the case in paired testing. Table II Key Words for Terms Used by the Consumers 1. General (nondescriptives) 2. Non-flavor 3. Flavor type 4. Flavor level 5. Hot factor 6. Aftertaste 7. Clean or not clean feel in mouth Like, prefer, better than... Whitening, sudsing, texture Kind of flavoring (mint, candy) Flavor impression (antiseptic, odd) Distinctness of flavoring Sensory impact Bite, burn, peppery After brushing tastes, feelings, and duration of same Impressions during or after brushing