THE ROLE OF CONSUMER STUDIES IN RESEARCH 45 Care is needed in analysing this type of question. In order to obtain one number representing the rating of the product by the panel, a weighting system can be used. For example "I liked it very much" could be given a weighting of 3, "I liked it moderately" a weighting of 2 and so on down to "I disliked it very much" with a weighting of -3. The total obtained in this way is divided by the number of panelists to give a mean score. This weighting system, from +3 down to --3 means that a product which was neither liked or disliked would havre a suitable neutral mean score of 0, one that was liked would have a positive mean score and one that was disliked would have a negative mean score. However, this weighting makes assump- tions about the relative value of the answers and also assumes that the answers cover the range of possible answers in equidistant steps. This is not necessarily true. Free answer questions are very useful for obtaining opinions where all the alternatives are not known in advance or when stating them might prompt the answers. Having asked whether or not panelists have a prefe- rence between two products, those who express a preference can be asked their reasons in a free answer question. The answers indicate the aspects of the product which the panelist considers important and this is useful in preliminary tests. Once the important aspects are clear it is possible to ask prompted attribute questions. These are easier to analyse than free answer questions which are very difficult to analyse. This is because two panelists may use different words but intend them to describe the same effect while two other panelists may use the same words but intend them to mean something different. Therefore great care is necessary in interpreting free answers. When there is doubt about whether two expressions mean the same thing, both can be listed. When the same expressions are used to mean different things this is sometimes apparent from the general context of the questionnaires but can be missed. Questionnaires may either be completed by judges, when they are used in preliminary tests, by an interviewer after she has interviewed the panelist or by the panelist herself. For early subjective panel tests when little is known of the type of response expected and extensive use is made of free answer questions or when a very long questionnaire is involved it is probably better to use an interviewer. She can probe for information without prompting answers and she can retain interest through a long questionnaire. Where a simple, short questionnaire can adequately cover the information required, it may be sent to panelists to complete themselves. Since the questionnaire is so important it is often worth testing it out

46 JOURNAL OF THE SOCIETY OF COSMETIC CHEMISTS in a pilot trial before using it in an actual test. Alternatively, two slightly different questionnaires could be used in a test, one for each half of the panel. In the example quoted for a multiple choice question above, the effect of reversing the order and placing "I disliked it very much" first could be tested. The effect of an overall preference question on subsequent prompted attribute questions could be tested by omitting the overall preference question on half the questionnaires. The actual order of questions on a questionnaire may be important: questions at the end being answered less accurately due to questionnaire fatigue or more accurately as a result of practice gained in answering earlier questions. In some cases there is a logical sequence of questions which follow the use sequence of the product. It would be more logical in a washing product test to ask about the washing process before asking about the whiteness of the dry clothes, etc. Upsetting the logical order may dis- concert panelists but might stimulate more thought. Unfortunately there are no hard and fast rules for questionnaires and often the shortcomings only come to light when the results are being examined. However, Payne (3) gives a lot of useful pointers for question wording. ANALYSIS OF RESULTS It is usually the case that a higher percentage of a panel will prefer one product to the other or that one product will obtain a higher mean score than the other. The question then arises as to whether the difference in preference or mean score is one that might occur frequently by chance, that is as a result of sampling variation, etc. If not then one might infer that the difference is due to a difference between the products. The usual test is to compare the difference with the standard error of the difference and consult t tables to establish significance levels. For a series of values x •, x 2, ß ß ß xn the standard deviation i=•l n--1 where Y. xi--• stands for the sum of the squares of the deviation i•l of each observed value of x from the mean ([) of the series of values and n is the number of values of x.