DECISION ANALYSIS 17 9 preferences of these experts have been recorded along with the actual group choice Xs, --or was to be recorded over a period of time, information of the following nature would be available. Number of occasions Number of occasions a(O and X• agreed a(i) and X• disagreed Result of tender a(t) S S x unknown X• 7 18 0 a(x) 5 6 14 a(a) 6 7 11 a (a) 4 1 20 This table indicates the number of times each individual agreed with a selection which was subsequently found to be either successful or unsuccess- ful, or to have disagreed with a selection decision which was unsuccessful. Accordingly it is possible to associate a prediction measure to each of the individuals and so identify, on historic grounds, the 'best' predictor. One possible such measure is to give, say, one mark to each correct prediction, 0 to a known incorrect prediction and perhaps 1In (where n is the number of competing firms) in the case of an unknown result. Thus, if n -- 4 we have for a preference measure Xa=7 a (O = 8.1/2 a % = 8.3/4 a (a) = 9. In this particular instance, the current practice of using X• could be super- seded by any a (ø with an anticipated increase in successful tenders. The best apparent predictor is, of course, a (a), or in this case the production manager. With a © the prior probability measure of success is 9/25 = 0.36 as opposed to 0.28 with ,Y•.. Clearly such a prediction measure could and should be updated as results of each tender become available. Care must be taken when deciding upon a 'measure' of prediction performance. Above we used the principle of Insufficient Reason to assign equal probabilities to a set of events of unknown probability. Equally well one may have reasoned as follows: the production manager has made 25 selections, five of which we know split into four successful selections and

180 JOURNAL OF THE SOCIETY OF COSMETIC CHEMISTS one unsuccessful. It might be assumed, therefore, the unknown results will split in the same ratio given a total of 20 successful selections and five un- successful selections. Here there is a danger of putting too much trust in too little information. If, for instance, another party, the Managing Director, say, h•d only agreed with X• once, and on this occasion the selection was successful in gaining the contract, his expected prediction rating would be top at 25, or 100%. Panel test used If a panel test is used, the perfumer has the task of selecting a welfare function W. There are two immediate approaches to this problem. (a) Knowing the preference orders of panel members over past sets of decisions, it is possible to experiment with various welfare ftmctions over this data and perform an analysis similar to the above, where individual preferences a (ø are now replaced by welfare function selection W (i). That welfare function W (ø leading to the best ex- pected number of successes is then selected. Of course, this analysis may be complicated by different costs associated with evaluating the various welfare function. The analysis of non-panel decision would presumably be incorporated here by defining W ø) = a% W(•)= a © etc., that is the relative effectiveness of the simplest welfare function, an individual's preference, and more sophisticated function W ©, i3, would be measured. (b) It might be possible, knowing the soap manufacturer's problem, to predict to some degree the welfare function, or likely welfare function, W* of the manufacturer. Suppose the perfumer thinks welfare function Wx, W•,... Wn are likely candidates, and also in his opinion the probability of Wx being used is px and in general of W• being used is pi etc... i = 1, -, n. If there are three possible tender samples to choose between Sx, S•, using function W• the choice is Si*= W•(Sj), i = 1, -, n. We form a weighted sum of the number of times Si is selected as S*, namely N(S) = Zpj, where./is such that Sj* = S•. J The selection is then made on the basis of selecting Si to maximize N(S•). An example will make this procedure clear. EX. Wx(S•, S•, Sa) = Sx - Sx* /191 = 0.2 p=0.2 w3(&, $3, $3) = & = $3* p3 = 0.3