Advertising
Polls apart
fhilip Kleinman
The Market Research Society is holding its seventeenth annual conference at Bournemouth this week (March 13-15), and the eggheads of the Persuasion Industry will be listening to learned papers from each other on such topics as "The effect of order in the presentation of samples in paired comparison product tests." (For those unfamiliar with market research gobbledygook, this is all about the problem of housewives who, when invited to say which of two canned soups tastes better, are liable to plump for whichever they are given a spoonful of first.) But I guess that the main topic of conversation among the delegates is likely to be one which is not on the conference agenda, and that is how much harm have the political opinion polls done their business? And harm means harm. One research company I am acquainted with was rung up the day after the election by a prospective client and told that if the election forecasts were anything to go by he thought he would be better off without the company's services after all. This particular firm's chagrin was only'increased by the fact that it does not indulge in political opinion polls, and indeed takes a very dim view of them.
The pollsters' defence, as echoed by Market Research Society chairman Gerald Goodhardt, is that most of the polls in fact got their predictions right — or, rather, within the normal, stated margin of error — as far as votes were concerned and that they are not to blame if the vagaries of the British electoral system turn a small Conservative lead in votes into a small Labour lead in Parliamentary seats. The defence further runs that the research firms which carry out the polls are not responsible for the headlines in the newspaperswhich publish them.
The defence is slightly disingenuous, since the pollsters know very well before they start that what newspapers — and indeed their readers — are interested in is a straightforward political racing tip and not an academic discussion on the complications of opinion sampling. If, knowing the 'abuses' which their work is likely to suffer, they still go ahead and risk the misleading headlines, it is because they are as much guided by an ulterior motive as any politically biased editor.
Market research consultancies, like the advertising agencies for which so much of their commerical work is done, are, in short, in business to get business. Jeremy Fowler, head of Business Decisions, the firm which got nearest this time to guessing the election outcome correctly, puts it bluntly: "From this company's .point of view it's prestige and publicity. Who'd heard of us before we did the Observer poll?"
Nor is it by any means to be supposed that it is only in the area of voting intentions that the researchers' work arouses doubts and scepticism. Despite all the statistical jargon with which market research firms pepper their reports, their statistics are not always based on rock-hard foundations. Extremely suggestive in this respect is a paper due to be presented to the MRS conference by P. L. Atkinson, of the British Market Research Bureau, and A. J. Ogden, of Shell-Mex and BP. The paper refers to a study in which people were interviewed in garage forecourts about the brand of petrol they bought.
The interviewers had the unusual advantage of being able to check the responses against the respondents' observed behaviour just before being interviewed, and the following remarkable results emerged: 15 per cent were unable to remember that brand of petrol they had bought a few moments before, and 7 per cent remembered it wrongly; in the case of one brand of petrol three in five people who had bought it were unable to recall its name. Atkinson and Ogden point out that if consumers' memories are so faulty about petrol, they are liable to be equally faulty about other purchases for which their word has to be taken.
The saving grace of market research is, however, as this example shows, that researchers are their own sternest critics — or at least critics of each other. Which other body than the MRS would have arranged for its conference to kick off with a keynote speech — to be given by Winston Fletcher, managing director of MCR Advertising — which will accuse its members of, in effect, gaining money by false pretences?
Perhaps that is too strong. What Fletcher actually intends to say is that many housewife samples used by market research companies are too small to be commercially reliable. His solution is that the researchers should use bigger samples and charge more for their work. It's a point which advertisers, from whom most of the money comes, might also like to ponder.
Previous page
