THE EXAMINERS EXAMINED
[To the Editor of THE SPECTATOR.] Sin,—I have not yet had an opportunity of reading the pamphlet An Examination of Examinations, reviewed by Mr. Verschoyle in your issue of January 3rd, though I hope to do so as soon as possible ; so that I cannot say whether the following remarks are applicable to it or to your reviewer. But in any case, many thousands will See The Spectator article who have no intention of consulting its original, and I should like if possible to do something to check the flood of misapprehension which so easily arises when a semi- • technical question is also one of extreme public interest.
In general my criticism is that the statistics cited by Mr. Verschoyle for the most part either prove nothing against the examination system or prove . defects that have long ago been remedied by experience. The system is weak enough but not in these ways.
• The following points are put as briefly as is consistent with clearness, and might easily be enlarged.
(1) In every case (except that of the viva, which is of course on a different footing) the verdict criticised is that of a single examiner. As far as I know, no important examina- tion is conducted by a single person without a colleague or reviser to cheek his idiosyncrasies. The necessity for such a check needs no statistical proof.
(2) An examiner, like any person concerned with measure- ment, must have some standard footrule on which to base his markings—in his case a paper or set of papers of which he knows the value in terms of marks. Marking, in so far as it sets a standard, is in fact arbitrary, Way, not 4rticrei, but once the conventional standard is established the examiner's business is to match the papers he is marking against the scale which he has established. The standard may be worked out alone or in collaboration with colleagues, but is in either case of fundamental importance. Now, 'Mr. Verschoyle fails altogether to raise this point, and as a result many *of the figures he gives are quite meaningless. For instance, if the investigators did not in fact provide the examiners with any standard, it is hardly surprising that some took fl as the lowest available mark, and some as the highest, and marked the rest of their scripts to corre- spond ; nor is it surprising that. an examiner should after . the interval of a year forget what precise standard he, had previously applied, and should vary son-IC*11A from his former marking. The. point which. needs attention Is- not variation. in the marks of individuals, but variations in their _relative placing. in the order of the groajp, of examinees ; riot variations in the standard, but bid application of 'the standard, which is probably less frequent but certainly kss
easy to regulate. . " • • . .
(3) An. English Essay is a notoriously hard paper to niark —who, for instance to take the last number of The Spectator) would care to put a relative value on the compOSitions of
Sir Herbert Samuel and Miss Rose - ,,, Macaulay ? has some • • • • value in scholarship exams., giing more . chance to
individuality than any " knowledge " paper,. but is otherwise almost useless, so that variation in marking here means
very little. The legendary don who marked one of these creations as aley ?+ was perplexed, but not silly.
(4) The account of the viva voce is very strange—surely even Civil Service Commissioners do not make awards on the unsupported testimony of a viva.. Indeed it must be practically impossible to conduct one without some previous knowledge of the candidate, from written work or otherwise.
(5) And what is the remark about. the "possible range of 24 grades " in " the Oxford system of literal marking " ? I had always thought it the great virtue of the system to have plenty of room at the bottom. 8 and c are in fairly regular use, and the lower reaches round about 4,x1Pro are not entirely unvisited, even by Oxford.
I hesitate to make any general criticism of the investigation on the basis merely of Mr. Versehoyle's report ; but his report does suggest that there has been the expensive and elaborate futility of careful research into a system which does not and could n-+ exist. Nothing in the review suggests reality, except the fact that the examiners were paid " at the usual rates," to produce the right atmosphere—an odd and perhaps unduly cynical view of the examiner's psychology.
[Mr. Verschoyle writes : " If Mr. Mackenzie will read An Examination of Examinations he will find all the questions he lists systematically discussed. If he will re-read my review of it, he will find that I did not neglect the point which he says I failed to raise, and that he has ascribed to me views which I did not express and assumptions which I did not make."]
Previous page
