Selection of medical students

Dear Sir, Edouard and Harris’s paper, ‘On the pattern responses to multiple choice questions’ on which you asked me to comment (this Issue), suggests that students find greater difficulty in answering questions when the correct answer is false than when it is true. May I offer three reasons why this should not necessarily be accepted as true? (a) They offer no evidence that the false questions were of the same intrinsic level of difficulty as the true ones. This would need a mirror image paper in which all the trues were converted to false and the fakes to true. (b) A relative count of a random sample of our own one-from-five questions showed no difference in difficulty between those with positive stems (Which one of the foltowing . . ?) and those with negative stems (Which one of the following is NOT . . . ?)-not, admittedly, a wholly valid comparison. (c) An alternative explanation of their findings is at least possible. I showed a long time ago in this journal (Lennox, 1967) that it is dangerous to have a non-random distribution of trues and fakes. If a candidate knows that there is an excess of trues, he can gain marks by scoring as trues all the questions whose answers he does not know. This paper included 180 trues and only 120 fakes. If the existence of such a decided bias was known to or suspected by many of the candidates, or if they had simply come to regard it as normal, they would be much more willing to score a question as true when doubtful about it. Only if one can be absolutely sure that the students who sat this examination had no possible means of suspecting this very large bias towards true answers would it be safe to accept the Middlesex results. BERNARD LENNOX Department of Pathology, Western Infirmary, Glasgow GI1 6NT.


MCQ responses
Dear Sir, Edouard and Harris's paper, 'On the pattern responses to multiple choice questions' on which you asked me to comment (this Issue), suggests that students find greater difficulty in answering questions when the correct answer is false than when it is true. May I offer three reasons why this should not necessarily be accepted as true?
(a) They offer no evidence that the false questions were of the same intrinsic level of difficulty as the true ones. This would need a mirror image paper in which all the trues were converted to false and the fakes to true.
(b) A relative count of a random sample of our own one-from-five questions showed no difference in difficulty between those with positive stems (Which one of the foltowing . . ?) and those with negative stems (Which one of the following is NOT . . . ?)-not , admittedly, a wholly valid comparison. (c) An alternative explanation of their findings is at least possible. I showed a long time ago in this journal (Lennox, 1967) that it is dangerous to have a non-random distribution of trues and fakes. If a candidate knows that there is an excess of trues, he can gain marks by scoring as trues all the questions whose answers he does not know. This paper included 180 trues and only 120 fakes. If the existence of such a decided bias was known to or suspected by many of the candidates, or if they had simply come to regard it as normal, they would be much more willing to score a question as true when doubtful about it. Only if one can be absolutely sure that the students who sat this examination had no possible means of suspecting this very large bias towards true answers would it be safe to accept the Middlesex results.

Selection of medical students
Dear Sir, I have read with interest the report of the General Medical Council Conference on the Selection of Medical Students in the September 1979 issue of Medical Education. I perceive that the problems associated with the assessment of personal qualities exist much the same in the U.K. as here in North America. We do however differ in the age of selection as we require previous university experience before entering medicine.
Dr W. D. Wylie, Dean of St. Thomas' Hospital Medical School, London, was quoted as saying that McMaster University screens personal qualities by inviting students to write a letter, but that we had not yet reported our results.
While it is true that we use the biographical letter as a screen, it is only one of several tools we use to assess the personal qualities of our applicants. It is our opinion that if we take the time and effort to select appropriately mature, motivated, considerate and obviously bright individuals (bright is not defined as merely achieving high academic standards) we will be able to provide them with an appropriate educational experience for them to become good sound physicians, capable of entering all facets of medicine.
At all stages of the assessment, we make use of medical teachers, students in our programme (the future profession), members of the community of physicians (trained in a variety of styles), and members of the lay community (the consumers of medical practice). Each expresses an opinion of the candidate and all opinions are viewed equally when making our final selection.
We have over 2500 applicants for one hundred places at our school. The assessment of the biographical letter and academic performance are combined to give a single score by which we rank our applicants to choose 440 to be brought to an interview. This combination allows a diversity of candidates to reach the next stage.
The interview consists of a group exercise or tutorial and a personal interview. Each is assessed and commented on by the various constituents discussed previously. In addition, the candidate solicits three personal references which are also taken into account. The final review makes use of many opinions recorded about the candidate to assess the overall acceptability to enter medicine at McMaster. Though the process is involved, we are pleased with the results. More extensive descriptions of the process have been reported (Hamilton, 1976;Ferrier, McAuley & Roberts, 1978). The involvement of the community, both professional and lay, in this emotional and provocative topic has served as a tremendous public relations exercise resulting in a diminution of the 'ivory tower' concept of university and a feeling of involvement in the future development of the profession.

Problem-solving learning
Dear Sir, In the November issue of Medical Education you mentioned that two medical schools, McMaster in Canada and Newcastle in Australia, have given a central place to problem-solving learning.
We have had a problem oriented educational system, which has now been functioning for almost 6 years, in the medical faculty at Maastricht of the Rijksuniversiteit Limburg.

Chairman Educational Development and Research, Rijksuniversiteit Limburg, Maastricht
Editor's note: The final day of the 1980 AMEE Conference, 5 September, will be a visit to Maastricht, when the system referred to above will be demonstrated.

Dear Sir
Your editorial on 'Problem-solving' clearly states the logic behind the use of this teaching-learning method to prepare medical students for their future tasks. Your review is also quite sensitive to the problems inherent in undertaking such a teaching approach and the demands it makes. Our experience has shown that many teachers in various medical schools are eager to apply this approach once its logic has become apparent to them. Others, of course, wish for documentation of success. Those of us who have taught in both systems become impressed by the effect of problem based learning on students. They enjoy learning, they remain motivated, they study areas of medicine to depths you wouldn't dare ask for in subject based curricula, and they seem to do very well. Neverthelejs, this is not enough and is subjective or anecdotal at best.
There are some studies that provide supportive evidence for the success of problem based learning but lack the scientific rigour many would like to see (Barrows & Mitchell, 1975;Barrows & Tamblyn, 1976;Schmidt, 1965). There are also those that clearly show the disadvantages of the conventional curriculum (Levine & Forman, 1973).
This last statement raises an issue that must be recognized. The conventional curricula employed in most medical schools have not been properly evaluated in light of the doctor's future tasks. The successes of schools in North America are largely measured by the factual recall of their students on objective, multiple choice examinations, not by their graduates' ability to care effectively and efficiently for a standardized patient problem presented in a manner that challenges their reasoning and decision-making skills. Those that have done this often find these essential skills lacking. This point has certainly been made before in your journal (Miller, 1978). Although I heartily agree that those who use problem based learning must carefully evaluate its effectiveness, the same must be required of all medical schools.
It is safe to say that such an evaluation has been a major preoccupation and activity at McMaster and more data will be available about how students in a problem based curriculum perform after graduation (Woodward and Neufeld, 1980). Nevertheless, we do not have the comparison between two treatments that is needed. Although we have proposed a blinded third party evaluation of our graduating students and those of several schools with more