By Author
  By Title
  By Keywords

May 2005, Volume 55, Issue 5

Review Articles

High Failure rate in Postgraduate Medical Examinations - Sign of a widespread Disease?

Saeed Farooq  ( Department of Psychiatry, Postgraduate Medical Institute, Lady Reading Hospital, Peshawar )

A young student pursuing a career in medicine can look forward to ten to fifteen years of examinations starting from the first year of medical school to the postgraduate qualifications. The worst scene is in the postgraduate medical education, where a doctor is not a student in a formal sense but has a career of one examination after another to face. Passing a membership or fellowship examination of various colleges, is a formidable task. The high failure rates in the fellowship examinations of the College of Physicians and Surgeons (F.C.P.S.) has become proverbial. This high failure rate is not unique to F.C.P.S., but is considered as a hallmark of all the 'prestigious' examinations mostly a legacy of British medical education system. 1 Weatheral 2 , while criticizing the whole system of examinations in British Medical Education portrays it rather contemptuously as:

"A trip to the Regent Park zoo offers the visitors a remarkable panorama of biological diversity. Yet just a short walk away the scene changes dramatically: there is virtually an identical failure rate for the examination for Membership of the Royal College of Physicians from year to year; surely one of the most remarkable examples of homogeneity of behavioral patterns in living organism."

The situation is particularly alarming for doctors from our country. In F.C.P.S., the failure rates are generally higher than the comparable contemporary examinations of the Royal Colleges. Even in the Royal college's examinations in UK, the candidates from Asian countries generally have higher failure rates when compared with the native doctors. While it is not possible for us to discuss the reasons for higher failure rates for another Asian countries, we must ponder about the high failure rates for our candidates.

In this article an analysis of the factors contributing to this high failure will be conducted in the light of author's personal experience as a trainer and examiner in postgraduate examinations as well as literature on the subject.

High Failure rate = High Standards

Before we discuss the reasons for high failure rate, it would be pertinent to address a commonly held misconception i.e., the examinations of high standard and traditions must have high failure rates. The medical educationists are seriously challenging this notion on several grounds.

Most important reason, however is that when an examination becomes an end in itself, the teaching and learning are distorted. 3 The powerful steering effect of examinations means that the predicted examination content is eagerly studies to exclusion of all the rate. This encourages a hurdle jumping approach to learning. T.H. Huxley eloquently elaborates this many decades ago as:

"Experienced friends of mine do not hesitate to say that student whose career they watch appear to them to become deteriorated by the constant effort to pass this or that examination. They work to pass, not to know and outraged science takes her revenge. They do pass and they don't know."4

This may be a personal view but is being taken seriously by the educational experts and the institutions concerned with medial education. Recently, the UK General Medical Council has introduced radical reforms in British medical education. In its historical document "Tomorrow's Doctor", recommendations on the future pattern of undergraduate medical education would be "wholly frustrated if the present examination system were to continue." 5

It is also important to consider the functions of the examinations before discussing their results. Examinations serve many functions, quite a few of which may not be obvious even to those who plan examination. These functions have been summarized by McManus 1 as following:

- To satisfy the public that its practitioners have a minimum level of competence - -the licensing function.
- To satisfy the universities that its graduates are academically fit to carry on their scholarly pursuits --- the graduating function.
- To discriminate between the best, good and poor --- the ranking function.
- To encourage the students to work --- the motivating function.
- To assist the students in putting together the materials that have been acquired separately - the integrative function.
- To provide feedback to the students and the staff on the performance of students and teachers - the informative function.

In his analysis of Brtish examination system Mc Manus suggested that one of the reasons for high failure rate in these examinations is that one exam is trying to satisfy all these functions. How far this applies to our system can be an important point for debate.

The Examinations

The jerk reaction to any poor results in examination is to blame the examination or assessment method itself.

Commonly, the high failure rates in examinations are attributed to the relative lack of objectivity of assessment methods. 6 Each assessment method has its own problems. The traditional long and short cases have been criticized most. 6-8 Similarly the viva voce examinations have been the subject of critism. 9 Long standing prejudices about there being only one way to elicit a sign, or its relative importance and individual quirks of examiners are just few of the reasons which can contribute to the high failure rates in examinations. 10,11 It has long been know that methods such as essay or the unstructured oral and clinical cases can give the misleading impression of students real abilities. These problems have often resulted in calls for improving our methods of assessment. 2,12 The examining bodies through out the world have now adopted better reliable and objective assessment methods. In Pakistan, for example College of Physicians and Surgeons has adopted more objective assessment methods such as M.C.Q.s, Task Oriented Assessment of Clinical Skills (TOACS) and structured viva. 13 These undoubtedly have resulted in improvement in results mainly by eliminating the examiners bias. However, the failure rate still remains unacceptably high, generally more than sixty percent in the final exam. It is therefore time to look beyond examinations for this high failure rate.

It has to be realized that the examinations are the end stage of a long process and results of an examination are the end product. The factors affecting whole process are often overlooked when we are discussing the outcome of process in dichotomous terms, i.e., pass or fail. A multitude of factors can affect the results starting from the education and training in medical college to the training received as an intern. It is not possible to discuss all of these factors here and we will focus only on three key factors. These are:

1. Health infrastructure of the country
2. Training in the medical colleges
3. The postgraduate training

Health Infrastructure of the Country

The health infrastructure of the country provides the foundation for medical education and training. It is surprising that we have almost completely overlooked the importance of this factor. Godfrey 10 described the crucial role of health infrastructure in simple, yet very comprehensive terms:

Quality of graduate = quality of health care infrastructure

(Note that it is the quality of the whole health infrastructure not specifically of training institutes or teaching hospital). The importance of this equation cannot be over emphasized but is generally seriously under estimated in the developing countries. While working in present health infrastructure, our trainees are caught in a paradox during most of the training. The standard clinical practice taught in their teaching does not seem to exist in real practice. Their routine clinical work is often determined by the enormous work load, scarce resources, limited facilities for investigations and treatment, individual quirks of the consultant, un-reasonable patients expectations based on very little health education and grossly uneven distribution of resources toward the rich. The enormous burden of disease means that the trainee (and the trainer) has always a demand on them to act; rather than to think and act. This with continuous unsupervised practice becomes a reflex action. When there is little time to think, ability to think critically about diagnosis and management is very difficult to develop. This ability forms the basis of clinical problem solving or clinical reasoning ability.14 Training in such a system fosters the inappropriate clinical skill so often observed by the examiners in clinical examinations. Thus we should not be surprised that the worst part of examination results is that of clinical component --- an essential component to pass the whole exam.

The ideal setting for training rarely exists even in developed countries and is least likely to exist in a developing country like Pakistan. It can however, be argued that the essential skills required for clinical reasoning can be developed in a setting which is geared towards proper training, based on clinical reasoning and critical thinking.

Postgraduate Training

The multiple problems which plague our training system are well known and have comprehensively been summarized elsewhere. 15 Here the focus will be only on the basic issues, which have often been overlooked.

The underlying problem in postgraduate training is that most of the trainers do not appreciate the difference between undergraduate and postgraduate education. This well described by Muzafar 16 as following:

Undergraduate education evolves around the presentation and assimilation of factual knowledge. Postgraduate education, on the other hand, would presume that this body of knowledge exists and use it as a basis for enunciating the principles of diagnosis and management of a case. This, of necessity, would entail bedside examination, review of laboratory data and further investigative work possible only in a hospital setting. As such it requires a much smaller number of participants in the decision making dialogue. The corollary becomes even more vivid in the surgical setting. The steps of a tracheotomy can be demonstrated to 200 students in a lecture hall but can be successfully taught to only one student at a time in the operation theatre and may successfully be learnt only after several repetitive exercises. The first difference between the undergraduates and postgraduate teaching is the student teacher ratio. In the undergraduate class this might be 100:1, in the postgraduate setting it is best at 1:1 or at the most 3:1."

A corollary to above is that learning requires active participation by the trainee. The major role of a teacher is to create an environment conductive to learning. However, because of deeply ingrained expectations in minds of trainees as a consequence of learning under teacher oriented and teacher directed methods in the undergraduate education, trainees become highly dependent on the trainers. 16 The trainer, unfortunately mostly tend to foster these expectations but are not able to satisfy these demands due to time constraints and multiple demands of his job. This result in a frustrated trainee and an unsatisfied trainer, each blaming the other for high failure rates - a common scene in the postgraduate training.

It is now time to accept that our selection criteria for teachers, especially those who are going to train postgraduate trainees is long outdated. Research and postgraduate training go hand in hand. It is paradoxical that at the entry point to the teaching cadre i.e., Senior Registrar and Assistant Professor, neither research experience nor any experience in medical education is amongst the selection criteria. Although the situation has started showing some improvement but the overall selection criteria remains inadequate. The practice is perhaps unique to our country. The experience of research and the publication are highly valued in selection even for the Senior Registrar (now called Specialist Registrar in U.K., where it is still considered a training job unlike ours), not to speak of higher academic grades. Even the requirement for research paper needed for promotion in the teaching cadre has been compromised by many factors. It is ironic, for example that the F.C.P.S. trainees can publish two papers in lieu of dissertation only in a journal indexed in Index Medicus, while the papers for the promotion of their teachers can be published in any of the 44 journals rcognized by P.M.D.C. The quality of peer review and scientific rigor required for publication in a number of these journals leaves much to be desired.

These factors along with inappropriate teaching methodologies, lack of structured training programmes, no evaluation of either trainee or the trainer lead to the high failure rates.

Undergraduate Medical Education

This is perhaps amongst the least considered factor in discussion on postgraduate medical education. It can be argued that the postgraduate examinations can serve as a measure of predictive validity for the undergraduate examinations to a certain extent. The importance of this can be gauged from the following example:

General Medical Council UK has the power to examine the British medical school and qualifying examinations. This power was never used for many decades and fell into "disuse" according to Smith. 17 However a round of inspections was started for the first time in 1982, only when the Royal College of Physicians complained that some British candidates for the Membership examinations seemed to have received an inadequate basic medical education. It will indeed be interesting to examine our basic medical education in the light our postgraduate exam results.

The number of problems in undergraduate medical education in Pakistan is discussed in extensive literature on the subject. These have been summarized by Naqvi 18 and can briefly be enumerated as follows:

1. Lack of a well-defined curriculum based on the health needs of local population.
2. Selection of medical students purely on the basis of academic criteria without testing the abilities and aptitude for the profession.
3. Teaching which is mostly based on theoretical knowledge and is geared only towards passing the examinations.
4. Assessment methods, which mostly test factual recall. The students are then bewildered in postgraduate examinations, which mostly test problem solving.
5. Lack of teaching expertise in medical teachers who are selected solely on the basis of postgraduate qualification in the subject with very little or no regard for the teaching experience or aptitude. This further compounded by lack of any incentive for teaching, inappropriate student-patient and student-teacher ratio ad no concept of teacher evaluation.

The initial years in medical college where basic medical sciences are taught can be considered as nursery years for medical education, particularly for postgraduates. This unfortunately remains the most neglected area in whole of the medical education. If we nourish and nurture our future postgraduates in these nurseries, we should not be surprised of the results in our postgraduate examinations.

What is the way forward?

It can be concluded from the above discussion that the high failure rate of our docors in postgraduate examinations is a sign of a widespread disease afflicting the whole system of medical education and the health infrastructure. Unfortunately, majority of the factors contributing to high failure rate in F.C.P.S., are beyond the remit of the College. The college however will have to bear the brunt of this high failure rate, as the examinees perceive the college as a gatekeeper at the end of a very long corridor with too many obstacles.

It is not appropriate to suggest specific measures in this brief article. The implementation of clinical protocols for the common conditions encountered in our health system can go a long way in improving both services and the training.

It is clear, however, that this demands an integrated effort on many fronts. The improvement in health infrastructure is of particular importance. Various measures to improve the objectivity and reliability of examinations have improved the results but overall situations still remains dismal mainly due to fact that the health infrastructure is not conducive to the quality training.This has serious implications for those concerned with academics and teaching in this country. They will have to look beyond the curricula's, training programmes and examinations in order to improve the output.

References

1. Mc Manus IC. Examining the educated and trained. Lancet 1995;345:1151-3.

2. Weatherall DJ. Designing a doctor: examining undergraduate examinations. Lancet 1991;338:37-9.

3. Godfrely RC. Undergraduate examinations: a continuing tyranny. Lancet 1995;345:765-7.

4. Huxley TH. Aphorisms and reflections. London: Watts and Co. 1911. In: Mc Maus IC examining the educated and trained. Lancet 1996;345:1151-3.

5. General Medical Council. Tomorrow's doctor: recommendation on undergraduate medical education. London GMC 1993.

6. Even LR, Ingersoll RW, Smith EJ. The reliability, validity and taxonomic structure of the oral examination. J Med Edu 1996;41:652-7.

7. Williams RG, Barrows HS, Vu NV, Verhulst SJ, Colliver JA, Marcy M, et al. Direct standardized assessment of clinical competence. Med Edu 1987;21:482-9.

8. Maguire P. Assessing clinical competence: need for improvement. BMJ 1989;298:4-5.

9. Jafarey NA. Viva voce examination (editorial). J Coll Physicians Surg Pak 1993;4:72.

10. Godfrey R. Designing a doctor - all change? Lancet 1991;338:297-9.

11. Manzar S. The evolution of objectivity in CPSP examination. J Coll Physicians Surg Pak 1998;8:243-4.

12. Fazal E. Assessing clinical competence in postgraduate examinations. J Coll Physicians Surg Pak 1992;3:92.

13. Khan AUN. Postgraduate examinations in Pakistan (editorial). J Coll Physicians Surg Pak 1996;6:131-2.

14. Newble DI, Entwhistle NJ. Learning styles and approaches implications for medical education. Med Edu 1986;20:162-75.

15. Ali SM, Ali SK. Structured training programme. In: Postgraduate medical training; proceedings of moot on postgraduate training. College of Physicians and Surgeons Pakistan. Karachi: 1999, pp. 60-61.

16. Muzzafar K. Postgraduate medical education in Pakistan. J Pak Med Inst Med Sci 1991;2:130-3.

17. Smith R. Medical education and the G.M.C.: controlled or stifled? BMJ 1989;298:1372-5.

18. Naqvi AS. Problems of medical education in Pakistan. J Pak Med Assoc 1987;47:267-9.

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: