By Author
  By Title
  By Keywords

June 1999, Volume 49, Issue 6

Original Article

Results of Faculty Evaluation at The Aga Khan University, Karachi, Pakistan

Rashida Ahmed  ( Departments of Pathology, The Aga Khan University, Karachi. )
Zoon Naqvi  ( Educational Development, The Aga Khan University, Karachi. )
Mohammad Khurshid  ( Departments of Pathology, The Aga Khan University, Karachi. )

Abstract

Objective: The aim of the retrospective correlational analysis was to identify the attributes valued most by students for assessing the overall effectiveness of a teacher.
Methods: Responses of the students to the two versions of evaluation questionnaires, each attempting to assess 4 and 8 characteristics respectively on a scale of 1-5 were included in the analysis. The third and fourth year students, at the end of each course/module completed a total of 2110 evaluation forms, which were studied.
Results: The over all effectiveness of the teacher showed statistically significant correlation of .914 and .895 with ability to communicate ideas effectively and clarity and organisation of the lectures. Whereas the knowledge of subject and the successful use of teaching aids showed a correlation of .658 and .637 with a statistical significance of P<.01.
Conclusion: Students need a basic outline of what they have to learn and guidance to plan their studies OPMA 49: 135, 1999).

Introduction

Evaluation of teaching effectiveness by the students through structured questionnaires is a common practice in the developed world1-4. Currently, several methods are being used to evaluate teaching methods and the extent of faculty, nobody is in a better position to comment the most frequently used methods for the purpose is student evaluation. It is advocated that since students are the recipients of the knowledge and skills delivered by the teaching than the recipient students4. A number of papers have appeared in the literature in relation to evaluation teaching in clinical setting, for nurses and doctors1,3,5, but evaluation of teaching and teachers in medical schools in Pakistan has received insufficient attention. The need to identify and define descriptors, which can gauge the effectiveness of the teacher as well as objectively define the expected outcomes for the educators of health professionals is now being considered extremely important to improve the standard of medical and nursing education nationally and internationally.
One of greatest concerns of the reasearchers and educators in terms of the utility of the evaluation forms completed by students is the validity of the proformas used for the purpose. Broadly speaking, validity of the evaluation forms and the criteria it aims to assess can be approached from two different viewpoints6. In one the evaluation and the ratings are valid if they accurately reflect student opinions about the quality of instruction, regardless of what is learnt. In the second view the evaluation is valid if it reflects instructional effectiveness.
The Aga Khan University (AKU) Medical College started functioning in 1983. Since the inception, faculty evaluation by the students at the end of course /module, has been a regular feature. Presently faculty evaluation is used for providing feedback to the faculty as a part of faculty development. In some clinical disciplines promotion is dependent on the performance as a teacher, and student evaluation contributes objectively to this.
Each discipline carries out its own evaluation by circulating the evaluation form at the end of module! course. The course/module comprises of lectures, tutorials, practical and laboratory sessions.
From 1993 to 1996 a structured evaluation form was being used for the purpose. In late 1996 it was felt that there was a need to review and revise the form and make it more efficient and unambiguous for the students. The data collected through the administration of the forms is kept confidential and serves as an effective tool for the self-development of faculty. However the usual concerns expressed by the faculty of AKU regarding the usefulness of the ratings obtained, are the consistency of students, age, maturity of the students, unreliability of the form and the concern that these can become a popularity contest etc are similar to as reported elsewhere8.
Six criteria identified by Jacobson7 are widely used by many workers for assessment of teachers. They are broadly classified into professional competence, teaching and evaluation practices, personal characteristics, interpersonal relationship and availability to the students. Keeping this in view the study aimed to identify the
attributes in a teacher in a classroom setting, which our students value the most while assessing the overall effectiveness.

Materials and Methods

This was a retrospective correlational analysis of the responses of the students to the evaluation questionnaire. The students evaluated 4 criteria in the old form (Appendix A), whereas 8 characteristics are evaluated through the new form (Appendix B). The students were asked to use a five point scale, I for poor, 2 for fair, 3 for good, 4 for very good and 5 for excellent, in both the forms.
Third and fourth year students, studying Pathology, from 1993-1997 were included in this study. The sample collected was the sample of convenience and the participation was purely voluntary. The questionnaire was administered at the end of course or module. Data from a total of 2110 Questionnaires (new forms = 747, old forms = 1363) was stored in Fox-Pro and subsequently analysed by using SPSS programme for Microsoft Windows by using Regression and a correlational analysis.
One of the limitations of evaluation using a questionnaire is an element of subjectivity and fear of consequences in the students. Although the element of subjectivity could not be eliminated, the fear of consequences was taken care of by keeping the identification optional and by not letting the individual faculty member look at the individual evaluation done by the students.
Whenever any statement was considered to be ambiguous or thought to be lacking in clarity a number of students in the campus were interviewed.

Results




Table I & II depict the mean scores obtained for each attributes in the two forms separately. The mean ratings in the new form are lower than those in the old form.
The over all effectiveness of the teacher showed statistically significant correlation of .914 and .895 with significance of P<.01 each, with ability to communicate ideas effectively and clarity and organisation of the lectures. Whereas the knowledge of subject and the successful use of teaching aids showed a correlation of .658 and .637 with a statistical significance of P<.01.
The results of the correlational analysis identified ability to communicate ideas effectively as the most important criterion with total correlation with an over all effectiveness of the teacher. The second most important criterion was the positive attitude towards teaching with significant P value (Table III).

The least important criteria were, use of audio-visual material and knowledge of the subject, with insignificant P values. Remaining criteria related to role modelling, provision of positive feed back, ability to answer the questions clearly, organisation of the sessions, ability to challenge student’s thinking were rated as less important as compared to the two top most characteristics.
The Old Form
The data collected from the old form showed that a teacher’s interest in teaching correlated most strongly with the overall effectiveness of the teacher, whereas the teacher’s knowledge about the subject depicted the weakest correlation.
The New Form
The results of the correlational analysis identified the ability to communicate ideas clearly as the attribute correlating maximally with the over all effectiveness of the teacher. The second most important criterion was the positive attitude towards teaching.
The smallest correlation of overall effectiveness was found with the use of audio-visual material.
Excellent: Faculty member consistently provides lectures, tutorials or clinical sessions regarded as enjoyable, helpful learning experiences. The student looks forward to his/her teaching session. Normally a small percent of the faculty would fall in this category.
Good: Faculty member provides well-organised learning opportunities from which the student benefits most of the time. Most faculties will fall in this category.


Poor: Faculty member consistently provides confusing or unhelpful teaching sessions from which the student learns very little. The student is temptedto skip his/her sessions. Only a small fraction of the faculty may be expected in this category.
Comments: The 5-point scale on the previous form has practically changed to a 6-point scale due to the introduction of ‘N.A.’. This may be the reason for the lower markings in the new form.


Interviews
The most important characteristics, which were identified, were the ability of the teacher to answer questions clearly and a positive attitude towards teaching.
Teacher who was considered to have a positive attitude towards teaching:
* ready to give extra time to the students beyond the designated time
* encourages questions even if they sound ‘dumb’
* makes sure that the students understand whatever is
being communicated to them
* is easily approachable
* is always willing to provide help to the students
The teacher who was judged as possessing
knowledge of the subject:
* frequently quotes from latest online information and literature
* goes into veryfine details of the subject presented.

Discussion

The difference of in the mean ratings of the new and old form could be due to the more objective modified structure and design of the new form, leading to reduction in the element of subjectivity. The new form has been designed in the form of behavioural descriptors, which are explanatory and probably easier to assess. Whereas the old form required the students to evaluate characteristics without specified behaviour making it ambiguous and probably difficult to assess. On the other hand familiarity of the students to the techniques could also have led to an objective assessment.
In literature the inter-correlation among the specific and global outcomes has varied and was often not highly positive7. Although literature helps us to identify characteristics of an ideal clinical teacher1,3,5,9,10,11, but the teaching in a classroom environment (as in pre-clinical years) has comparatively received less attention. The statistically significant strong correlation of all the characteristics with the overall effectiveness of the teacher in both the forms in the present study indicates that the forms though self constructed are valid tools for evaluation of faculty effectiveness.
However the results of the correlation reflect students’ increased inclination for the process of teaching rather than what the teachers or instructors can give to the students in terms of the teachers1 subject knowledge and the use of teaching aids etc. This suffers a serious shortcoming for acceptability by the teacher on the whole, as the failure to establish how these processes can validly measure effectiveness without resorting to the product or outcome measures.
The factor considered least contributory to the effectiveness of the teacher was the knowledge possessed by the faculty member related to the area they were teaching. This was contrary to the reported study2. Thus we infer that the students need basic outline of what they are to learn and want guidance to plan their studies instead of being passively told what they should know.

Acknowledgements

We wish to acknowledge the help of Ms. Lubna Sharif from Department of Pathology for data compilation and Mr. Syed Iqbal Azam from Department of Community Health Sciences for statistical analysis.

References

1. Brown, S. 1. Faculty and Student Perceptions of Effective Clinical Teachers J. Nurs. Educ., 1981; 20: 4-15.
2. Das, M., Subban, F., Bener, A. Student and Faculty Perception of the characteristics of an ideal teacher in a Classroom setting. Med. Teach., 1996; 8: 141-146.
3. Premdasa, 1G., Hijazi, Z., Moosa, A. An instrument to evaluate clinical instructional skills. Med. Educ., 1995; 29: 355-359.
4. Metcalfe, D.H., Matharu, M. Students perception of good and bad teaching, Med. Educ., 1995; 29: 193-197.
5. Kirchbaum, K. Clinical teaching effectiveness described in relation to learning outcomes of Baccalaureate nursing students. J. Educ. Psychol., 1994; 33: 306-315.
6. Abrami, P.C. Apollaniad, S Cohen, P A. Validity of student rating of instructions: What we know and what we do not. J. Educ. Psychol., 1990; 82: 2 19-231.
7. Irby, D.M. Evaluating instruction in Medial Education. J. Med. Educ., 1983; 844-849.
8. Aleamoni, L. M. Typical faculty concerns. In Techniques for evaluating and Improving Instruction. New Directions of teaching and learning. No. 31. San Francisco: Josey-Bass, Fall 1987, pp. 25-31.
9. Bergman, K., Gaitskill, 1. Faculty and student perception of effective clinical teachers: an extension study. J. Prof. Nurs., 1990; 6: 33-44.
10. Knox, J.E., Mogan, J. Important Clinical teacher behaviours as perceived by university faculty, students and graduates. J. Adv. Nurs., 1985; 10: 35-30.
11. Zimmerman, L., and Waltinan, N. Effective clinical behaviour of faculty, a review of literature. Nurs. Educ., 1986; 11: 31-34.

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: