By Author
  By Title
  By Keywords

December 2015, Volume 65, Issue 12

Original Article

Introducing regular formative assessment to enhance learning among dental students at Islamic International Dental College

Fatima Riaz  ( Department of Physiology, Islamic International Dental College, Riphah International University, Islamabad. )
Shahina Yasmin  ( Department of Pathology, Islamic International Dental College, Riphah International University, Islamabad. )
Raheela Yasmin  ( Islamic International Medical College, Riphah International University, Rawalpindi. )

Abstract

Objective: To evaluate the effectiveness of Formative Assessment in enhancing learning among dental students, and to interpret the assessment from students\\\' perspective in this regard.
Methods: The experimental non-randomised controlled study was conducted from January to June 2013 at Islamic International Dental College, Islamabad, and comprised first year Bachelor of Dental Surgery students attending regular physiology lectures and tutorials. Summative assessments conducted at the end of each unit were included as pre-intervention tests. After one month\\\'s planning, central nervous system unit was delivered in a month\\\'s trial with four formative assessment and feedback sessions (one per week). Likert scale-based student feedback questionnaire was administered. Post-intervention summative assessment was done by Multiple Choice and Short Essay Questions. Data was analysed using SPSS 17.
Results: Out of 68 students, 64(94.1%) agreed that a conducive environment was maintained and 62(90%) agreed that such sessions should be continued throughout the year; 59(87%) reflected that the feedback provided by the teacher was timely and positive and ensured equitable participation; 56(82%)agreed that it enhanced their interest in the subject; 56(68%) agreed that they were now more focussed; and43(63%)were of the opinion that they have progressed in the subject through these sessions. There was highly significant improvement in the monthly post-intervention test scores compared to pre-intervention test (p=0.000).
Conclusions: Formative assessment sessions enhanced motivation and learning in first year dental students. Organised regular sessions with students\\\' feedback may contribute to the development of pedagogic practice.
Keywords: Formative assessment, Learning, Dental students. (JPMA 65: 1277; 2015).


Introduction

In recent decades, the conceptual framework of assessment has drastically changed from \\\'assessment of learning\\\' to \\\'assessment for learning\\\'. Formative assessment (FA) facilitates students\\\' learning through a continuous process of feedback and at the same time helps the teacher to make instructional modifications to adjust to students\\\' needs.1 Through FA, students have an opportunity to promote critical thinking and to generate comprehensive understanding of their on-going course.2 Without any grading it eventually supports a deep-learning attitude which is desired in learning medical sciences.3 FA provides feedback to students that allows them to make adjustments to what and how they are learning.4
We are all familiar with the examinations being conducted at the end of each course, which is to determine whether the students have acquired sufficient knowledge and skill. However, there is a growing awareness that in addition to summative assessment, we need additional educational interventions to foster the learning of each individual student to the maximum of their ability.1 Curriculum learning outcomes are well understood by students when wrapped in these FA sessions.5
FA is generally regarded as a significant factor in motivating learning among the students and to bridge the gap between their current understanding and the goals of the course.4,6 The feedback can be provided in a variety of ways depending upon local resources and other criteria like comments written on assignments, or be given orally following an assessed presentation or electronic feedback methods.5,7 It has been observed that sometimes this feedback is not properly delivered by the teacher or not read by the student. Feedback with a negative impact on a student is not formative.8 It has to be specific, clear and focussed upon what is attainable. In addition, the teacher should also be aware of the psychosocial needs of the student. Regarding the type of feedback, it has also been suggested to provide immediate feedback to low achievers (LAs), while for students with high achievement level delayed feedback is proposed.9
It is no doubt a difficult task to provide feedback in a large class and to keep a check on individual learning pace. The faculty may assume this activity as an additional burden. To sort out this problem, the institution need to provide an enabling environment for the faculty and may acknowledge their contribution through various incentives.10 FA is a collaborative act between staff and student whose primary purpose is to enhance the capability of the latter to the fullest extent possible11 and hence may improve healthcare professional\\\'s capability to better serve the community.
FA can also be negatively perceived as demanding by some students.3 To ensure students\\\' active involvement in these planned sessions, the faculty can attach some credits to it which will motivate students to take these formative tests. Also, with time they will start recognising this concept as a normal routine in academic life and an unavoidable practice in medical education.12
However, we cannot deny the impact of summative assessment in the educational process. There has to be constant switching of academic roles from that of a supporter of learning to an assessor of achievement.13 For effective integration of FA, space needs to be made in curricula with minimum effect on content delivery. However, unnecessary information regarding dental students\\\' curriculum should be omitted. Higher authorities and medical education department should provide proper training and monitoring to the faculty, thus maintaining an ideal balance between summative and formative assessments.10 Furthermore, it can help teachers design programmes that are more effective in promoting and motivating students towards the desired goals.
Undergraduate medical education in Pakistan has undergone major reforms. The current study was planned to evaluate the effectiveness of FA sessions with the larger aim of enhancing student learning skills and to clarify the learning outcomes through these sessions. Introducing regular FA sessions in curriculum has substantial benefits in the form of an on-going dynamic process and not to forget enhancing the overall course grade. Students\\\' feedback is an essential part of curricular evaluation and helps to identify gaps and weaknesses in the teaching learning process. Thus at the end of four-week FA intervention in Physiology, a questionnaire was also distributed to get students\\\' perception and make corrective adjustments if required.


Subjects and Methods

The experimental non-randomised controlled study was conducted from January to June 2013 at Islamic International Dental College (IIDC), Islamabad, and comprised all first year Bachelor of Dental Surgery (BDS) students attending regular physiology lectures and tutorials. Those who skipped 2 or more summative and/or formative assessments were excluded.
The study was approved by the Review Committee of Riphah International University, Islamabad.
In a traditional teaching setup at IIDC, constant efforts have started recently for an integrated dental curriculum designing, with the aim of involving students as active learners. A task force consisting of experienced medical educationalists, basic and clinical faculty members have also been formed. In the context of these efforts, the Physiology Department in assistance with the Head of the Department of Basic Medical Sciences conducted this survey. Physiology is taught in a large group, teacher-centred environment along with a tutorial per week. At the end of each course/module, a summative test is conducted after an average of one-and-a-half months. Three summative assessments were taken of this class up till the start of this study and these were considered pre-intervention tests.
Central nervouos system (CNS) physiology course was planned and revised to be conducted in one month involving FA and feedback sessions. Students were briefed about this FA in the beginning. There were three interactive lectures per week followed by FA and feedback session. For these sessions, the class was divided into three equal batches. In the first week, the FA session started with multiple choice questions (MCQs)to be solved in 20 minutes on the topics taught in that particular week. These were specially formulated for the sessions including all C1, C2 and C3 levels. Each MCQ test paper was shared by two students, so that they could discuss with each other, enhancing peer collaboration. After completion of the test, it was exchanged with other students to be marked by them. In the next 40min, the teacher discussed the attempted options and gave constructive feedback with active participation of students. The same method was repeated in the second week. In the third week, Exit slips/2 min paper was given to each student at the end of interactive lectures. This helped to give an insight of where the students were standing in their understanding and also to analyse the flaws in teaching methodology. The next step was to bring them to the desired level of learning outcome by instructional modifications, including concept maps and video illustrations, and then asking them to present it in groups. Teacher\\\'s oral feedback was accompanied all the time.
In the fourth week, students were asked to prepare a three-column table. During the class lectures, they wrote the concept understood in the first column, any questions or problems in the second column, and answers or solutions in the third column. Students were asked to bring the same pages for the 60-minute FA session. Answers to those problems were sorted out by discussion with the students and then written in the third column after satisfactory explanation.
A post-intervention, 15-item questionnaire related to this FA session was developed and administered to the class. They were asked to choose the answers on Likert scale from 1(strongly agree) to 5(strongly disagree). This was filled in before the post-intervention test, so that students\\\' answers about these sessions would not be influenced by their performance on the test. Also, these were filled anonymously.
Finally, at the end of the CNS course, a post-intervention test was taken of this class to examine the impact of FA sessions. Test included MCQs and student evaluation questionnaires (SEQs), as previously done, and the result was compared with the average of previous three-month test scores of the same class. It was ensured that all the items were linked to the curricula taught.
SPSS 17 was used to calculate frequencies and percentages in terms of statistical inference, and to calculate mean and standard deviation of pre- and post-intervention test scores using paired sample t-test.


Results

The feedback questionnaire was administered to 75 students, but 68(91%) fully completed questionnaires were returned (Table-1).


The first category related to session continuity and discipline maintained which provided a learning environment for the student. Answers to the question whether a learning environment was maintained during these sessions showed the highest positive response of 64(94.1%) collectively.
The second category concerned the provision of timely and positive feedback, and 34(50%) students strongly agreed that positive feedback was provided to them, while there were only 4(5.9%) total disagreements regarding timely feedback.
The third category was related to effective teaching strategies. Question asked about various tools used in these sessions scored the highest disagreement response of 11(16.1%).
The most important aspect of FA was about the effectiveness of this session in motivating students on that relevant subject, and 50(73%) students agreed that this system enhanced their critical thinking capabilities.
The last question was related to the continuity of such sessions, and 62(91.2%) students had a consensus on that again.
The mean values of the pre- and post-intervention tests showed strong evidence (p=0.001) that students\\\' performance was highly significant after FApost-intervention (66.60+/-11.9) than pre-intervention test results (58.08+-11.8).
The individual performance of students in the pre- and post-intervention test analysis showed visible improvement in majority of cases (Figure-1).

These sessions were especially beneficial for the LAs.
The sample was divided into two groups. Those who scored above the median score were categorised as high achievers (HAs), whereas those scoring at or below the median were classified as low achievers (LAs).
A highly significant p value (p=0.01) was indicative that formative assessment markedly enhanced students\\\' performance among LAs (4.65±5.09)compared to HAs (1.48±4.70) (Figure-2).

Mean difference in the performance change between LAs and HAs was 3.18±1.24.


Discussion

The aim behind all curricular reforms is to improve the quality of curricular activities with ultimate benefits to the students who will be serving the community. The current medical education system in Pakistan is based on high-stake assessments with inadequate emphasis on appraising the student. In order to nurture their learning process and self-efficacy, FA needs to be an integral part of curriculum. Consequently, we decided to carry out a research on first year IIDC students studying in a traditional setup. Four-week FA session was planned for a CNS course taught in Physiology and its effects were observed in terms of students\\\' course grades. Students\\\' perception about these sessions was also highlighted.
Although the time provided for FA was very short, it did provide means, like self-testing, teacher consultation, peer collaboration, that allowed students to measure their progress in learning. A well-designed FA promotes self-directed learning and effective utilisation of higher order thinking.14
Our study also showed that FA was very well accepted by students. The comments included that it was more interactive, thought-provoking and urged them to search more for knowledge plus generated a more comprehensive understanding of the designed course.
Feedback given to students has an inevitable role in FA. In our study, we also drew help from the seven principles of good feedback practice.15 In another study conducted in Bangladesh,16 it was stated that feedback provided to students helps to identify their weaknesses and helps them to be more investigative in their learning endeavours. Our efforts in this regard are also evident from 50% agreed responses of students about positivity of feedback facilitating self-directed learning prior to the summative assessment.
In our FA sessions, we put in all efforts to keep students as active participants by engaging them in different activities. There is no doubt that classroom climate plays a very important role in FA. A study in a first year module with the aim of developing a learning environment is more conducive to students\\\' active participation and learning.17 With the intervention, the study reported having improved the students\\\' attendance and contribution through the introduction of group presentations and FA. The questions, \\\'Whether a learning environment was maintained\\\', in our sessions scored highest positive response in all the questions asked. Motivating students is another important milestone to improve students\\\' contribution and responsibility towards their learning. A systematic review on motivation in medical education highlighted the traditional curricular setup that might be robbing the students of this motivation.18 Our results regarding student motivation and achievement scores are quite contrary to a study that stated that teachers being the role model were extremely influential on students\\\' motivation, achievement, and conceptual change, and, hence, overshadowed the FA treatment effect.19 Hence, an embedded FA can be influenced by teachers\\\' classroom management and the way they use informal FA. For this reason, we are constantly highlighting on teachers\\\' training programmes and the means for delivering an effective feedback.
The teacher at IIDC enjoys a unique role in influencing students\\\' learning and mentoring their development. Different strategies need to be adopted depending upon the socio-cultural background of the student. Through FA sessions, not only the students have an opportunity to learn, but they also contribute to teachers\\\' own professional development. With practically getting into action research, new questions emerge as evaluation takes place.20 Through these sessions, teachers\\\' role becomes multidimensional as a guide, counsellor and mentor, tailoring to the individual student\\\'s needs.
A similar study highlighted that student when engaged in assessment discussions, performed significantly higher on embedded post tests.21 However, the sample size of that study was very small. Another study suggested that students who took part in practice tests significantly outperformed those who did not take part.22 However, the ways in which FA can affect the motivation and self-esteem of students is not an easy task, but requires careful monitoring and attention.23
Pre- and post-intervention test results are clearly indicative of the individual student improvement. Initially, our main focus was the weak students. For them it is very difficult without FA and careful monitoring to achieve learning outcomes within the desired timeframe.24 A study drew together over 250 studies spanning all educational sectors based on teacher-made assessments and self and peer assessment. The meta study of educational intervention showed that FA helped LAs more than the Has.23 Our results are in agreement with those of the earlier study.
There were certain limitations in our study like limited time available and less number of trained faculty staff. It is no doubt a difficult task in a large class to set up the FA process with timely feedback, focussing on the individual changes each student needs to make. It requires careful planning, monitoring and lots of research keeping in mind the ground realities. With the encouraging results in this limited time and resources, we are aiming to foster this process throughout the academic year with proper evaluation and essential modifications for the upcoming first year BDS class.
It should be kept in mind that this is a slow process.25 It may not show its effects immediately in terms of improved grades. Repetitive self-assessment tasks and frequent opportunities to reflect on goals throughout the entire period will prepare the students for lifelong learning which would support a higher education model.
For future studies, we recommend that other Basic and Clinical sciences should be involved to analyse FA\\\'s impact on student learning. FA needs to be entertained as mandatory part of curriculum in medical/dental setup if we want to inculcate deep learning attitude and motivation among students. That being so, regular faculty training and development programmes by experienced medical educationists are essential, and for that to happen, a certain budget has to be allocated for conducting a well-organised FA session in terms of further research, teachers\\\' training programmes along with increase in faculty. And, finally, evaluation of the whole programme at the end of session, with both external and internal reviews should be encouraged.


Conclusion

FA had a fruitful influence on learning of first year BDS students. It supported deep learning attitude and achievement standards, especially for the low achievers. It can be introduced even in a traditional teaching setup. Feedback from students helps to evaluate and modify the teaching-learning process and develop strategies to make them lifelong learners.


Acknowledgement

The study was first presented at theAssociation for Excellence in Medical Educationforum held at the University of Health Sciences (UHS), Lahore, in March 2014.


References

1. Schuwirth LW, Van der Vleuten CP. Programmatic assessment:From assessment of learning to assessment for learning. Med Teach. 2011; 33: 478-85.
2. Jarene F, Yvonne TV, Rebecca P, Kathy D. Formative Feedback. Involving Students as Partners in Assessment to Enhance Learning. College Teaching. 2010; 58: 136-40.
3. Ahmed Abu-Zaid. Formative assessment in medical education. Perspect Med Educ. 2013; 2: 358-9.
4. Havnesa A, Smith K,Dystheb O,Ludvigsen K. Formative assessment and feedback.Making learning visible. Studies Educ Eval 2012; 38: 21-7.
5. Yorke M. Formative assessment in higher education. Moves towards theory and the enhancement of pedagogic practice. Higher Educ 2003; 45: 477-501.
6. Chang Zhu.Providing formative feedback to students via emails and feedback strategies based on student metacognition.Reflecting Educ 2012; 8: 78-93.
7. Jung-Lung H , Huey-Wen C, Hsiu-Hua C . Using text mining for automatic formative assessment. Expert Sys Appl. 2011; 38: 3431-9.
8. Valerie J. Shute. Focus on Formative Feedback. Review of Educational Research. 2008; 78: 153-89.
9. Mason BJ, Bruning R. Providing feedback in computer-based instruction. [Online] 2014 (Cited 2014 September 4). Available from URL: http://dwb4.unl.edu/dwb/Research/MB/MasonBruning.html.
10. Rauf A, Shamim MS, Aly SM, Chundrigar T, Alam SN. Formative assessment in undergraduate medical education.Concept, implementation and hurdles.J Pak Med Assoc. 2014; 64: 72-5.
11. Stiggins R, Chappuis J. Using Student-Involved Classroom Assessment to Close Achievement Gaps. Theory Pract. 2005; 44: 11-8.
12. Ronald M. Epstein; Assessment in Medical Education. N Engl J Med. 2007; 356: 387-96.
13. Ramsden P, Moses I. Associations between research and teaching in Australian higher education. Higher Educ 1992; 23: 273-95.
14. Velan GM, Jones P, McNeil HP, Kumar RK. Integrated online formative assessments in the biomedical sciences for medical students.BMC Med Educ 2008; 8: 52.
15. Nicol DJ, Dick DM. Formative assessment and self-regulated learning: a model and seven principles of good feedback practice. In: Studies in Higher Education. 2006; 31: 199-218.
16. Alam KK, Begum SN, Nargis T, Faruque MD. Feedback on formative assessments in undergraduate medical education of Bangladesh. Bangladesh J Physiol Pharmacol 2009; 25: 18-22.
17. Jon Dart. Developing a Learning Environment Conducive to Active Learning and Participation. Group Presentations and Formative Assessment at Level One. J Hospitality Leisure Sport Tourism Educ. 2006; 5: 58.
18. Brissette A, Howes D. Motivation in Medical Education.A Systematic Review [Online] 2010 [Cited 2010 December 13]. Available from: URL: http://www.webmedcentral.com/article_view/1261.
19. Yue Yin, Shavelson RJ, Ayala CC, Ruiz-Primo MA, Brandon PR, Erin Marie Furtak, et al. On the Impact of Formative Assessment on Student Motivation, Achievement, and Conceptual Change. Applied Measure Educ 2008; 21: 335-59.
20. Wiliam D. Formative assessment. In: getting the focus right. Educational Assess. 2006; 11: 283-9.
21. Ruiz-Primo MA, Furtak EM. Informal Formative Assessment and Scientific Inquiry. Educational Assess 2006; 11: 205-235.
22. LeithSly.Practice Tests as Formative Assessment Improve Student Performance on Computer?managed Learning Assessments. Assess Eval Higher Educ. 1999; 24: 339-43.
23. Paul B, Dylan W. Assessment and Classroom Learning. Assessment in Education: Principles, Policy & Practice 1998; 5:7 -74.
24. Mansfield P, Boase-Jelinek D. Optimising Automated Feedback Systems to Motivate Students. E Culture. 2010; 3:9.
25. Heritage M. Formative Assessment and Next-Generation Assessment Systems. National Center for Research on Evaluation, Standards, and Student Testing 2010. [Online] 2010 (Cited 2014 September 6). Available from: URL: http://www.edweek.org/media/formative_assessment_next_generation_heritage.pdf.

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: