By Author
  By Title
  By Keywords

June 2018, Volume 68, Issue 6

Research Article

Preclinical medical students\' perspective on technology enhanced assessment for learning

Zehra Jamil  ( Aga Khan University )
Syeda Sadia Fatima  ( Aga Khan University )
Amna Afzal Saeed  ( Medical Student, Aga Khan University, Karachi )

Abstract

Objective: To explore the potentials of technology-assisted assessment for learning using Kahoot software in teaching session.
Methods: This cross-sectional study was conducted at Aga Khan University, Karachi, to investigate the usefulness of formative assessment based on the use of Kahoot, a quiz-based learning platform, in undergraduate setting.Six lectures were offered to undergraduate medical students with integration of assessment for learning (AfL) activities. Students\\\' perception was sought via questionnaire regarding effectiveness of quizzing on classroom dynamics, meaningful learning and assessment practice.
Results: Of the 171 respondents, 155(91%) stated that technology-enriched methodologies were in line to their learning strategy while 138(81%) students rated their experience with technology-supported assessment for learning as "Excellent". The students perceived highest positive influence on the classroom dynamics [109(63.8%)], followed by assistance to learning [100(58.58%)] and assessment performance [88(51.7%)]. Overall, 133(78%) students agreed to the notion that quizzes aided in summarisation of concept and consolidation of essential content. Additionally, 113(66%) participant expressed that anonymity helped them take quizzes as earnest opportunity to learn without any fright of failure.
Conclusion: AfL leads to a paradigm shift in the classroom, transferring the ownership of learning to the students. There is a need to implement such activities as a routine across diverse educational settings such as labs, lectures or even clinical rotations.
Keywords: Assessment for learning, Active engagement, Technology enhanced learning, Formative assessment. (JPMA 68: 898; 2018)


Introduction

The quality of learning in any higher education setting is driven by the power of summative assessment. However, it has been noted that students\\\' experiences of assessment creates an opportunity to learn, hence supporting the critical role of formative assessment or assessment for learning (AfL) in adult education.1 Benefits of AfL in the classroom are heavily documented in literature. It is said to enhance the "cognitive, social, emotional and motivational profits"2 along with "engaging and stimulating students to learn".3-5 As each individual learns differently, diverse teaching pedagogies are required to cater needs of any classroom.6 Consequently in order to engage the audience with such diversity, best practice entails a facilitator to create learning that supports five fundamental student engagement strategies i.e. relating, experiencing, applying, cooperating and transferring (REACT) strategy. 7 As we focus on refining the quality of higher education, learning outcomes and their assessment dominates the list. 8 Unfortunately, majority of the assessment systems merely focus on it as a tool of measurement. 9 However, there is sufficient evidence that assessments themselves drive learning and therefore, when conducted formatively, may even create a better teaching and learning environment. 10 Inclusion of AfL motivates adult learners to achieve better learning outcomes. 11 One of the most obvious advantage of utilising AfL is the fact that it generates evidence of learning straight away; not only for the students but for the teachers as well. 12 Such evidences assist the teachers while pondering on the effectiveness of their teaching to cater to diverse learning styles in their sessions. 13 With the advent of information technology, educational resources have become unlimited. 14 Various web-based tools that support student-centred teachings have surfaced that serve as a major contributor to fulfil the needs of diverse learning styles. 15,16Similarly, freely available online websites can be utilised to develop tailored AfL for any learning sessions. Incorporation of quizzes as a teaching and learning strategy helps to actively engage the learner and to create links between the components of REACT strategy. Dr. Siegle elaborates that "engagement increases students\\\' interactions with content and in many instances, with each other".17 Diverse assessment exercises are practiced within medical school to enhance active learning yet there are associated challenges. The quiz should not only cater to individual learning, group learning and peer-to-peer learning but in addition, it should also be affordable and user-friendly to create. Regrettably, though e-learning has become a common practice in many institutes yet the opportunity to incorporate technology based assessment within teaching sessions is yet to be fully employed. Thus, in order to explore the potential of AfL via technology-based quizzes, this study was planned to assess students\\\' perception regarding benefits of AfL in augmenting adult learning in a medical education settings. The use of Kahoot, a freely available online software, in creating tailored quizzes was also looked at.

Materials and Methods

This cross-sectional study was performed in the Neurosciences Module (Year-II) and Cardiovascular Module (Year-I)at Aga Khan University, Karachi, after approval from the institutional review committee. The questionnaire was administered to explore the impact of visibility of game points on the students\\\' concentration, engagement, enjoyment, motivation, learning outcomes and classroom dynamics. Across the globe, majority of the medical curricula are designed with an emphasis on basic sciences with selected integration of clinical skills in the first two years of undergraduate education. Hence, our convenient sampling population can be considered to generate generalised data that is applicable to all the target population that is preclinical medical students. Kahoot-based quizzes were introduced in six different sessions (4 for Anatomy and 2 for Physiology) mixing the time of launching it at the middle and at the end of the session.
i) The sessions began with introduction of learning objectives, followed by the discussion on the core content for 15-25 minutes. This was followed by the quiz activity based on screen display of question along with four options to choose from. The students responded through their electronic gadgets such as mobile phones. The correct option was displayed after each question that led to addressing the shortcomings in students understanding. Each quiz was created to assist students in achieving the learning outcomes of that session. Lastly, after experiencing this activity in six sessions, the students were requested to fill a questionnaire to reflect their perception. Kahoot is a game-based student response system (GSRS). The facilitators can create their own quiz, surveys, and online discussion forums to be launched during a teaching session. It even allows adding images or videos to the questions (Figure).



ii) A 15-item questionnaire was developed to collect data on students\\\' perceived usefulness, concentration, engagement, enjoyment, learning, and motivation. It was constructed to seek views on the extent to which assessment enables the learning based on earlier findings. 18 We also adopted some of the questions from the "Motivated Strategies for Learning Questionnaire" (MSQL). 19,20The study questionnaire used a five-point Likert scale itemised from strongly disagree to strongly agree, including an option for uncertainty. The students\\\' perception on questionnaire was sought by an independent individual unrelated to this study. SPSS 21 was used to analyse the data. As with the case of categorical variables, the results are summarised in the form of frequencies and percentages.

Results

Of the 171 responders, 94(55%) students were from year-II and 77(45%) were from Year-I. Overall, 156(91%) students reported being comfortable while studying with technology-enriched methodologies. Besides, 139(81%) students rated the overall effectiveness of technology supported AfL as "excellent". The three broader themes explored for the benefits of AfL were noted (Table-1).



i
) Seven components were designed to seek students\\\' perception regarding the effects of AfL on the classroom environment. Besides, 124(71%) participants were of the opinion that constructive change in the classroom environment was observed as AfL challenged their learning. Regarding the opportunity to personally participate in cognising the concepts, 132(77%) agreed to it. The absence of the fear of failing the test was reported as beneficial by 139(81%) subjects. Moreover, 132(77%) students found themselves focussed to their learning as they aimed at attempting AfL at the end of the sessions. As this technology helped to maintain anonymity, 113(66%) reported being encouraged to answer the quizzes while 41(24%) students disagreed to anonymity being a noteworthy factor for attempting a formative assessment. Contrary to these fruitful findings, majority of the students were either undecisive 75(44%) or disagreed 36(21.2%) to the notion that in these sessions, there was less distraction as their electronic gadgets were enrolled to quiz website. Four components were designed to evaluate students\\\' perception regarding usefulness of AfL on their learning. The maximum response of agreement 144(84%) was achieved in the fact that these assessments led to summarisation of their learning. Besides, 103(60%) participants responded as AfL being a useful activity in emphasising the misconceptions made during the lecture as well as augmenting critical thinking. However, when inquired about whether there was an opportunity to revisit the concept in case majority of the class responded incorrectly, we received varied responses. Majority of the students were indecisive 67(39%), 41(24%) disagreed while 63(37%) reported the brief description given while sharing the answers as being helpful in clearing their concepts.
ii) Lastly, we sought students\\\' opinion regarding effectiveness of Kahoot-based quizzes on their assessment and 132(77%) students reported that as the correct answers were shared on the spot with each question, they had an evidence of their learning; a unique benefit linked to AfL-driven learning. Similarly, as students\\\' active engagement is driven by interaction with their classmates, 137(80%) agreed that by this activity, they were able to compare their learning with the learning of their peers; yet in anonymity. Next we inquired students\\\' perception regarding allocation of a limited percentage of such AfL as part of their summative assessments, a large cohort disagreed 118(69%) while only 46(27%) supported this idea. Finally, as one of an essential aspect of any assessment is to drive autonomous learning in our students, we inquired whether AfL in the session enhanced their pre-lecture learning as they were looking forward to attempting the quizzes. Unfortunately, 125(73%) disagreed with the notion and reported that even though they looked forward to attempting quizzes, they came unprepared for the sessions (Table-2).



While using the Kahoot quiz, the whole class showed high spirit, was more focused, there was laughter, quick discussions with peers and the facilitators during and at the end of the session. Overall, the atmosphere was cheerful and the students showed joy on getting the answers correct by clapping or dancing to the Kahoot tunes. Based on these observations it was obvious that the use of Kahoot had the largest impact on classroom dynamics in terms of interaction, response and spirit. More so, the audio simply produced more energy in the room, and opened up for a more interactive environment.

Discussion

AfL-driven active engagement is the buzzword across the globe. This study compiles students\\\' perception regarding benefits of formative assessments in undergraduate medical education. The initial years comprises greater number of concept-based lectures compared to bedside learning, this paper reports the effects of AfL on the classroom environment as well as students\\\' learning in lecture settings. Lectures are often criticised as an ineffective tool for engaging the students with meaningful learning yet literature compiles conflicting results in terms of their pros and cons. 19 While comparing learning outcomes of lectures with that of problem-based learning (PBL), there is insufficient evidence to support higher learning in PBLs. 20 Moreover, lectures that inspire the students are reported to stimulate further enquiry and critical thinking that leads to achieving higher learning outcomes. 21 Therefore, it seems that lectures are and will stay as a resilient pedagogy to deliver the content, however, the roles of teacher as well as student will keep shifting to a newer paradigm; learner-centred education with the teacher as a guide on the side. 22The participants of our study reports substantial role of AfL ineffectively engaging the learner incognising the concepts in a large class format (LCF) setting. The facilitators noticed an obvious shift of paradigm while using Kahoot based quizzes, where students owned their learning while teacher facilitated critical thinking by clarifying misconceptions or revisiting concepts that came up as challenging for them. The student further revealed a positive change in theoverall classroom environment as they found themselves more focused while being provoked by the questions. Motivational theories suggest that once the learning environment created by the facilitator motivates a student extrinsically (reinforcement driven motivation) and/or intrinsically (self-determined motivation), their tendency to focus and engage in meaningful learning is enhanced. 23 We found out that in our study, quizzes served dual function as they boosted the learning as well as performance of our students. On one hand, they processed the information and comprehend the knowledge while on the other hand visualised their own performance via scores of the quizzes. This created a virtuous cycle where students could see their learning achievements as well as weaknesses on the spot; adding motivation to further build up their knowledge. Guarding the self-esteem of individuals has been of great value in adult learning. 24 Provision to participate as anonymous in the quiz has been found to be a worthy driving force in our study. This quizzing technology allows participants to enrol with nickname; known to themselves only. Further, the result for each question is presented as a summarised bar chart where the teacher gets an overall impression of the performance of the class while enabling individuals to judge their own performance. The students reported this activity as an earnest opportunity to learn without any fright of failure or offence, adding to the gratification of self-esteem. In our study, although majority of the class was opportunistic about AfL-driven learning, yet 15% students were of the opinion that this exercise seemed overwhelming and confusing to them. On further interrogation, few reported limited time duration to attempt each question as a challenge while others reported ease of attempting the quiz at least a day ahead of the lecture. This highlights the role of teacher\\\'s support that seems essential even while technology enhance strategies to engage the audience are incorporated. Identification of struggling students and individualised guidance may enhance their learning as well as performance, essential to become an independent life-long learner. Advancements in technology such as portable electronic gadgets have many advantages yet simultaneously, they act as a source of distraction in any classroom setting. In their presence, active engagement of the students with their learning remains an Achilles\\\' heel in today\\\'s world. As we sought students\\\' opinion, majority remained unclear whether enrolling into the quiz engaged their gadgets to an extent that could reduce distraction. Diversions even in the presence of personal response submission via their own mobile phones inevitably support the fact that adult learning can be maximise only when its ownership rests within the student body. Newer pedagogies such as flipped classroom are gaining fame as responsibility to learn lies with the students in such teaching methodologies and AfL gel in very successful with such pedagogies. 25 As we further explored the gains on learning, students agreed to the notion that such activities lead to summarisation of the core concept and consolidation of essential content. They viewed AfL as an opportunity to identify their misconceptions and to be able to re-visit challenging concepts with their facilitators. A large proportion suggested allocation of more time to clarify concepts where the overall class performance was generally weak. In the past, audience response systems that worked on the same principles have been adopted in many higher education setting. 18 However, development of a number of newer websites that allow students\\\' participation with their own mobiles have led to save us from investing on these costly systems. Indeed, technology enriched tools are playing a pivotal role in our education and it was enlightening to see that a huge majority of the students acknowledged that with these activities they had an \\\'evidence of their own learning\\\'. Researches in medical education emphasises on the importance of a sound knowledge background and effective memory networks upon which, further clinical expertise are built. 26 Assessment-driven learning of core concepts can potentially strengthen the clinical knowledge required to develop better medical expertise. Finally, as we look through the challenges hooked with fruitful use of AfL in any setting, all three elements i.e. technology, students and teachers, require critical consideration. Back-ups for internet connection, a copy of quiz on PowerPoint slides as standby tools can be helpful in some circumstances. Students\\\' compliance is directly or indirectly linked to the motivation of the facilitators to make their session stand out; teachers\\\' urge to ensure active learning via AfL acts as the epitome. 27 Our study is limited in the fact that it collects the perception of students studying at one centre. We recommend implementation of such activities as a routine in teaching and learning sessions. However, appropriate training of the trainers to utilise these tools and to be cognisant with the learning theories are essential to achieve highest goals. To conclude with the notion that teachers\\\' learning is the way forward, Assessment Reform Group\\\'s 10 principles13 categorically values it as: "Assessment for learning should be regarded as a key professional skill for teachers. Teachers require the professional knowledge and skills to: plan for assessment; observe learning; analyse and interpret evidence of learning; give feedback to learners and support learners in self-assessment. Teachers should be supported in developing these skills through initial and continuing professional development".

Conclusion

Assessment for learning creates a scholar-centred environment where ownership of learning rests with the students. Summarisation of the core concept and consolidation of essential content through these quizzes drive meaningful learning. Indeed, technology enriched tools can play a pivotal role in our education as the \\\'evidence of learning\\\' generated by AfL supports both learning as well as performance of our students.

Disclaimer: None.
Conflict of Interest: None.
Source of Funding: None.

References

1.  Maclellan E. Assessment for learning: The differing perceptions of tutors and students. Assessment Evaluation Higher Educ 2001; 26: 307-18.
2.  Brophy J. Motivating Students to Learn. 2nd ed. London: Lawrence Erlbaum Associates, Publishers; 2004.
3.  Jabbar AIA, Felicia P. Gameplay Engagement and Learning in Game-Based Learning A Systematic Review. Rev Educ Res 2015; 85: 740-79.
4.  Dickey MD. Three-dimensional virtual worlds and distance learning: two case studies of Active Worlds as a medium for distance education. Br J Educ Technol 2005; 36: 439-51.
5.  Ermi L, Mäyrä F. Fundamental components of the gameplay experience: Analysing immersion. Worlds in play: International perspectives on digital games research 2005; 37: 2.
6.  Fleming ND. Teaching and learning styles: VARK strategies: IGI Global; 2001.
7.  Ültay E. Investigating the effect of the activities based on explanation assisted REACT strategy in context-based learning approach on impulse, momentum and collisions. [PhD Thesis]. Trabzon, Turkey: Karadeniz Technical University,. 2014.
8.  Knight PT. Summative assessment in higher education: practices in disarray. Stud Higher Educ 2002; 27: 275-86.
9.  Gibbs G, Simpson C. Conditions under which assessment supports students\\\' learning. Learning Teaching Higher Educ 2005; 1: 3-31.
10.  Nicol DJ, Macfarlane-Dick D. Formative assessment and self regulated learning: A model and seven principles of good feedback practice. Stud Higher Educ 2006; 31:199-218.
11.  Taras M. Assessment-summative and formative-some theoretical reflections. Br J Educ Stud 2005; 53: 466-78.
12.  Davies BS, Rafique J, Vincent TR, Fairclough J, Packer MH, Vincent R, et al. Mobile Medical Education (MoMEd)-how mobile information resources contribute to learning for undergraduate clinical students-a mixed methods study. BMC Med Educ 2012; 12: 1.
13.  Spendlove D. Putting Assessment for Learning into Practice. UK: Bloomsbury Publishing; 2009.
14.  Han H, Resch DS, Kovach RA. Educational technology in medical education. Teach Learning Med 2013; 25(sup1): S39-S43.
15.  Laurillard D. The teacher as action researcher: Using technology to capture pedagogic form. Stud Higher Educ 2008; 33: 139-54.
16.  Laurillard D. Technology, pedagogy and education: Concluding comments. Technology Pedagogy Education 2007; 16: 357-60.
17.  Siegle D. Technology Learning Can Be Fun and Games. Gifted Child Today 2015; 38: 192-7.
18.  Kay RH, LeSage A. Examining the benefits and challenges of using audience response systems: A review of the literature. Computers Educ 2009; 53: 819-27.
19.  Jones SE. Reflections on the lecture: outmoded medium or instrument of inspiration? J Further Higher Educ 2007; 31: 397-406.
20.  Malicky DM, Lord SM, Huang MZ. A design methodology for choosing an optimal pedagogy: the pedagogy decision matrix. Int J Engineering Educ 2007; 23: 325.
21.  Savoy A, Proctor RW, Salvendy G. Information retention from PowerPoint™ and traditional lectures. Computer Educ 2009; 52: 858-67.
22.  McCoy L, Pettit RK, Lewis JH, Bennett T, Carrasco N, Brysacz S, et al. Developing technology-enhanced active learning for medical education: challenges, solutions, and future directions. J Am Osteopath Assoc 2015; 115: 202-11.
23.  Brophy JE. Motivating students to learn. 4th ed. UK: Routledge 2013.
24.  Kahu ER. Framing student engagement in higher education. Stud Higher Educ 2013; 38: 758-73.
25. Moffett J. Twelve tips for "flipping" the classroom. Med Teacher 2015; 37: 331-6.
26.  Larsen DP, Butler AC, Roediger III HL. Test-enhanced learning in medical education. Med Educ 2008; 42: 959-66.
27.  Dennis AA, Cleland JA, Johnston P, Ker JS, Lough M, Rees CE. Exploring stakeholders\\\' views of medical education research priorities: a national survey. Med Educ 2014; 48: 1078-91.

Journal of the Pakistan Medical Association has agreed to receive and publish manuscripts in accordance with the principles of the following committees: