Medical students' perception on the usefulness of online formative assessment: A single-center, mixed-method, pilot study
Himel Mondal1, Manas Ranjan Sahoo2, Ritushri Samantaray3, Shaikat Mondal4
1 Department of Physiology, Nil Ratan Sircar Medical College and Hospital, Kolkata, West Bengal, India
2 Department of Psychiatry, Fakir Mohan Medical College and Hospital, Balasore, Odisha, India
3 Department of Physiology, Pandit Raghunath Murmu Medical College and Hospital, Baripada, Odisha, India
4 Department of Physiology, Raiganj Government Medical College and Hospital, Raiganj, West Bengal, India
|Date of Submission||08-Sep-2020|
|Date of Acceptance||03-Dec-2020|
|Date of Web Publication||30-Jul-2021|
Dr. Shaikat Mondal
Department of Physiology, Raiganj Government Medical College and Hospital, Raiganj, West Bengal
Source of Support: None, Conflict of Interest: None
BACKGROUND: Distance digital learning is newly implemented in the Indian medical institutions. Formative assessment is also conducted online. In this context, this study aimed at finding the perception on the usefulness of online formative assessment along with online classes among 1st-year medical students.
MATERIALS AND METHODS: This cross-sectional, observational study was conducted in August 2020. The students were exposed to online classes and online formative assessments with five multiple choice questions for a period of 4 months before the conduct of the survey. An online survey was conducted (both quantitative and qualitative data collection) to ascertain the perception on the online formative assessment quiz.
RESULTS: A total of 36 (14 female, 21 male, 1 did not disclose sex) 1st-year medical students with a mean age of 19.97 ± 1.16 years participated in the survey. The majority of the students agreed that the online quiz was a valuable learning activity and has potential to replace the face-to-face assessment. The online quizzes provided them feedback of classroom learning and helped in identifying the weak area and motivated students to the study. The qualitative data showed that students like to get a greater number of questions and also like to participate in chapter wise multiple quizzes in spare time.
CONCLUSION: First-year medical students considered the online formative assessments quiz coupled with the online classes as a valuable learning activity. It provided them feedback of learning and a motivation for further study on the topic. They like to participate in anonymous quiz with a greater number of questions with online classes. The finding of this pilot study should be further evaluated with multicenter study with more participants.
Keywords: Distance education, learning, medical students, quiz, self-assessment
|How to cite this article:|
Mondal H, Sahoo MR, Samantaray R, Mondal S. Medical students' perception on the usefulness of online formative assessment: A single-center, mixed-method, pilot study. J Edu Health Promot 2021;10:243
|How to cite this URL:|
Mondal H, Sahoo MR, Samantaray R, Mondal S. Medical students' perception on the usefulness of online formative assessment: A single-center, mixed-method, pilot study. J Edu Health Promot [serial online] 2021 [cited 2021 Oct 18];10:243. Available from: https://www.jehp.net/text.asp?2021/10/1/243/322542
| Introduction|| |
Medical education in India has observed a sudden shift from traditional face-to-face learning to distance digital learning due to the coronavirus disease – 2019 (COVID-19) pandemic. Distance education is no new concept in India as several universities are offering distance education through different medium (e.g., books are sent to students via posts and students write the assignment and send back to teachers, attend online teaching sessions, and finally appear in one final face-to-face examination). Along with the other universities, health universities also started blended learning with the help of different digital medium. However, the COVID-19 pandemic forced the institution to adopt a fully digital teaching-learning activity as students are staying home. In our institution, along with online classes we were conducting online formative assessments through Google form immediately after the class. Hence, the students could get feedback about their learning from the online class.
Formative assessment is a type of self-assessment by the students to judge their learning on a particular topic. It may also provide feedback to the students about their classroom learning if the assessment is conducted after the class. Several studies have ascertained the impact of the online formative assessment on students' learning in undergraduate medical and dental students.,, Perception of students about the usefulness of online formative assessment has also been ascertained. Pharmacology students from USA perceive online formative assessment to be an valuable learning activity. In addition, the formative assessment is perceived to identify the weakness in learning and help in directing orientation toward improvement in final year medical students in Chile. However, compulsory online assessments are not so popular in medical students in South Africa. To the best of our knowledge, no previous study has explored the perception of Indian medical students on the usefulness of online formative assessment.
With this background, we aimed to conduct this pilot study to find out the perception on the usefulness of online formative assessment conducted along with online classes among 1st-year medical students.
| Materials and Methods|| |
Type, settings, and ethics
This was a cross-sectional, observational, single-center pilot study. The survey was conducted on August 27, 2020. This survey was conducted in a government-run medical college in the eastern region of India. The survey tool was a self-administered questionnaire. Hence, the students were exposed to negligible risk. The survey was a fully anonymous one, and the participation in this survey was voluntary. The survey questionnaire had a segment with informed consent, and any students agreeing to participate voluntarily were allowed to go to the survey questionnaire. This was considered equivalent to the paper-based signed informed consent form. We declare that the study was conducted in accordance highest ethical standard as per the World Medical Association Declaration of Helsinki (Ref. RBB-17; IC-26,2013/2020).
The survey was conducted with 1st-year medical students. The students were exposed to online classes (synchronous distance learning) and online formative assessments immediately after the class with five multiple choice questions (MCQs) for a period of 4 months. As this was a single-center pilot study, we did not calculate the minimum sample size but expected a minimum of 30 responses to be a credible pilot study.
We created the survey questionnaire for this survey after consulting relevant literature. The questionnaire had three parts. The first segment is the informed consent. This segment has only one response option – “I agree” and the response was compulsory. Any students who would select on the “I agree” can only proceed to the next segment (i.e., the survey form). Part I of the survey collected data on the age, sex, number of participations, device, and technical difficulty faced. The Part II collected data on the perception and opinion of the students about the online formative assessment. First four questions were to know their perception on the usefulness of online formative assessment in learning and rest there were to collect the preference about implementation method [Annexure 1]. This segment had an option to share text comments on the online formative assessment. The questionnaire was checked by three subject experts for content validity and was found to have good agreement among the experts.
The formative assessment sessions were conducted at the end of the online classes. A Google form (i.e., quiz) link was shared in students' WhatsApp group and the students participated in the quiz. Similarly, on August 27, after the online class, we shared the online survey link to the students' group and asked them to provide their opinion. The students were informed that the participation in this fully anonymous survey is voluntary. The form was kept online for 3 days from sending the link to the group. Then, the form was stopped for further responses. The form response was exported as a Microsoft Excel sheet for further analysis.
We had only the age of the participants as a continuous variable which was expressed in mean and standard deviation and compared between male and female with unpaired t-test. We expressed the categorical values as numbers and percentages and those were compared by the Chi-square test. For all the statistical analysis, we fixed the P < 0.05 to be considered as the statistical significance. All the statistical analyses were carried out in Microsoft Excel and in IBM SPSS. The open text responses were analyzed thematically by the first author. Cronbach's alpha was computed as an indicator of internal consistency of the questionnaire.
| Results|| |
A total of 36 students participated in the survey. There were 44 students present in the online class at the end of which the survey was conducted. Hence, the survey response rate may be considered as 81.82%. Among the participants, there was 14 (38.89%) female, 21 (58.33%) male, and 1 (2.78%) did not want to divulge sex (χ2 = 17.17, P = 0.0002). The mean age of the participants was 19.97 ± 1.16 years (female 19.93 ± 0.83 years, male 20.1 ± 1.3 years; unpaired t-test between male and female t = 0.42, P = 0.67; one who did not disclose the sex was of 18 years).
All of the students use smartphone to participate in the quiz. Fourteen students (38.89%) never faced any technical difficulty in participation, 15 (41.67%) students rarely faced, and 7 (19.44%) students sometimes faced technical difficulty (χ2 = 3.17, P = 0.21). Among the 36 respondents, 19 participated in 16–20 quizzes followed by 10, 6, and 1 students in 11–15, 6–10, and 1–5 quizzes, respectively (χ2 = 19.33, P = 0.0002) [Figure 1].
|Figure 1: Students' participation in 20 sessions of online formative assessment|
Click here to view
The survey response is presented in [Table 1]. Majority of the students agree that the online quiz is a valuable learning activity. It enables them to get feedback of their classroom learning and helps in identifying the weak area and motives themselves to study. The online quiz may be considered a replacement for offline quiz. Although many of the students think the online formative assessment should be compulsory, a significant number remains neutral and disagree on that. However, an anonymous formative assessment was favored by majority of the students.
|Table 1: Students' perception on online formative assessment with online classes|
Click here to view
Among the 36 students, 13 (36.11%) shared their views in addition to the compulsory questions. The thematic analysis of the qualitative data is presented in [Table 2] where the statements of the students are quoted and identifier is designed according to age, sex, and number of participations in 20 sessions of formative assessments. For example, 20F11–15 indicates that the commentator is a 20-years-old female who attended 11–15 sessions (range was given in questions as they may not remember the exact number of participation).
Students want more questions after online classes. In addition, they would like to participate in topic or chapter wise formative assessments, and it would help them to revise the topic according to the score. They expressed that the online quizzes are interesting and it would help them in their future National Exit Test (NEXT) examination.
With confidence level 95%, the confidence interval was 13.13. This indicates that if 52.78% of the students strongly agree that online formative assessment is valuable learning activity, 39.65% (52.78 - 13.13) to 65.91% (52.78 + 13.13) of the 1st-year medical students of other centres would have responded like our students.
The internal consistency of the questionnaire to know the perception of the students about the online formative assessment (i.e., first four question) was “good” (α = 0.848). Hence, this segment of the questionnaire may be used in future similar studies. For further insight, item wise statistics for internal consistency is shown in [Table 3].
| Discussion|| |
The majority of the students participated in all sessions of the formative assessment without facing any technical difficulty on their smartphone. Hence, online quiz may be a quick feasible method for conducting formative assessment among 1st-year medical students.
Students opined it to be a valuable learning activity. This finding is corroborative with the finding by Lull and Mathews and Labarca et al., Formative assessments provides continuous feedback to the learners. With the feedback, students can plan their strategy of further study on the topic. We found that majority of the 1st-year medical students think that the online formative assessments provided them feedback about the learning from the online classes. It helped them to identify their weakness in learning and motivated them to further study the topic. The result of this study on online formative assessment is found to be similar to the offline formative assessment.
Although formative assessment is an effective tool for the improvement of students' academic performance, it should be anonymous. Students of our institution also supported this view and majority of them wanted it to be faceless. In addition, it should be an optional academic activity. So, that the students are not bound to participate. However, the assessments should be made interesting so that students participate with their personal interest. Offline formative assessment may be coupled with 1-h lecture class where the teachers can provide the self-assessment questions at the end of the class. Our students suggested that online quiz have the potential to replace the face-to-face quiz. The major advantage of online quiz is that the students get immediate customized feedback on the learning which difficult in face-to-face paper based quiz.
Face-to-face formative assessment with objective structured practical examination has been found to be an effective method for improvement of practical skills among Indian medical undergraduate students. Furthermore, a frequent conduct of face-to-face formative assessment with feedback for filling the lacunae in learning improves the performance of the students. However, the current study was about the online formative assessment. In our institution, we were using five MCQs at the end of the online classes. However, in the open comment section, majority of the students commented that they want more number of questions. However, this may not be feasible due to time limitation as we cannot compromise the class time. Besides, students may engaged in solving the questions in their next immediate class too if there is more questions. To solve the issue, students suggested conducting additional formative assessment in spare time. They would like to self-assess after completing a segment of the chapter. The online formative assessment was an interesting academic activity to the students. Hence, other institutions may also start this method of formative assessment after online classes. As pointed out by two students in the open comment that these MCQ-based quizzes would help them in setting revision strategies and would help in their future NEXT examination. NEXT is a national level qualifying examination for medical graduate that is to be implemented in India in future.
The result of this pilot study can be considered as a reference for future studies. As per the result of this study, 1st-year undergraduate medical students are interested to participate in online formative assessment. Medical teachers may design their online classes along with MCQ based quizzes. However, care should be taken so that the quizzes are optional and anonymous. A chapter-wise online formative assessment may also be designed for students.
We tested the questionnaire for any further usage and found the internal consistency to be good.
This is the first study to ascertain the perception of Indian medical students on online formative assessment conducted with online classes. We conducted a fully anonymous survey so that the students could provide their opinion freely. The qualitative data provided further insight from the students that where not otherwise collected with the questionnaire we designed.
This study was conducted in a medical college from eastern part of India. Although the participants in the survey were adequate for this pilot study, a survey with more number of students would be better in future.
| Conclusion|| |
First-year medical students perceive that the online formative assessments along with the online classes are valuable learning activity that provide them feedback about the learning and motivates them for further study. Students would like to participate in optional anonymous quiz and they think the online quiz may be a replacement of face-to-face formative assessment quiz. The finding of this pilot study should be further evaluated with multi-center study with more participants.
We would like to thank the students for their active participation in the survey and providing their valuable opinion on the online formative assessment.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| Annexure 1|| |
My opinion on online formative assessment quiz
This is an anonymous survey. Please provide your opinion freely.
This is a survey to know your opinion on the usefulness of online formative assessment quiz conducted at the end of online classes. Participation in this survey is voluntary. Your identity remains undisclosed. Anonymous data of this survey will be analysed and published in future. To respond to the survey, please select “I agree” and click on the “Next” to go to the survey form. If you do not wish to participate, you may exit the browser. If you want to exit from the survey while in the form, please do not click on the “Submit” button. Your response will not be captured.
| References|| |
Panda S. Higher education at a distance and national development: Reflections on the Indian experience. Dis Educ 2005;26:205-25.
Ahmed H, Allaf M, Elghazaly H. COVID-19 and medical education. Lancet Infect Dis 2020;20:777-8.
Black P, Wiliam D. Developing the theory of formative assessment. Educ Asse Eval Acc 2009;21:5.
Mondal H, Mondal S. Trait emotional intelligence and self-assessment of classroom learning in medical students. J Adv Med Educ Prof 2020;8:109-14.
Velan GM, Jones P, McNeil HP, Kumar RK. Integrated online formative assessments in the biomedical sciences for medical students: benefits for learning. BMC Med Educ 2008;8:52.
Kibble J. Use of unsupervised online quizzes as formative assessment in a medical physiology course: effects of incentives on student participation and performance. Adv Physiol Educ 2007;31:253-60.
Olson BL, McDonald JL. Influence of online formative assessment upon student learning in biomedical science courses. J Dent Educ 2004;68:656-9.
Lull ME, Mathews JL. Online self-testing resources prepared by peer tutors as a formative assessment tool in pharmacology courses. Am J Pharm Educ 2016;80:124.
Labarca J, Figueroa C, Huidobro B, Wright AC, Riquelme A, Moreno R. Perception of medical students about formative assessments during clinical courses. Rev Med Chile 2014;142:1193-9.
Brown GA, Bice MR, Shaw BS, Shaw I. Online quizzes promote inconsistent improvements on in-class test performance in introductory anatomy and physiology. Adv Physiol Educ 2015;39:63-6.
Whitehead AL, Julious SA, Cooper CL, Campbell MJ. Estimating the sample size for a pilot randomised trial to minimise the overall trial sample size for the external pilot and main trial for a continuous outcome variable. Stat Methods Med Res 2016;25:1057-73.
Arja SB, Acharya Y, Alezaireg S, Ilavarasan V, Ala S, Arja SB. Implementation of formative assessment and its effectiveness in undergraduate medical education: an experience at a Caribbean medical school. Med EdPublish 2018;7:63.
Prashanti E, Ramnarayan K. Ten maxims of formative assessment. Adv Physiol Educ 2019;43:99-102.
Ho VW, Harris PG, Kumar RK, Velan GM. Knowledge maps: a tool for online assessment with automated feedback. Med Educ Online 2018;23:1457394.
Malhotra SD, Shah KN, Patel VJ. Objective structured practical examination as a tool for the formative assessment of practical skills of undergraduate students in pharmacology. J Educ Health Promot 2013;2:53.
Shahedi F, Ahmadi J, Sharifi T, Seyedhasani SN, Abdollahi M, Shaabani N, et al
. A new method of “student-centered formative assessment” and improving students' performance: An effort in the health promotion of community. J Educ Health Promot 2020;9:136.
Ranjan P, Ranjan R, Kumar M. National exit test: How will one size fit all? Ann Indian Acad Neurol 2020;23:145-9. [Full text]
[Table 1], [Table 2], [Table 3]