The feasibility of simulation-based high-stakes assessment in emergency medicine settings: A scoping review
Loui K Alsulimani
Department of Emergency Medicine, Faculty of Medicine; Clinical Skills and Simulation Center, King Abulaziz University, Jeddah, Saudi Arabia
|Date of Submission||30-Aug-2020|
|Date of Acceptance||17-Jul-2021|
|Date of Web Publication||30-Nov-2021|
Dr. Loui K Alsulimani
1 Jamha Street, Zip Code. 80215, P. O Box: 21589, Jeddah
Source of Support: None, Conflict of Interest: None
Using simulation in high-stakes assessments has been evolving as a method to improve the assessment process. There is a concurrent need to address challenges and establish best practices to ensure the best quality when implementing high-stakes evaluations. The aim of this study is to provide an insight for stakeholders about using multiple modalities of simulation in high-stakes evaluations by presenting challenges, best practices, and future directions described in the relevant literature. A scoping review of original studies (from the year 1994–2021) including the use of common modalities (standardized patients, high-fidelity mannequins, part-task trainers, virtual simulation, and hybrid simulation) was conducted. The search covered the common databases: PubMed, Education Resource Information Center, Cumulative Index to Nursing and Allied Health Literature, and the Cochrane library. Initial screening in the databases resulted in 111,253 articles. After the application of a refining search strategy, 47 articles were included for comprehensive evaluation. Most articles were about credentialing/licensure purposes of assessment. Many articles were specialty-specific, especially focusing on anesthesia. Most challenges described were related to the validity of the assessment that should be considered in the planning phase. Best practices were mostly related to planning for measures to assure the validity of the assessment tools and process. The use of multimodality simulation for high-stakes assessment is growing despite challenges; this growth is associated with the accumulation of experience shared in literature. This growth will help to improve planning, practices, and goals achievement from such an application.
Keywords: Competency based, education, educational assessment, emergency medicine, examination question, patients simulation, review literature
|How to cite this article:|
Alsulimani LK. The feasibility of simulation-based high-stakes assessment in emergency medicine settings: A scoping review. J Edu Health Promot 2021;10:441
|How to cite this URL:|
Alsulimani LK. The feasibility of simulation-based high-stakes assessment in emergency medicine settings: A scoping review. J Edu Health Promot [serial online] 2021 [cited 2022 Jan 22];10:441. Available from: https://www.jehp.net/text.asp?2021/10/1/441/331424
| Introduction|| |
Assuring the competency of health care providers (HCPs) is an expected outcome of training desired by accrediting institutions. Stakeholders (e.g., the learners, teachers and educational institutions, and the wider society) deliver competency assessment of grading and practicing healthcare provider's paramount attention based on its impact on the quality of health care. Moreover, the process of assessment is a key driver for the development of educational systems. Competency assessment of healthcare providers has been growing concurrently with the development of the healthcare system, yet it is still challenging to provide a high-quality evaluation. This challenge is further magnified when dealing with high-stakes assessments which could be defined as, “an evaluation process that has a major academic, educational, or employment consequence....” In this case, the sensitivity and the demands for quality assurance are much higher than the demands for low-stakes (or formative) assessments.
Simulation has been introduced to the field of training and education successfully., Recently, using simulations in high-stakes assessments have been evolving as a method to enhance and improve the assessment process. There are many examples of successful implementation of high-stakes assessment integrating simulation. In 1998, the Educational Commission for Foreign Medical Graduates introduced the Clinical Skills Assessment, which uses standardized patient (SPs) as the modality of simulation, for certification of foreign medical school graduates. In Israel, simulation has been implemented in many high-stakes assessment projects for more than 10 years. These projects include medical schools' candidate selection, national board examination in anesthesiology, and national accreditation for paramedics. In 2003, the Royal College of Physicians and Surgeons of Canada introduced a hybrid simulation in the Comprehensive Examination in Internal Medicine.
Although medical literature shows a positive attitude toward such an implementation, articles from nursing literature demonstrated a more conservative attitude., The limited experience with such use of simulations and the lack of relevant expertise in the field of nursing can make the process of applying quality assurance measures more challenging. Without the ability to assure the quality, the validity of the assessment would be questionable. After a meeting discussion at the International Nursing Association for Clinical Simulation and Learning, a conclusion came to show that students should not be judged by one modality until further experience grows to demonstrate that this assessment method provides a quality assessment.
In relation to the specialty of emergency medicine (EM), the Australasian College for Emergency Medicine planned to introduce high-stakes simulation-based summative assessments in the form of Objective Structured Clinical Examinations into the Fellowship examination starting from 2015. Regarding pediatric EM, there is an increasing use of simulations in the assessment of candidates for the programs. However, the existing literature about the experience around using simulation-based assessments in high-stakes EM assessment is quite limited. This article intended to shed a light on the status of simulation-based assessments in high-stakes settings and to drive best practices that should be applied in high-stakes EM assessments. Specific attention is paid here toward using simulations for credentialing and licensure purposes that lead to the decision of pass/fail that permits HCPs to practice on real patients since those settings are believed to provide a more authentic reflection of high-stakes assessments.
The aim of this study is to provide a scoping review of the currently existing literature about the utilization of different simulation modalities in high-stakes evaluations to describe its challenges, best practices, and future directions. Since the aim of this research is not to get a summarized answer, and the research topic is broad, the scoping review approach was used to review the exciting literature instead of a systematic review or meta-analysis.
| Materials and Methods|| |
Scoping review is an approach to synthesize evidence by providing an overview of the existing research on a, usually, broad research question with limited information in the literature., This study follows steps of conducting a scoping review which include: defining research question, searching relevant studies, selecting relevant studies, charting data, and describing results.
First, the research question was formulated as the following: What are the best practices, challenges, and future directives of using multiple modalities of simulation in high-stakes clinical assessments, and how do these relate to the specialty of EM? Second, relevant studies were initially screened using terms were: “simulation,” “high-stakes,” and “assessments.” The term “summative” was also added after the initial screening of titles. Furthermore, the search term “board-certification” was added later to further focus on the high-stakes aspect of the aim.
Third, regarding selecting relevant studies, the search covered the common well-known databases in medical education: PubMed, Education Resource Information Center, Cumulative Index to Nursing and Allied Health Literature, and the Cochrane library. The intention was to include all studies discussing the application of any modality of simulation in high-stakes clinical assessment. The exclusion criteria were:
- Studies which are not for credentialing/licensure purposes, reentry, and selection processes
- Studies which are mainly intended for patient safety and quality improvement
- Studies with a focus on the formative/feedback aspects of assessment
- Studies about computer-based simulated cases since these are not considered a high-fidelity simulation and it can be considered with the computer-delivered MCQs examination
- Studies without any of the well-known modalities of clinical simulation
- Studies from specialties other than medicine or nursing
- Abstract or full-text article is not available for analysis
- Remotely conducted (virtual) examination.
Fourth, regarding data charting, the search was conducted on July 15, 2021, from 8 am to 4 pm, and included studies published from 1994 to July 2021, since the modern practice of simulation had started in the early 90s. The electronic searches in many databases start with that year, and it is thought that searching for articles older than this year would not reflect the current understanding of educational best practices. Articles were fully read searching for data that fits under ant of the three key themes of the study (challenges, best practices, and future directives); extracted data were categorized accordingly. Finally, results were summarized and reported in tables in concordance with the three key themes.
A certain strategy was used to perform this scoping review, [Figure 1] for the outlines of search strategy. The initial screening in databases was using the phrase “simulation assessment” to get a general impression about exciting data. All types of publications were included initially (including editorials, reviews, etc.). The search needed to be refined using the Booleans and specific key terms. After consulting with the librarian, the decision was to use the following search strategy: “simulation” AND “assessment” AND “summative” OR “high-stakes” OR “board certification.” All studies written in English, with or without another language, were included.
Then, the titles were screened using all the exclusion criteria. Second, using the same exclusion criteria, the abstracts of these articles were screened to yield articles to be read in full. Then, the references were manually reviewed for possible articles to be included. Specific attention was paid to articles directly related to EM to be comprehensively analyzed.
| Results|| |
The initial screening for simulation assessment in the databases resulted in 1, 11, 253 articles using the following search strategy: Simulation AND assessment AND “summative” OR “high-stakes” OR “board certification;” this resulted in a total of 3711 articles included for title check.
After applying the exclusion criteria, 91 articles remained after checking the titles. Then, after screening the abstracts of these articles, only 44 articles remained to be read in full. When references were manually reviewed further 3 articles were added. All studies found were written in English, although three articles were multilingual. Specific attention was paid to 4 articles found to be directly related to EM.
Finally, this search yielded 47 articles, which were included for analysis [Table 1] and [Table 2]. The full text of those articles was retrieved, and they were read in full. All articles discussed at least one aspect of at least one modality of simulation applicable to high-stakes clinical assessments. Most articles were about credentialing/licensure purposes of assessment; few articles were related to reentry or selection. Most of the articles which discuss simulation-based assessments were too general. However, many were specialty-specific, with articles on the topic of anesthesia being the most abundant among those. There were variations in different variables between all articles as presented in [Table 1] and [Table 2].
Most publications gave a general overview and were not specific to any specialty. This could be due to the scarcity of publications describing practical experiences coming from clinical specialty; this is more obvious when talking about EM. It was not surprising to find out that most of the specialty-specific publications discuss the topic of anesthesia, as it is the leading specialty in the field of clinical simulation. In addition, it was not unexpected to find that SP was the most common modality of simulation to be applied in high-stakes assessments with well-established standards and relatively long experience of the application. Most of the studies were about credentialing/licensure purposes (which aim to assess the competency of healthcare providers to practice on real patients).
Many modalities of simulation have been used in training and assessments. The most commonly used modality in high-stakes assessments is the SP. Other modalities include high-fidelity mannequins, part-task trainers, virtual simulators, and hybrid simulation (a combination of any two or more of these modalities)., Only few studies describe the use of more than one simulation modality in high-stakes assessments, and almost all of them are from fields outside of EM.
While significant variability in articles' types, methodology, and the manner of presentation was found, the following themes emerged: Challenges, best practices, and future directions. I decided to correlate the findings to these themes; when in doubt, the opinion of a 2nd reviewer expert in the field was taken (acknowledged). The best practices were elaborated on extensively, especially the psychometric measures, across the publications. Hence, it was decided to subdivide this theme into subthemes. For the sake of practicality, this theme got divided into the following subthemes: Preassessment, during the assessment, and postassessment. A summary of best practices is provided in [Table 3]. Further discussion of the findings is provided in the discussion section.
|Table 3: Classification of best practices based on the stage of assessment implementation|
Click here to view
Because the purpose of high-stakes assessment affects the ways in which the assessment should be designed, the assessment should be planned carefully to reflect a true evaluation of the desired tasks from the examinee.
Much logistics should be considered during the lifetime of the assessment. Usually, each station will take 5–10 min; in acute care medicine, timing can be shortened to 5 min. Video and audio recording is a quality measure to be considered during the examinations., If the examination is conducted in multiple centers, the timing and operations should be synchronized and standardized; real-time technical support should be available in all centers. Measures to prevent cheating should be reinforced. The responsibility for each room during the examination should be designated, i.e., the examiner, the SP, or the technician.
Scores collected should facilitate psychometric analysis to support the validity of the assessment. Scores post hoc analysis should be conducted; driven data should reflect the desired outcomes; exploring for confounding factors that influence the outcome. Issues that appeared during the exams should be reviewed and actions should be taken based on that.
Although simulation technology has shown tremendous development, there are still limitations in displaying certain manifestations. The high cost is a major challenge. Rater recruitment and training are time- and effort-consuming processes. The variability between the training centers imposes a big challenge to be addressed when preparing for the examinations. Another challenge is the presence of a wide variety of assessment tools with low validity.
With the expansion of aspects of clinical competencies to be assessed, there will be a growing need for validated assessment tools in the field of EM. The interest in applying entrustable professional activities would reflect on the design of the assessment. With the increasing popularity of simulation, mobile units of simulation assessment tools may get integrated into the assessment system soon.
| Discussion|| |
The increasing utilization of high-stakes assessment is reflected in this review by the increasing number of the found related studies in the past 5 years. This increasing utilization is associated with growing experience which is presented by the diversity of findings related to the three main themes of this review. Overall, most of the studies were about licensure/credentialing; which is a finding consistent with what has been described in the literature before. On the other hand, this review is pointing to a limited experience in using simulation for the selection process or reentry/renewal purposes; however, it is expected to be see more implementation of simulation for those purposes as the scope of utilization of simulation is increasing. Most of the experience with using simulation in these settings was for postgraduates; this can be explained by the need to get a higher level of assessments as the level of the targeted group is getting higher.
It is important for stakeholders to be familiar upfront with the anticipated obstacles that may hinder the application of a high-quality simulation-based high-stakes assessment [Table 3]. Regarding the first theme about challenges to implement simulation in high-stakes examinations, related articles found were prevalent in this review. This is an expected finding as using simulation is logistically demanding, especially in high-stakes situations. The application of quality assessment using multiple modalities of simulation faces many challenges. These challenges can delay the translation of the rapid growth of simulation-based examinations (SBEs) into the area of high-stakes assessments. Some of the challenges discovered by this review are common to most types of assessments, whereas others are unique to simulation-based assessments. In addition, it was found that the higher the stakes of the assessment the greater the challenges will get, and some are easier than others to overcome.
As expected, most of the articles gave recommendations regarding best practices of implementing simulation in high-stakes assessments; however, seldom it is to find a comprehensive list of best practices in a single article. This reflects the width and complexity of this subject and further justifies the need for this review and similar studies in this area. As this review demonstrates, to optimally assess learners, it is recommended to utilize multiple modalities of assessment to evaluate different aspects of the examinee's competency. Hence, a simulation-based assessment is not recommended to be used as the sole method to judge the learner's competency.
It is paramount to give special efforts for the planning phase when designing the scenarios.,, For example, designing an assessment for board certification would be different from an assessment meant for a selection for residents., Furthermore, planners should choose the right modality of simulation for the tasks to be evaluated. For example, high-fidelity mannequins and part-task trainers can assess the skills of airway management. The design should reflect the real practices; designers should aim for the highest possible fidelity. The assessment center should have the structure and sources to promote the highest standards. Cautious security measures should be taken to prevent a breach of the exams' content, especially after piloting the examination., The portrayal of scenarios should be clear and user-friendly for SPs, examiners, and technical support staff.
A very important part of the planning is to determine the metrics of the assessment. Assessment tools, wither checklists or global ratings, should be carefully chosen/designed to facilitate the production of high standards metrics. This decision should be done by the examination committee. Raters should be trained and qualified based on a preset protocol. Measures to assure high reliability and validity (e.g., increasing the number of scenarios, designing task-specific stations, and standardizing the examinations administration) should be given a special dedication., For high-stakes assessments, there should be targeted metrics; for example, Kappa should be >0.75. Metrics such as reliability, validity, accessibility, and feasibility are important parameters to ensure the quality of the assessment process.
The real-time implementation of the modalities at the time of the exam gets the most attention; however, this step does not come in a solo. Although it has a strong impact on then impression about the assessment process, it does not reflect the quality of the whole process. Some specifics in relation to this phase were mentioned in detail in the studies involved in this review. Those parameters can help in benchmarking, however, there is room for development and improvements to suite different needs.
Test's quality assurance should be followed to go beyond the time of the assessment itself. There should be a system to evaluate the assessment process. A continuous improvement plan should be in place. Furthermore, manuals of protocols should be updated. Results and the appeal process should be demonstrated to the learners as per the plan. The process of remediation should be conducted according to the plan and events raised during the assessment. All previously mentioned points should be worked on.
Since simulation-based assessment is an evolving area, it is important to know the future direction of the field. As demonstrated in this review, the application of simulation in different disciplines is growing. The increasing attention toward competency-based assessments will increase the call for using simulation in high-stakes assessments. There are many opportunities for growth of this modality of assessment with the rapid growth of the applications of simulation. For EM specialists, this should open a growing opportunity for adapting experiences of other clinical disciplines.
This study is not without limitations. As this field is relatively new, the number of relevant studies was limited. Because of the newness of this field, there was still a significant amount of differing perspectives, and general best practices were yet to consistently emerge in the literature. The heterogeneity and complexity of the articles included also limited the ability to collect all the detailed information presented. To overcome this limitation, I tried to follow a chronological approach and to include the most prevalent topics presented in the literature. There could be valuable information about challenges, best practices, and future directions that could be derived from formative assessment literature which was excluded in our search. Furthermore, half of the articles reviewed here included some discussion about formative assessments. However, the novelty in this study is providing a layout for guidance to use all modalities of simulation to assess the competency of healthcare providers. There is no other study in the literature collecting the exact same data.
| Conclusions|| |
The use of simulation for high-stakes assessment is promising. There should be a balance between the need to measure specific competencies with the impeded limitations of simulators. Using a simulation-based assessment should not be the sole method to assess learners' performance; it should be one part of many for a holistic approach. Because of the impeded obstacles and challenges, the growth and spread of such an experience are relatively slow when compared to SBE. To get stakeholders' approval, this method of assessment should demonstrate reliability, validity, accessibility, cost, and feasibility. Stakeholders should aim to establish an accrediting process to train and credential all simulation-based high-stakes assessment programs. Hence, the simulation has a great potential to promote safe clinical practice on patients by assuring the competency of healthcare providers serving the community.
Deborah Navedo would like to thank Deborah Navedo, PhD, CPNP, FNAP for her assistance and guidance in this scholarly project.
Financial support and sponsorship
Conflicts of interest
There are no conflicts of interest.
| References|| |
Calhoun AW, Bhanji F, Sherbino J, Hatala R. Simulation for high-stakes assessment in pediatric emergency medicine. Clin Pediatr Emerg Med 2016;17:212.
Rizzolo MA, Kardong-Edgren S, Oermann MH, Jeffries PR. The national league for nursing project to explore the use of simulation for high-stakes assessment: Process, outcomes, and recommendations. Nurs Educ Perspect 2015;36:299-303.
Kordi M, Fakari FR, Mazloum SR, Khadivzadeh T, Akhlaghi F, Tara M. Comparison of the effect of web-based, simulation-based, and conventional training on the accuracy of visual estimation of postpartum hemorrhage volume on midwifery students: A randomized clinical trial. J Educ Health Promot 2016;5:22.
Kordi M, Erfanian F, Fakari FR, Dastfan F, Nejad KS. The comparison the effect of training by means of simulation and oral method on midwives' skill in management of shoulder dystocia. J Educ Health Promot 2017;6:50.
O'Leary F. Simulation as a high stakes assessment tool in emergency medicine. Emerg Med Australas 2015;27:173-5.
Chambers KA, Boulet JR, Gary NE. The management of patient encounter time in a high-stakes assessment using standardized patients. Med Educ 2000;34:813-7.
Ziv A, Rubin O, Sidi A, Berkenstadt H. Credentialing and certifying with simulation. Anesthesiol Clin 2007;25:261-9.
Hatala R, Kassen BO, Nishikawa J, Cole G, Issenberg SB. Incorporating simulation technology in a canadian internal medicine specialty examination: A descriptive report. Acad Med 2005;80:554-6.
Kardong-Edgren S, Hanberg AD, Keenan C, Ackerman AA, Chambers K. A discussion of high-stakes testing: An extension of a 2009 INACSL conference roundtable. Clin Simul Nurs 2011;7:e19.
Leblanc VR. Review article: Simulation in anesthesia: State of the science and looking forward. Can J Anaesth 2012;59:193-202.
Sucharew H, Macaluso M. Progress notes: Methods for research evidence synthesis: The scoping review approach. J Hosp Med 2019;14:416-8.
Pham MT, Rajić A, Greig JD, Sargeant JM, Papadopoulos A, McEwen SA. A scoping review of scoping reviews: Advancing the approach and enhancing the consistency. Res Synth Methods 2014;5:371-85.
Furman GE, Smee S, Wilson C. Quality assurance best practices for simulation-based examinations. Simul Healthc 2010;5:226-31.
Boulet JR. Summative assessment in medicine: The promise of simulation for high-stakes evaluation. Acad Emerg Med 2008;15:1017-24.
Petrusa ER. Current challenges and future opportunities for simulation in high-stakes assessment. Simul Healthc 2009;4:3-5.
Bensfield LA, Olech MJ, Horsley TL. Simulation for high-stakes evaluation in nursing. Nurse Educ 2012;37:71-4.
Adamo G. Simulated and standardized patients in OSCEs: Achievements and challenges 1992-2003. Med Teach 2003;25:262-70.
Weinger MB, Banerjee A, Burden AR, McIvor WR, Boulet J, Cooper JB, et al.
Simulation-based assessment of the management of critical events by board-certified Anesthesiologists. Anesthesiology 2017;127:475-89.
Norcini J, Anderson B, Bollela V, Burch V, Costa MJ, Duvivier R, et al.
Criteria for good assessment: Consensus statement and recommendations from the Ottawa 2010 conference. Med Teach 2011;33:206-14.
Cannon K, Hartsell Z, Ivanov I, Charles J, Joshi H, Blair J, et al.
Perceptions of internal medicine residency program candidates on the use of simulation in the selection process. J Grad Med Educ 2014;6:338-40.
Collins JP, Harden RM. AMEE medical education guide No. 13: Real patients, simulated patients and simulators in clinical examinations. Med Teach 1998;20:508.
Boulet JR, Murray D. Review article: Assessment in anesthesiology education. Can J Anaesth 2012;59:182-92.
Tavakol M, Mohagheghi MA, Dennick R. Assessing the skills of surgical residents using simulation. J Surg Educ 2008;65:77-83.
Swing SR. Assessing the ACGME general competencies: General considerations and assessment methods. Acad Emerg Med 2002;9:1278-88.
Srinivasan M, Hwang JC, West D, Yellowlees PM. Assessment of clinical skills using simulator technologies. Acad Psychiatry 2006;30:505-15.
Boulet JR, De Champlain AF, McKinley DW. Setting defensible performance standards on OSCEs and standardized patient examinations. Med Teach 2003;25:245-9.
Glavin RJ, Gaba DM. Challenges and opportunities in simulation and assessment. Simul Healthc 2008;3:69-71.
Sadeghi M, Taghva A, Mirsepassi G, Hassanzadeh M. How do examiners and examinees think about role-playing of standardized patients in an OSCE setting? Acad Psychiatry 2007;31:358-62.
Leo N, Foot C, Venkatesh B, Corke C, Manoj S, Lucey M, et al.
“High-stakes assessment of the non-technical skills of critical care trainees using simulation: Feasibility, acceptability and reliability. Crit Care Resusc 2014;16:6-12.
Harvey P, Radomski N. Performance pressure: Simulated patients and high-stakes examinations in a regional clinical school. Aust J Rural Health 2011;19:284-9.
Berkenstadt H, Ziv A, Gafni N, Sidi A. Incorporating simulation-based objective structured clinical examination into the Israeli National Board Examination in Anesthesiology. Anesth Analg 2006;102:853-8.
Turner JL, Dankoski ME. Objective structured clinical exams: A critical review. Fam Med 2008;40:574-8.
McNaughton N, Ravitz P, Wadell A, Hodges BD. Psychiatric education and simulation: A review of the literature. Can J Psychiatry 2008;53:85-93.
Feldman M, Lazzara EH, Vanderbilt AA, DiazGranados D. Rater training to support high-stakes simulation-based assessments. J Contin Educ Health Prof 2012;32:279-86.
Levine AI, Schwartz AD, Bryson EO, Demaria S Jr. Role of simulation in U.S. physician licensure and certification. Mt Sinai J Med 2012;79:140-53.
Schuwirth LW, van der Vleuten CP. The use of clinical simulations in assessment. Med Educ 2003;37 Suppl 1:65-71.
Yudkowsky R. Should we use standardized patients instead of real patients for high-stakes exams in psychiatry? Acad Psychiatry 2003;26:187-92.
Oermann MH, Kardong-Edgren S, Rizzolo MA. Summative simulated-based assessment in nursing programs. J Nurs Educ 2016;55:323-8.
Goldberg A, Samuelson S, Levine A, DeMaria S. High-stakes simulation-based assessment for retraining and returning physicians to practice. Int Anesthesiol Clin 2015;53:70-80.
Holmboe E, Rizzolo MA, Sachdeva AK, Rosenberg M, Ziv A. Simulation-based assessment and the regulation of healthcare professionals. Simul Healthc 2011;6 Suppl:S58-62.
Banerjee A. Using simulation for primary certification. Int Anesthesiol Clin 2015;53:42-59.
DeMaria S Jr., Samuelson ST, Schwartz AD, Sim AJ, Levine AI. Simulation-based assessment and retraining for the anesthesiologist seeking reentry to clinical practice: A case series. Anesthesiology 2013;119:206-17.
Orledge J, Phillips WJ, Murray WB, Lerant A. The use of simulation in healthcare: From systems issues, to team building, to task training, to education and high stakes examinations. Curr Opin Crit Care 2012;18:326-32.
McGrath JL, Taekman JM, Dev P, Danforth DR, Moheb D, Kman N, et al
. Using virtual reality simulation environments to assess competence for emergency medicine learners. Acad Emerg Med 2018;25:186-95.
Warner DO, Isaak RS, Peterson-Layhe C, Lien CA, Sun H, Menzies AO, et al
. Development of an objective structured clinical examination as a component of assessment for initial board certification in anesthesiology. Anesth Analg 2020;130:258-64.
Bauer D, Lahner FM, Schmitz FM, Guttormsen S, Huwendiek S. An overview of and approach to selecting appropriate patient representations in teaching and summative assessment in medical education. Swiss Med Wkly 2020;150:w20382.
Nadir NA, Hart D, Cassara M, Noelker J, Moadel T, Kulkarni M, et al.
Simulation-based remediation in emergency medicine residency training: A consensus study. West J Emerg Med 2019;20:145-56.
Monti M, Klöckner-Cronauer C, Hautz SC, Schnabel KP, Breckwoldt J, Junod-Perron N, et al.
Improving the assessment of communication competencies in a national licensing OSCE: Lessons learned from an experts' symposium. BMC Med Educ 2020;20:171.
Arrogante O, González-Romero GM, López-Torre EM, Carrión-García L, Polo A. Comparing formative and summative simulation-based assessment in undergraduate nursing students: Nursing competency acquisition and clinical simulation satisfaction. BMC Nurs 2021;20:92.
Isaak R, Stiegler M, Hobbs G, Martinelli SM, Zvara D, Arora H, et al.
Comparing real-time versus delayed video assessments for evaluating ACGME sub-competency milestones in simulated patient care environments. Cureus 2018;10:e2267.
Sabzwari SR, Afzal A, Nanji K. Mimicking rashes: Use of moulage technique in undergraduate assessment at the Aga Khan University, Karachi. Educ Health (Abingdon) 2017;30:60-3.
[Table 1], [Table 2], [Table 3]