|
Designing a comprehensive clinical competency test for operating room technology student: Using Delphi technique and CIPP model evaluation
Hamideh Yazdimoghaddam1, Ezat Samadipour2, Fatemeh Ghardashi3, Fateme Borzoee4, Roya Akbarzadeh5, Roghayeh Zardosht6, Mohammad Hosseinzadeh Hesari7, Sedigheh Khalili7
1 Non-Communicable Diseases Research Center, Department of Operating Room, Faculty of Paramedices, Sabzevar University of Medical Sciences, Sabzerar, Iran
2 PhD Health in Disasters and Emergencies, School Paramedical, Sabzevar University of Medical Sciences, Khorasan Razavi, Iran
3 PhD in Health Research and Technology Management, Assistant Professor of SUMS, Non-Communicable Diseases Research Center, Vice Chancellor of Education and Research, School of Paramedical Sciences, Sabzevar University of Medical Sciences, Sabzevar, Iran
4 Non-communicable Disease Research Center, Department of Operating Room, Faculty of Paramedics, Sabzevar University of Medical Sciences, Sabzerar, Iran
5 Non-Communicable Diseases Research Center, Department of Anesthesia, Faculty of Paramedices, Sabzevar University of Medical Sciences, Sabzerar, Iran
6 Iranian Research Center on Healthy Aging, Department of Operating Room, Faculty of Paramedices, Sabzevar University of Medical Sciences, Sabzerar, Iran
7 Department of Operating Room, School of Allied Medical Sciences, Non-Communicable Diseases Research Center, Sabzevar University of Medical Sciences, Sabzevar, Iran
| Date of Submission | 01-Dec-2020 |
| Date of Acceptance | 14-Dec-2020 |
| Date of Web Publication | 30-Jun-2021 |
Correspondence Address:
Roghayeh Zardosht
Shohadayeh Gomnam Blv. School of Paramedical, Sabzevar University of Medical Sciences, Khorasan Razavi
Iran
 Source of Support: None, Conflict of Interest: None  | Check |
DOI: 10.4103/jehp.jehp_1563_20
BACKGROUND: The ultimate goal in educating medical students is to train skilled workforce who by obtaining the required knowledge of the discipline, personal and professional skills, and attitudes to enter the national health-care system. This study was conducted with the goal of designing a comprehensive clinical competency test of operating room technology student using the Delphi technique, and then, the clinical skills of the operating room technology students of the paramedical school were assessed using the CIPP evaluation model.
MATERIALS AND METHODS: The present study was conducted in the University of Medical Sciences to evaluate clinical skills with Delphi technique and CIPP evaluation model. This was a mixed methods study. Purposive sampling method was employed in the qualitative stage, and a census was conducted in the quantitative part. A comprehensive clinical competency test was designed in this research using the Delphi technique and was conducted with 18 students. Then, clinical skills evaluation was performed by descriptive-analytical statistical tests and evaluator's observation using the CIPP model.
RESULTS: The comprehensive clinical competencies test of the surgical technology students was designed in four sections: content, evaluation method, test conditions, and the criteria for passing the comprehensive test. The results in the implementation stage showed that the operating room technology students had a range of excellent to weak performances in exhibiting basic skills at different levels.
CONCLUSIONS: The results of the present study showed that the comprehensive test, designed based on the Delphi technique of experts, and using the CIPP model can be a good criterion for the evaluation of the operating room technology students before entering the clerkship.
Keywords: Clinical competency, Delphi techniques, outcome and process assessment
How to cite this article:
Yazdimoghaddam H, Samadipour E, Ghardashi F, Borzoee F, Akbarzadeh R, Zardosht R, Hesari MH, Khalili S. Designing a comprehensive clinical competency test for operating room technology student: Using Delphi technique and CIPP model evaluation. J Edu Health Promot 2021;10:240 |
How to cite this URL:
Yazdimoghaddam H, Samadipour E, Ghardashi F, Borzoee F, Akbarzadeh R, Zardosht R, Hesari MH, Khalili S. Designing a comprehensive clinical competency test for operating room technology student: Using Delphi technique and CIPP model evaluation. J Edu Health Promot [serial online] 2021 [cited 2023 May 30];10:240. Available from: https://www.jehp.net//text.asp?2021/10/1/240/320254 |
| Introduction | |  |
In order to evaluate the clinical skills, a comprehensible conceptual framework is needed, in which the education process is evaluated. One of the important kinds of evaluation, given the goal or the subject, is the evaluation of students, which is an important aspect in the process of educational activities. This type of evaluation process enables us to determine the strengths and weaknesses based on the results and by reinforcing the positive aspects and eliminating the weaknesses, proper steps can be taken to reform and transform the education system.[1],[2] Evaluation is an important part of the learning process in education and students refer to it as the main incentive that guides their learning.[3] Evaluation methods are important for students. They show their responses to the curriculum and determine the learning methods for students. Learning method is a dynamic feature that constantly changes based on the student's perception of the learning environment. The way the evaluation is performed has been reported as one of the most important factors that affect the way students learn.[3],[4]
Bachelor of Science in Operating Room Technology in Iran is a branch of medical sciences, in which students get to know the new principles and modern surgical technologies in specialized surgeries and learn how to take care and handle the patient before, during, and after the operation. Graduates in this major are the part of the surgery team that work in surgery rooms of hospitals, diagnostic-therapeutic intervention wards, and ambulant care centers and assist in performing a surgery operation with desirable results. Clinical environment has now been widely accepted as a key environment to train students, and it assists students in connecting theoretical education to clinical performance. However, many factors affect the clinical education in the meantime. Researchers have emphasized many times that the quality of the clinical education environment is an extremely reliable indicator for the quality of the entire training course.[5]
Clinical education can be considered as a set of learning activities in the clinical environment, in which the clinical instructor and the student make the same amount of contribution and the goal is to create the measureable changes in the student to perform clinical care.[6] Competency is a series of integrated capabilities consisted of knowledge, skill, attitude for accurate performance in the workplace, problem solving, and effective conduct in a particular profession, organization, job, and role. Weakness in different aspects of basic and important clinical competencies of Operating Room Technology students can endanger the patient's health. The role of competency in education has grown significantly and represents the gap between education and the clinical environment.[5],[7] The evaluation process identifies, describes, estimates, affects the comprehensive usefulness of education and is a proper tool to modify the training goals, plans, and methods. Evaluation is essential in educational technology as a key activity and achieving educational goals in impossible without it.[8] Employing evaluation is the most efficient method to improve the quality and effectiveness of the education. Clinical evaluation is a systematic process to collect, analyze, and interpret information to make a judgment on whether the student has achieved the goals and on the quality of his/her work in the clinical environment. Evaluation is an important and inseparable part in every educational program. The results of different researches have shown that most Operating Room Technology students believe that they were not able to acquire the necessary skills for employment during their educational courses. Currently, the Operating Room Technology students get evaluated by their instructor at the end of their internship course. However, there is not any comprehensive evaluation that can assess them in terms of their skills or clinical competencies.[6],[7],[9],[10],[11] Such evaluation requires the employment of methods and tools that can generalize the skill level, but according to the evidence, there has not been a single method presented in this field.[12] The revised curriculum of the B.S. of Operating Room Technology (2014) specifies the evaluation of the students as continuous, term, and final assessments.[13] Therefore, it seems necessary to design a scientific and comprehensive test that can evaluate the clinical competency of students before graduation. Universities are obliged to train graduates who have adequate skills in preventing, treating, and promoting the health of their community. Because by training these workforces, we will be able to take good and effective care of patients before, during, and after an operation.[12]
The evaluation commonly used in clinical settings is unstructured, and sometimes arbitrary and subjective. The results indicate that Operating Room Technology students were not satisfied with their evaluations and they believe the reason is the irrelevancy of the objectified tools in the clinical evaluation to practical conditions, artificiality of the evaluations, lack of adequate supervision on the clinical education process, and unavailability of accurate, objective, and clear criteria.[14],[15],[16],[17],[18],[19],[20] Guaranteeing the care quality and making sure the surgical technologists have the necessary clinical skills are the main concerns of instructors and officials in the paramedical faculties and health-care provider systems.[20] If the comprehensive clinical competency tests are not accurately designed and implemented, they can have devastating effects such as loss of educational resources, suspicion, and mistrust. Evaluation in a clinical environment is necessary and essential and without it, we will not be able to obtain the evidence that testify to the competency of the students. Professional competency is a multi-dimensional concept that includes all human components and technical and situational skills in care and education.[5] Therefore, a comprehensive clinical competency test must be designed in the curriculum to evaluate the clinical skills of the students. By reporting the results to the instructors and students, it will lead to improvement and correction of defects and will increase the acquisition of skills in the students. Given the absence of a model, it is necessary to use the opinions of experts in this field. Designing the indicators of a comprehensive test that is adjusted using the opinions of education experts, global approaches, and local conditions of the country can be of great use for paramedical faculties, because clinical training and preparing students to take various roles in health and medical services are among the tasks and goals of these faculties. The Delphi method (focus group) is one of the systematic research methods in which the opinions of a group of experts on a subject are extracted, and the method is implemented based on the consensus of experienced experts using rounds of questionnaires. This method has been accepted in medical studies and health-care services and is employed by researchers.[21] Despite the good reception by the instructors, the traditional evaluation has not been effective and cannot provide a comprehensive evaluation of the student. Thus, using evaluation models, instead of the traditional evaluation, ensures the evaluator about whether or not the student has obtained proper clinical skills and if the student is able to utilize the skills acquired in the university years when confronting the changes in the actual clinical environment.
There are numerous methods and models to evaluate the educational program. One of these models is the CIPP Evaluation Model. The CIPP model was first developed by Cuba and then Daniel Stufflebeam in 1960. CIPP is an abbreviation consisted of the first letters of four words: Evaluation of context (C) which means determining the requirements to make decisions on the goals of the program. Evaluation of the input (I): examining all the factors that affect goal achievement and selecting or developing a program with the best goals specified in the context evaluation. Process (P) evaluation: understanding how to implement the program in the real world and recognizing successes and failures of the implemented program. Product (P) evaluation: Assessing the results of program implementation and recognizing the relation between expectations and the actual results. The most important goal of CIPP-based evaluation is to improve the program performance. Using the CIPP evaluation model, in addition to being able to examine the clinical skills of the students, the educational problems and shortcomings can be discovered and efforts can be made to remove the gap between theory and practice.[22] The traditional evaluation that uses no particular model cannot help us in determining whether or not the students have achieved the specified goals of clinical skills and is not effective in identifying goal achievement and educational strengths and weaknesses. Therefore, in addition to designing a comprehensive clinical competency test of Operating Room Technology using the Delphi technique, the CIPP model was employed to assess the clinical skills of Operating Room Technology students of the paramedical faculty.
| Materials and Methods | | |
This is a mixed methods research, in the qualitative stage, the content of the comprehensive clinical skills test was designed using the Delphi technique. A group of experts in the Operating Room Technology discipline (12 individuals) was used to extract the opinions, which based on the inclusion criteria consisted of the dean of the paramedical school, head of the Operating Room Technology department, Operating Room Technology faculty members, the head nurse of the operating rooms of teaching hospitals, experienced operating room staff, and the education expert. The inclusion criteria were based on the occupation and position of the experts, including Operating Room Technology faculty members with >3 years of experience in clinical teaching of this major, and the head nurse of one of the clinical wards with >1-year experience as a head nurse in teaching hospitals. After obtaining written consent, through four questionnaire rounds, in-person meetings, and giving feedbacks, we reached a consensus in the opinions of the panel members in designing the comprehensive test. In the Delphi technique and in order for experts to enter the study, they need to have four characteristics to be a panelist which includes knowledge on the subject, inclination, sufficient time to participate, and effective communicative skills.[21] For data collection, demographic information form and basic and special skill assessment checklists were utilized according to the syllabus. Data collection instruments included the indicators of the comprehensive test which was conducted in two steps and four rounds [Figure 1].
First stage
In order to extract the comprehensive test components, researchers conducted an extensive literature review in journals and articles in scientific and reliable domestic and foreign databases (English and American universities). Therefore, a text was prepared that included the research summary and topics that require the opinion of the experts and was sent to these people and then follow-ups were done in 1-week intervals through E-mail or in-person meetings. The opinions were collected and were sent to the experts to confirm or to give a final statement. Thus, we corresponded with the Operating Room Technology departments in paramedical schools to make sure that none of them has performed a comprehensive test for the Operating Room Technology department. Then, the curriculum of the B.S. of Operating Room Technology was studied, which was approved in the 58th session of the High Council for Medical Sciences Planning on January 4, 2015. Then, this test was performed with the aim of conducting a final evaluation, which is mentioned in the comprehensive evaluation part of the curriculum. Considering that the curriculum of the B.S. of Operating Room Technology was revised in 2014 and implemented since 2015, this test was designed for final evaluation of the practical skills of the students before entering internship with the goal of empowering them in practice and was implemented and assessed at the beginning of academic year (before the clerkship of the students) according to the following process.
The general structure of the comprehensive test has four sections: Content, evaluation method, testing conditions, and acceptance criteria.
In order to design the comprehensive model, the following steps are recommended:
- Group design and holding group meetings
- -Assessing the needs of the target group (student) and surveying the colleagues by focusing on practical and scientific capability of the students to enter clerkship
- An extensive literature review in journals and articles of scientific and reliable domestic and foreign databases (English and American universities) to extract the components of the comprehensive clinical competency test and their characteristics
- Holding a Delphi session: In this study and based on the objective, the qualified individuals are selected according to the inclusion criteria and enter the study. Based on the studies, the number of experts in the same disciplines is 10–30 individuals and individuals from different professions in each group are 5–10 people.[5] In this study, the experts included 12 individuals including the dean of the paramedical faculty, head of the Operating Room Technology department, faculty members of the Operating Room Technology department, supervisors of operating rooms in teaching hospitals, experienced operating room staff, and the education expert
- Examining the curriculum of the B.S. of Operating Room Technology approved by the 58th session of the High Council for Medical Sciences Planning on January 4, 2015
- Holding a session on determining test stations and designing the scenarios
- Developing educational content for the comprehensive test in four rounds.
First round
the first round was held with the presence of eight faculty members of the Operating Room Technology department and decisions were made regarding the indicators and components of the comprehensive test which is supposed to be performed in the form of an Objective Structured Clinical Examination (OSCE) test. Such that in the meetings, a brief report of the research and the results was reported and then the research results were discussed and debated.
In the next session, the content of tests and sources were discussed and talked about. Then, we studied to make decisions regarding the evaluation method and different methods of evaluation. The most appropriate way to consult the experts was included in the form, consisted of discussions on the test stations and sources, which was sent to faculty members and head supervisors through E-mail to ask for their opinions. Follow-ups were done in 1-week intervals through E-mail or in-person meetings. The opinions were collected and were sent to the experts again to confirm to give a final statement. After collecting the final comments, the focus group session was held.
Second round
In six general areas: Topics of stations, test sources, test conditions and contents, evaluation method, implementation procedure, and the acceptance criteria were examined and decided. Regarding the acceptance criteria, various methods of determining the minimum passing score were studied, and the best methods were extracted and included in the initial form based on the articles.[23],[24],[25] Regarding the test conditions and considering the comprehensive tests of other university majors (midwifery, medicine, and PhD courses), the colleagues were asked to prepare a checklist and a scenario for stations given their profession. This list was sent to colleagues for validity, and its content validity was checked.
Third round
This round was held with the presence of all the faculty members of the Operating Room Technology department, and the six general areas were again reviewed, and the number of stations was also determined for the practical test which is going to be an OSCE test. The stations include nursing skills (vital signs, urinary catheterization, venous catheterization, NGT), and special operating room skills (getting the medical history of the patient, establishing communication and patient's admission to the operating room, surgical scrub, prep and drape, wearing a scrub and gloves, recognizing surgical sets and instruments, suture, surgical dressing). Some stations such as prep and drape, suture and surgical dressing require scenario designing and therefore, a few scenarios were presented and the checklists were filled out based on them. The opinions of the faculty professors were employed to examine the content validity and internal reliability using the Cronbach's alpha coefficient of the checklists. The validity was reported as suitable and the alpha coefficient of the checklists was above 70%.
Regarding the order of the stations, we asked our colleagues regarding the required time for each station and the resting time between the stations. Based on the agreement, the required time was considered for each station. Obtaining the minimum score was considered as the passing condition in each practical station based on literature review and comments made by colleagues. For some stations (such as medical history, establishing communication, admitting the patient to the operating room, checking the vital signs), a standardized patient was used.
Fourth round
Critical review: Before performing the test and in the presence of all the experts, a decision was made about the criteria for acceptance or rejection of students based on the test contents.
The second step: The designed comprehensive test was performed with 18 B.S. students of Operating Room Technology of the paramedical School. The inclusion criteria were 6th semester students of the paramedical school before entering clerkship. Exclusion criterion was unwillingness to participate in the study. Samples were selected by a census and were evaluated based on the prepared checklists, Operating Room Technology syllabus, and the evaluator's observation. Data analysis was used SPSS software version 17; (Chicago, IL, USA).
In order to perform the clinical competency test, the OSCE stations were determined and the necessary equipment were prepared the day before the test. The professors of each station were asked to review the stations in terms of conditions and equipment based on the checklist prepared for evaluation.
Participants in the Delphi stage were 11 women and 1 man with an age average of 33–50 years old. Four of which had PhD degrees, 5 had M.Sc operating room and three individuals had Bachelor's degree. Their work experience varied between 6 and 23 years.
The topics of the clinical competency test were designed as an OSCE test in four rounds and 6 stations using the Delphi technique and the consensus of the experts in clinical training in the surgery room. The stations included nursing skills (vital signs, urinary catheterization, venous catheterization, NGT), and special operating room skills (getting the medical history of the patient, establishing communication and patient's admission to the operating room, surgical scrub, prep and drape, wearing a scrub and gloves, recognizing surgical sets and instruments, stitching, surgical dressing). The topics of the stations, test sources, conditions and content, evaluation method, implementation procedure, and acceptance criteria were determined based on the consensus of the experts. After designing the comprehensive test, the test was performed and the results of clinical skills evaluations of the Operating Room Technology students based on the CIPP model were as follows:
Context evaluation
The goals of clinical skills of Operating Room Technology students (according to the revised syllabus in 2014) were reviewed by the experienced faculty professors. Based on the expert reviews, the goals in the curriculum of B.S. of Operating Room Technology are vivid, clear and objective. Based on the comments of the experts and given the facilities and equipment of the hospitals supervised by medical universities, the requirements to achieve educational goals were identified.
Input evaluation
In the present study, human resources, environment, financial facilities and equipment can be mentioned. Employing various methods and procedures of clinical skills education, preparing the clinical environment, the facilities and the equipment in the surgery rooms of the internship hospital were reviewed. In order to perform each of the required skills, the interaction level between the head supervisors and other personnel with the students, practical readiness of the evaluator, and the evaluation procedure for the intended skills were reviewed and in case of any problem about the facilities and equipment, we held meetings with the head of hospitals and the faculty dean and the required facilities were provided.
Process evaluation
The most appropriate characteristic of the CIPP evaluation model is that based on the conditions, one or two of its four aspects can be utilized. Given the objective of this study (designing a comprehensive test), we also paid special attention to the process and product stage of this valuable model. In this OSCE test stage, the evaluator carefully observed the students from the beginning to the end of the skill performance. After designing the related checklists in the first step of designing the test, each evaluator individually examined the practical skill performance of students in his/her station, using the checklist.
Depending on the station, the student was asked to practically perform the proposed scenario related to the station's topic on a moulage or a simulated patient (in some of the stations where a simulated patient is present). Based on the developed checklist, the evaluator observed and examined the skill performance from beginning to the end. By doing each item in the checklist, the evaluator puts a positive sign in front of the item and if the students fail to do an item, they receive a negative sign.
Product or outcome evaluation
The goals of this stage include determining the extent to which the Operating Room Technology students of the paramedical school have achieved the educational goal of obtaining clinical skills, and assisting the growth and improvement of policies related to the educational program. Another goal was to figure out whether the method of teaching clinical skills should continue the way it currently is for the future students or whether the teaching method should change. To perform this step, the final outcome of the skill performed by the student was reviewed and the items that were considered were for instance whether or not the skill has been accurately completed? Did the student need any help in performing the skill? Is the student completely unable to perform this skill? Evaluation of this step was performed according to the Student's score from the checklist. This process was performed for all the students in all four steps. At the end of the skill performance and when needed, the evaluator gave feedbacks to the students and gave the necessary instructions to resolve the problem.
Implementation stage
The result of the last two parts of the CIPP evaluation including process and product (outcome) was determined. A total of 18 students participated in the study, 67% were girls, and 33% were boys. The age average and standard deviation of 20 ± 1.1 and 19 ± 1.1 were obtained for girls and boys, respectively.
The test designed to evaluate the clinical skills acquired by students was implemented before they enter clerkships (7th semester). In the evaluation of the education process and after the evaluation of all the students was finished, the scores of every checklist were reviewed. Then, according to the number of questions in each checklist and the corresponding score attached in the end, the results of the comprehensive clinical competency test are evaluated. Skill evaluated consisted dedicated operating room stations (for example; History and communication with the patient - Dressing Gan and gloves -Tool layout- Perp and drop – Scrub-Suture) and nursing skills stations (for example; Vital sign - IV line - NG tube- Fully catheter).
After announcing the results of the checklists and giving the final score to the students, the scores were assessed and decisions were made on the criteria to pass or fail students based on the decisions made in the designing stage.
| Discussion | | |
The results of the present study showed that the comprehensive test based on the Delphi technique of experts can be a suitable indicator for evaluation of the Students' skills before Intership course. Implementing a comprehensive test for bachelor students of Operating Room Technology for the first time showed us the prerequisites, facilities, equipment, and the strength and weaknesses of the test. Performing the comprehensive test showed that the Operating Room Technology students lacked some of the skills.
According to ArabKherad and HajiAghajani, the results obtained from the evaluation have been useful for curriculum planners and professors in improving teaching methods and have led to better decisions.[23],[26] Reviewing the evaluation methods to improve and revise these methods and programs leads to effective educational feedbacks and adapts them to the standards of the High Council of Planning.[27]
Investigations showed that there has not been a comprehensive test for the Operating Room Technology discipline in Iran that has been designed with the cooperation of experts and provision of precise scientific steps. Given the improvement of the evaluation system and medical sciences tests, this study was implemented for the first time in Iran, with the aim of improving the professional competency of the Operating Room Technology students.
Mazloum conducted a study with the goal of designing a comprehensive practical test of the nursing discipline (a Delphi futures study) and the results showed that the OSCE and DOPS are the best methods of clinical skills evaluation.[5]
Designing comprehensive test
The present study also showed that designing a comprehensive test based on the requirements and consensus of the experts in the field and also performing the comprehensive test is a proper tool to measure the practical skills of the Operating Room Technology students. Based on the Miller pyramid, evaluation tools such as written questions and true/false questions, matching, short-answer, multiple-choice and oral questions are effective in measuring “understanding and knowledge” and they are limited in measuring “practice and performance.” Whereas in a comprehensive test, measurement is not restricted to understanding and knowledge and students need to apply their basic knowledge to perform psychomotor and sometimes emotional skills.[28] Experts such as Harden et al., Mitchell et al., Casey et al., and Eldarir et al. believe that one of the proper evaluation methods to assess the clinical skills of the students is the OSCE method which has appropriate criteria for clinical evaluation of the students.[29],[30],[31] The results of the study by Gandomkar and Amini also showed that among the clinical skills evaluation tests, OSCE test is the most used test in the clinical medical courses in Iran.[32],[33],[34] Results of the study by Rafiee et al. imply the improperness of the evaluation methods and problems in the process of evaluations.
Implementing comprehensive test
In this study, the implementation step demonstrated that using this designed test is simple and easy and the evaluation method identifies the strengths and weaknesses of each student in a completely professional and specialized way, and can be used in the next courses.
The researcher has suggested that in addition to trying to improve the current evaluation forms, the instructors need to upgrade their knowledge on comprehensive and complete clinical evaluation. They also need to implementation of other appropriate and objective clinical evaluations and conduct formative and cumulative clinical assessments.[35] In the study by Roohi et al. (2017), 180 nurses and nursing students were subjected to clinical evaluation in four areas (evaluation content, role of the instructor in the evaluation, evaluation process, and evaluation outcome). The results showed that nursing interns and staff rated the instructor as good, the evaluation content and process as medium and the evaluation outcome as weak. In fact, the quality of the clinical evaluation of the nursing students shows the need for revision in current processes and tools, and that clinical evaluations and plans also need to be reconsidered by planners and instructors.[36] The results of the study by Riahi indicates the importance of the evaluation process among students of medical sciences.
Many universities use a mixture of evaluation methods such as portfolio, self-evaluation, peer assessment, and the station method for clinical education and unlike traditional evaluations at the end of a course, the students are evaluated during the course via mini-cx and OSCE. This way, the focus will shift from written learning to constant high-quality learning, which is way more attractive.[37]
The importance of evaluation and its influence on the students' learning and screening is known to everyone. Given the role of evaluation in modifying teaching models and evaluation of professors, the results of this study can upgrade the scientific and practical level of the students and thus, improve the national educational-medical system.
Innovation
This study for the first time in Iran, was conducted with the goal of designing and Implementation a comprehensive clinical competency with systematic research methods (Delphi technique and CIPP evaluation model) to Bachelar of scienses in operating roomtechnology students.
Limitations of this study
The design and implementation of clinical competency was done for Sabzevar University operating room technology students, and has not been evaluated in other Medical Sciences universities.
Suggestions
Use other evaluation models such as Tyler to determine how well students are achieving the desired clinical skills. On the other hand, it can be suggested to professors, trainers, and managers of the operating room group to use the above designed test to evaluate and measure students' clinical skills and replace traditional methods with new evaluation methods such as this model so that they can recognize the weakness of students and try to fix it.
| Conclusions | | |
This goal of this research was to design a comprehensive clinical competency test for the B.S. of Operating Room Technology using the Delphi technique and evaluating it by the CIPP model. Given the importance and sensitivity of clinical skills of B.S. students of Operating Room Technology and since training competent students will create professional and skilled technologists in the future, it is necessary ensure the acquisition of skills by proper instruments. The present study has designed a proper tool for comprehensive clinical competencies test and by implementing it, it proved that it is possible to employ this test to accurately and scientifically evaluate the clinical skills of Operating Room Technology students. The evaluation results also indicate that the students have gained the basic clinical skills to a medium extent and they are still far from the ideal level. This important and urgent need motives the faculty members to evaluate their students in a planned manner and based on a special model, so that they can find the strengths and weaknesses of their students and take the necessary measures to resolve the problem and if needed, change the teaching method for future students. We suggest that this designed comprehensive test and evaluation method be used in more extensive studies to measure the clinical skills of Operating Room Technology students, so that by fixing the defects, we can achieve a suitable and professional tool for evaluating clinical competencies of the Operating Room Technology discipline.
Acknowledgment
the authors sincerely than all the professors, medical staff, and the students who have participated and co-operated in this study.
Financial support and sponsorship
This paper is the result of the plan approved by the ethic committee of Sabzevar University of Medical Sciences with the ethic ID of IR.MEDSAB.REC.1399.106.
Conflicts of interest
There are no conflicts of interest.
| References | | |
| 1. | Pazargadi M, Ashktorab T, Khosravi S. Multi-rater evaluation in the clinical evaluation of nursing students: Instructorsexperiences and viewpoints. J Qual Res Health Sci 2012; 1:102-11.  |
| 2. | Mohammadi A, Khaghanizadeh M, Ebadi A, Amiri F, Raeisifar A. Log book; a method of evaluating education and feedback strategy in nursing. Educ Strategy Med Sci. 2010; 3 (1) :15-16 |
| 3. | Baig M, Ali SK, Ali S, Huda N. Evaluation of multiple choice and short essay question items in basic medical sciences. Pakistan journal of medical sciences. 2014; 30(1):3. |
| 4. | Ghorbanian N, Abdollahzadeh Mahlani F, Kazemi Haki B. Effective factors on clinical education quality anesthesiology and operating room student's view. Educ Strategy Med Sci 2014; 6:235-9. |
| 5. | Mazlom SR. Design of the structure of a comprehensive examination for bachelor of nursing: A delphi study. J Med Educ Dev 2018; 13:16-30. |
| 6. | Lotfi M, Zamanzadeh V, Sheikhalipour Z. Effect of peer clinical teaching method on the education of operating room students. J. Nurs. Educ. 2012;1 (1):78-83. |
| 7. | Mohamadpor Y, Pakpour V, Khalilzadeh H, Habibzadeh H, Jafarizadeh H, Ashraf Rezaee N, et al. Nursing students'perceptions of their educational environment: A comparison between expected and actual environment. J Urmia Nurs Midwifery Fac 2011; 9:102-11. |
| 8. | Aronson L, Niehaus B, Hill-Sakurai L, Lai C, O'Sullivan PS. A comparison of two methods of teaching reflective ability in year 3 medical students. Med Educ 2012;46:807-14. |
| 9. | Zardosht R, Karimi Moonaghi H, Etezad Razavi M, Ahmady S. Educational concern of surgical technology students in the operating room: A grounded theory study. J Edu Health Promot 2020;9:58. [Full text] |
| 10. | Moonaghi HK, Zardosht R, Razavi ME, Ahmady S. Perceived challenges by the Iranian Baccalaureate Surgical Technology students in their clinical education: A qualitative study. Biosci Biotech Res Comm. 2018; 10:542-0. |
| 11. | Farnia F, Abaszadeh A, Borhani F. Barriers to developing the nurse-patient relationship in operation room: A qualitative content analysis. J Qual Res Health Sci 2013;2:76-89. |
| 12. | Yazdimoghaddam H. Explaining the performance of operating room technicians: A qualitative study. J Mazandaran Univ Med Sci 2016;34-121:26. |
| 13. | Ministry of Health, Educational program of Bachelor of Science in Operating Room Technology. General profile, program, course syllabus and evaluation method. The Fifty two session of the Supreme Council of Medical Sciences Program. Islamic Republic of Iran Ministry of Health and Medical Education High Council of Medical Sciences Programming. Ministry of Heath of Iran, 2015. |
| 14. | Roghayeh Zardosht, Hossain Karimi Moonagh. Clinical Instructor Criteria in operating room: Qualitative content. J Sabzevar Univ Med Sci 2021;28:155-163. |
| 15. | Etherington N, Wu M, Cheng-Boivin O, Larrigan S, Boet S. Interprofessional communication in the operating room: a narrative review to advance research and practice. Canadian Journal of Anesthesia/Journal canadien d'anesthésie. 2019; 66(10):1251-60. |
| 16. | Paige JT, Garbee DD, Bonanno LS, Kerdolff KE. Qualitative Analysis of Effective Teamwork in the Operating Room (OR). Journal of Surgical Education. 2020; 5. |
| 17. | Taghipour A, karimi FZ, Mousavi Bazaz S M, Khosravi Anbaran Z, Abdolah M I. Factor Structure and Reliability of the Persian Version of the Jefferson Scale of Physician Lifelong Learning-Medical Students(JeffSPLL-MS). Iranian Journal of Medical Education. 2015; 14 (11) :988-997. |
| 18. | Pålsson Y, Mårtensson G, Swenne CL, Ädel E, Engström M. A peer learning intervention for nursing students in clinical practice education: A quasi-experimental study. Nurse education today. 2017;1;51:81-7. |
| 19. | Gobbo JL, Cardozo MV, de Lacerda LD, Karcher DE, Dias LG, de Nardi AB, Minto BW, Moraes PC. Evaluation of Operating Room Environment Contamination and Efficacy of 2% Chlorhexidine for Surgical Hand Scrubbing before and after Gowning and Gloving. Acta Scientiae Veterinariae. 2017; 45:1-0. |
| 20. | Roshanzadeh M, Toleyat M, Mohammadi S. Clinical evaluation tool for operating room students: Development and measurement of reliability and validity. Iranian Journal of Medical Education 2015: 15(15):98- 110. |
| 21. | Flostrand A, Pitt L, Bridson S. The Delphi technique in forecasting–A 42-year bibliographic analysis (1975–2017). Technological Forecasting and Social Change. 2020; 1; 150:119773. |
| 22. | Barzegari Esfeden Z, Dashtgard A, Ebadinejad Z. Evaluation of the realization of clinical nursing students' learning objectives using CIPP evaluation model. Iran J Nurs Res 2019;14:66-72. |
| 23. | O'Donoghue D, Davison G, Hanna LJ, McNaughten B, Stevenson M, Thompson A. Calibration of confidence and assessed clinical skills competence in undergraduate paediatric OSCE scenarios: a mixed methods study. BMC medical education. 2018; 18(1):1-8. |
| 24. | Majumder MA, Kumar A, Krishnamurthy K, Ojeh N, Adams OP, Sa B. An evaluative study of objective structured clinical examination (OSCE): students and examiners perspectives. Advances in medical education and practice. 2019;10:387. |
| 25. | Selim AA, Ramadan FH, El-Gueneidy MM, Gaafer MM. Using objective structured clinical examination (OSCE) in undergraduate psychiatric nursing education: Is it reliable and valid? Nurse Educ Today 2012; 32:283-8. |
| 26. | ArabKherad A, HajiAghajani S. Survey of Semnan University of Medical Sciences teachers' opinions about the effect of evaluation on teaching methods and their satisfaction with evaluation. Sci Inf Cent (Virtual) 2015;1:26-30. |
| 27. | Abbasi S, Einollahi N, Gharib M, Nabatchian F, Dashti N, Zarebavani M. Evaluation methods of theoretical and practical courses of paramedical faculty laboratory sciences undergraduate students at Tehran University of Medical Sciences in the Academic Year 2009-2010. Payavard Salamat. 2013;6 (5):342-53 |
| 28. | Hassanian ZM, Ahanchian MR, Karimi-Moonaghi H. Can knowledge management be implemented in the teaching of medical sciences? Acta Fac Med Naiss 2015;32:231-42. |
| 29. | Stollar F, Cerutti B, Aujesky S, Nendaz M, Galetto-Lacour A. Evaluation of a best practice approach to assess undergraduate clinical skills in Paediatrics. BMC medical education. 2020; 20(1):1-9. |
| 30. | Khan SA, Aaraj S, Talat S, Javed N. Students' perception and scores in Paediatrics end-of-clerkship and final professional Objective Structured Clinical Examination (OSCE): A comparative study. Pakistan Journal of Medical Sciences. 2021;2;37(2). |
| 31. | La Chimea T, Kanji Z, Schmitz S. Assessment of clinical competence in competency-based education. Canadian Journal of Dental Hygiene. 2020 Jun 1;54 (2) |
| 32. | Eldarir SA, Nagwa A, Hamid A. Objective structured clinical evaluation (OSCE) versus traditional clinical students achievement at maternity nursing: A comparative approach. IOSR J Dent Med Sci 2013;4:63-8. |
| 33. | Sadeghi T, Nejhad MS. Evaluation of nursing students to the OSCE: An approach to assess the merit students. JMED 2015;12:226-8. |
| 34. | Gandomkar R, Amini B. Application of various student assessment procedures by the department of Tehran University of Medical Sciences Iran. J Strides Dev Med Educ 2015;12:209-18. |
| 35. | Rafiee G, Moattari M, Nikbakht AN, Kojuri J, Mousavinasab M. Problems and challenges of nursing students' clinical evaluation: A qualitative study. Iran J Nurs Midwifery Res 2014;19:41-9. |
| 36. | Roohi ZR, Salehi S. Quality of clinical evaluation from viewpoint of nurse interns and nursing unit clerks; nursing students of the school of nursing and midwifery. Asian J Pharm Res Health Care 2017; 9:17-21. |
| 37. | Ferris H, O'Flynn D. Assessment in medical education; what are we trying to achieve? Int J High Educ 2015; 4:139-44. |
[Figure 1]
|
Search Pubmed for