Developing an Instrument to Assess Teaching Skills of Lab Instructors in Medical Education: Initial Validation

Sadia Ahsin; Muhammad  Iqbal; Hira Ashraf

doi:10.51253/pafmj.v75i3.10619

Authors

Sadia Ahsin Department of Physiology, Foundation University Islamabad Pakistan
Muhammad Iqbal Department of Surgery, Shifa Tameer-e-Millat University Islamabad Pakistan
Hira Ashraf Department of Physiology, Foundation University Islamabad Pakistan

DOI:

https://doi.org/10.51253/pafmj.v75i3.10619

Keywords:

Delphi’s method, factor analysis, Laboratory teaching evaluation, validity

Abstract

Objective: To devise and validate a direct observation tool for assessing lab teaching skills of laboratory instructors namely ‘Direct Observation of Teaching Skills’ (DOTS)

Study Design: Mixed method study

Place and Duration of Study: Foundation University Islamabad and Shifa Tameer-e-Milat University Islamabad, Pakistan from Apr 2021 to Apr 2022.

Methodology: More than three hundred and fifty qualified medical educationists who were registered with Pakistan Medical and Dental Council and having five years and above teaching experience participated through purposive sampling. Tool development was done in systematic seven recommended steps of AMEE Guide 87 including 1) Literature Search, 2) Focus group discussion, n= 7, 3) Synthesis of literature review and focus group discussions, 4) Item development, 5) Expert Validation, 6) Cognitive pretesting n= 5 and 7) Pilot testing n= 310. SPSS and AMOS were used for data analysis.

Results: A 29-item tool evaluating laboratory teacher performance including laboratory instructions, activities and management was validated for content, response process and construct with Scale Content Validity Index/ Average (S-CVI) of 0.867 and S-CVI Universal agreement of 0.989. Confirmatory analysis showed factor loading of > 0.50 for all items. The absolute fit value of Observed normed χ2 was 2.99, goodness of fit index was 0.79 and root mean square error of approximation was 0.08. Internal consistency for lab instruction, lab activities and lab management showed Cronbach’s α values of 0.909, 0.899, 0.912 respectively.

Conclusion: The devised tool, DOTS appears reliable and valid in content, response process and construct.

Downloads

Download data is not yet available.

References

Front Educ 2023; 8: 1217167.

https://doi.org/10.3389/feduc.2023.1217167

Wiggs NB, Reddy LA, Kettler R, Hua A, Dudek C, Lekwa A, et al. Convergence between teacher self-report and school administrator observation ratings using the Classroom Strategies Assessment System. J Psychoeduc Assess 2023; 41(1): 162–177. http://doi.org/10.1177/15345084221112858

Unissa R, Alhasan NA. Investigating educators’ perspectives on classroom observation and the impact of feedback on professional development. Innovare J Educ 2024; 12(6): 1–6. Available from:

https://journals.innovareacademics.in/index.php/ijoe/article/view/52896

Artino Jr AR, La Rochelle JS, Dezee KJ, Gehlbach H. Developing questionnaires for educational research: AMEE Guide No. 87. Med Teach 2014; 36(6): 463-474.

https://doi.org/10.3109/0142159x.2014.889814

Gibbs G. The nature of qualitative analysis. Analyzing qualitative data. 2007: 1-9.

Morgado FF, Meireles JF, Neves CM, Amaral A, Ferreira ME. Scale development: ten main limitations and recommendations to improve future research practices. Psicologia: Reflexão e Crítica 2017; 30.

https://doi.org/10.1186/s41155-016-0057-1

Hsu, C. and Sandford, B. The Delphi Technique: Making Sense of Consensus. Prac Asses Res Eval 2007; 12: 1-8.

Suryadi T, Alfiya F, Yusuf M, Indah R, Hidayat T, Kulsum K. Content validity for the research instrument regarding teaching methods of the basic principles of bioethics. J Pendidik Kedokter Indones 2023; 12(2): 186–202.

http://doi.org/10.22146/jpki.77062

Linneberg MS, Korsgaard S. Coding qualitative data: A synthesis guiding the novice. Qualitative Res J 2019.

https://doi.org/10.1108/QRJ-12-2018-0012

Moreno-Murcia J, Torregrosa YS, Pedreo NB. Questionnaire evaluating teaching competencies in the university environment. Evaluation of teaching competencies in the university. J New Approaches Edu Res 2015; 4(1): 54-61.

https://doi.org/10.7821/naer.2015.1.106

Felland N. Charlotte Danielson's enhancing professional practice: A framework for teaching. Perspect Learn 2001; 2(1): 16.

Couchenour D, Chrisman JK. The SAGE encyclopedia of contemporary early childhood education: SAGE Publications; 2016.

http://doi.org/10.4135/9781483340333.n138

Yusoff MSB. ABC of content validation and content validity index calculation. Edu Med J 2019; 11(2): 49-54.

https://doi.org/10.21315/eimj2019.11.2.6

Deniz MS, Alsaffar AA. Assessing the validity and reliability of a questionnaire on dietary fibre-related knowledge in a Turkish student population. J Health Popul Nutr 2013; 31(4): 497.

https://doi.org/10.3329/jhpn.v31i4.20048

Schermelleh-Engel K, Moosbrugger H, Müller H. Evaluating the fit of structural equation models: Tests of significance and descriptive goodness-of-fit measures. Methods Psych Res Online 2003; 8(2): 23-74.

https://doi.org/10.23668/psycharchives.12784

Smith J. An objective structured teaching exercise (OSTE) for physicians employing multi-source feedback. MedEd PORTAL 2015; 11. https://doi.org/10.15766/mep_2374-8265.10157

Cladera M. An application of importance-performance analysis to students’ evaluation of teaching. Educ Asses Eval Acc 2021; 33(4): 701-715.

https://link.springer.com/article/10.1007/s11092-020-09338-4

Al Ansari A, Arekat MR, Salem AH. Validating the modified System for Evaluation of Teaching Qualities: a teaching quality assessment instrument. Adv Med Educ Pract 2018; 9: 881–886. https://doi:10.2147/AMEP.S181094