P3.Q338: Development and Validation of a Multimedia Hypoplastic Left Heart Syndrome Assessment Tool for Pediatric Cardiology Fellows

Background: Pediatric cardiology fellows must have knowledge of cardiovascular structure to understand important physiologic and clinical concepts, especially those related to surgical palliation of single ventricle heart disease. However, validated, high quality means of evaluating trainee knowledge of these concepts are limited. Here we describe a multi-institutional collaboration to develop and validate a multimedia assessment tool to assess pediatric cardiology fellows’ knowledge of anatomy, physiology, and clinical concepts related to surgical palliation of hypoplastic left heart syndrome (HLHS).

Methods: Six pediatric cardiologists and fellows from four institutions collaborated to develop a 33-question assessment tool. Targeted educational domains were identified (Table). All questions were multiple choice with many including echocardiographic or angiographic videos related to HLHS and each stage of palliation (Figure). We identified four content experts from three institutions comprised of mid-career or senior pediatric cardiologists who were not involved in creation of the assessment tool. Each content expert completed the assessment and rated the relevance of each item to our educational domains using a 4-point Likert scale (1=not relevant, 2=somewhat relevant, 3=quite relevant, 4=highly relevant) and was given the option to provide qualitative feedback. We evaluated content validity for each question by calculating an item-level content validity index (I-CVI), with a score of 0.78 indicating acceptable content validity. An overall scale-level content validity index (S-CVI) was calculated, with an S-CVI of 0.90 indicating excellent overall content validity.

Results: The mean expert score on the assessment tool was 97.0% (range 90.9% - 100.0%). Four out of 33 questions elicited incorrect answers, and no questions yielded 50% incorrect answers. The I-CVIs for 30 out of 33 questions were 0.78. The S-CVI for the assessment tool was 0.98. Qualitative feedback was overall positive with a few comments recommending improvement in angiographic and echocardiographic images. Questions with I-CVIs < 0.78 were removed, yielding a 30-question assessment tool.

Conclusion: We report development of a validated 30-question assessment tool which can be used to evaluate pediatric cardiology fellow knowledge of important concepts related to surgical palliation of HLHS. Ongoing efforts include use of this tool to evaluate trainee performance on a novel virtual reality curriculum developed at our centers. Future efforts will focus on expansion and improvement of questions and development of additional validated tools to evaluate trainee understanding of important concepts in congenital heart disease.