Program Evaluation & Outcomes Assessment

Center faculty continued work with JMC fac­ulty who use the University Clinical Simulation Center to analyze students' clinical assessments in preclinical courses, clinical clerkships and the comprehensive third-year OSCE.  These included research related to methods of measuring clinical performance, the relationship between simulated patients’ ratings of students’ empathy and the students’ self-reported empathy, and self-assessment in surgery.  The following studies were com­pleted:

Clinical Skills Assessment in the Surgery Clerkship – The model of skills assessment in Jefferson's surgery clerkship is noteworthy because students must demonstrate their proficiency in performing simple procedures such as in­serting intravenous tubes or suturing on task training simulators, while interact­ing with live simulated patients. The findings from this three-year study of the va­lidity of the formal assessment of students' clinical skills at the end of the surgery clerkship was published in The American Journal of Surgery.

USMLE Step 2 Clinical Skills – A study to be published in Simulation in Healthcare in collabora­tion with researchers at Washington Uni­versity, Michigan State Univer­sity, and University of Kansas between 2008 and 2010 investigated the association between students’ perform­ance on local clinical skills assessments and the USMLE Step 2 Clinical Skills test.

Clinical Skills Assessments during Medi­cal School and Performance in Residency – The collaborative study with Washington University, Michigan State and Kansas also involved research on the rela­tionship between students’ performance on local clinical skills assessments and pro­gram directors' ratings of their performance in residency.  The findings were pre­sented at the Conference on Research in Medical Education at the Annual Meeting of the AAMC, and will be published in Simulation in Healthcare.

Residents' Proficiency in End-of Life Care – Center faculty collaborated with faculty in the Department of Medicine who were developing a program to foster first-year residents' skills in leading end-of-life discussions with dying patients and their families.  Residents' performance was measured before and after completion of the program using simulated clinical en­counters with actors posing as the family members of dying patients and written clinical vignettes describing comparable scenarios. The findings from this study were published in the American Journal of Hospice and Palliative Medicine.

Medical Students' Response to Challenging Counseling Tasks – Center faculty collaborated with faculty in the Department of Pediatrics to analyze data obtained during a clinical simulation de­signed to test medical students' proficiency in teaching and counseling patients about shaken baby syndrome. The findings from this study are helping faculty to identify students who could benefit from interventions to increase their sensitivity toward patient needs.

Center faculty worked with the Dean and the Alumni Office to develop a social network analysis of the senior class in order to provide an empirical base for the selection of class agents at graduation. A unique survey was developed to model students connections with their classmates.

The change from numeric grades to the honors/pass/fail grading system over a decade ago called for new approaches to the calculation of class ranks. Center faculty have provided psychometric support to the Associate Dean for Student Affairs for the calculation of class ranks for the selection of students for the AOA honor society. Analogous support has been provided to the Department of Surgery for selection of students for their honor society.

Center faculty provided special reports for the University Office of Financial Aid related to the career outcomes and geographic location of medical students. Data were also abstracted from the Longitudinal Study for the Department of Community and Family Medicine to satisfy reporting requirements related to federal grants and contracts.

Center faculty and staff collaborated with the Dean’s Office, Jeff-IT, and the clinical clerkship directors in the design, operation, and continuing improvement of Jefferson’s Patient Encounter Log System (PELS). JMC students use PELS to record information about their patient encounters during clinical clerkships. PELS enables clinical clerkship directors and the Dean’s Office to track student experiences compared to clerkship goals. PELS is a component of JMC compliance with the LCME requirement for setting of clerkship goals and evaluation of student achievement towards these goals. With a transition to web-based data entry, students, faculty, and administration can review progress during a clerkship to help students achieve appropriate clinical experiences. Under an agreement signed with TJU, the Center is assisting the Emilia-Romagna Region of Italy in adapting the PELS system for use by family medicine trainees to track their clinical experiences with community-based preceptors during their post-graduate training.

Center staff provided periodic reports to the Curriculum Committee, Dean’s Office, depart­ments and affiliated hospi­tals to assess the qual­ity of the educational pro­gram at TJU and affili­ated hospitals. Established in 1982, the clerk­ship database in­cludes students’ reports of their experiences at each site, satisfaction with their education, scores on written examinations, fac­ulty ratings of the students’ clinical perform­ance, and stu­dent reports of the clerkships’ im­pact on their career plans.