CDI 2024
Collection

Measuring the Impact of UVA's CDI

This is a collection of UVA Center for Teaching Excellence's peer-reviewed research demonstrating the efficacy and impact of its multi-day, intensive course design institute.

Updated January 2025
Michael Palmer headshot
Barbara Fried Director & Professor
Office of the Executive Vice President and Provost
View Bio
Lindsay Wheeler headshot
Senior Associate Director & Associate Professor
Office of the Executive Vice President and Provost
View Bio

Study #1: Measuring Participant Learning

To Improve the Academy

For the first six or seven years of UVA's CDI, pre-/post-surveys probed changes in Participant Learning about teaching, particularly their attitudes. Several hundred syllabi, including both pre- and post-CDI syllabi, were also analyzed using a validated rubric to probe changes in participants’ beliefs and teaching practices.

Headshot of Michael PalmerHeadshot of Lindsay Wheeler
Michael Palmer, Lindsay Wheeler

This initial CDI study, based on six years of participant data, anchored our research for the next eight years. The qualitative and quantitative studies into participants’ beliefs and teaching practices following participation in our CDI indicated that instructors believe they learn basic principles of learning-focused course design, report they are more confident enacting learning-focused concepts in the classroom, and believe they are better able to design learning-focused syllabi. This was also the first study to use our award-winning syllabus rubric.

View excerpt

The University of Virginia’s week long CDI is grounded in the literature and supported by the theoretical frameworks of backward integrated course design and Fink’s taxonomy of significant learning (Fink, 2013). The overarching goal is for participants to design or redesign a course—and future courses—grounded in learning focused, evidenced based principles...This focus and intentionality makes CDIs like ours potentially powerful instructional development experiences capable of shifting teacher beliefs, levels of confidence, and practices toward a learning focus. However, to what extent, exactly, do these types of intensive course design interventions impact teaching beliefs and practices and, ultimately, student learning? In other words, can CDIs be viewed as a high impact educational development practice?

Was this resource helpful?

Study #2: Characterizing STEM Teaching Practices

Science

This research was part of a multi-institutional study involving the observations of over 2,000 courses and 500 unique STEM instructors, including some who had participated in our CDI. We identified seven unique teaching profiles, ranging from “didactic” to “student-centered.”

Headshot of Michael PalmerHeadshot of Lindsay Wheeler
Michael Palmer, Lindsay Wheeler

We contributed observational data for over 500 courses from UVA instructors who had and had not participated in our CDI. The overall results were not promising, but a large percentage of UVA instructors who had participated in our CDI were among the “eighteen percent of observations [in the] ‘Student-Centered’ style” instructional profile.

View excerpt

These findings challenge institutions and STEM disciplines to reflect on practices and policies that sustain the status quo. Specifically, institutions should revise their tenure, promotion, and merit-recognition policies to incentivize and reward implementation of evidence-based instructional practices for all academic ranks…These policy changes would require institutions and STEM professional organizations to provide effective pedagogical training for the current and future professoriate, similar to the level provided for research. Further, these policy changes cannot be meaningfully implemented without research-based guidelines for measuring effective teaching practices. Funding agencies should prioritize the development of such guidelines.

Was this resource helpful?

Study #3: Measuring Participant Learning & Institutional Culture

Journal of Research in Innovative Teaching and Learning

We conducted 100+ classroom observations using a validated observation protocol, along with surveys and syllabi analysis, to better understand changes in Participant Practices following instructors’ participation in UVA’s CDI.

Headshot of Michael PalmerHeadshot of Lindsay Wheeler
Michael Palmer, Lindsay Wheeler

We were particularly curious about whether participants who developed more learning-focused courses, determined through syllabi analysis, adopted more learning-focused pedagogical practices in the classroom. The results here were mixed. While many participants did adopt more learning-focused pedagogical practices, the degree of change did not match the change observed in their syllabi. This work also helped the CTE team better understand the prevailing Institutional Culture at the institution and ultimately guided our development of longer-term learning communities that synergistically deepen the impact of our CDI.

View excerpt

Participants experienced significant increases to both dimensions of their sense of belonging to the institution with large practical effects. Participants increased their sense of connectedness after participating in CDI and maintained these levels throughout their FLC experience. For most participants, CDI marked the beginning of their cohort experience and with it their sense of being connected with members of the institution outside of their departments.

Was this resource helpful?

Study #4: Measuring Students’ Perceptions

International Journal for the Scholarship of Teaching and Learning

A syllabi perception study helped us probe Student Perceptions of non-CDI-designed and CDI-designed syllabi, considering how a syllabus might shape students' views of a course and instructor.

Headshot of Michael PalmerHeadshot of Lindsay Wheeler
Michael Palmer, Lindsay Wheeler

In this study, we were curious to learn whether students would perceive differences between syllabi designed in our CDI and those that were not. What we discovered was that it's not enough for an institution to require syllabi. Institutions need to require the right type of syllabi. Learning-focused onesthe type we encourage instructors to design in our CDIcan positively affect student motivation before they even enter the classroom, making meaningful engagement in the course much more likely.

View excerpt

Students’ Perceptions of Course Syllabi: The Role of Syllabi in Motivating Students

International Journal for the Scholarship of Teaching and Learning
Open resource

The present study examined students’ perceptions of a learning-focused and content-focused syllabus to better understand how the syllabus influences perceptions of the document, course, and instructor. Both quantitative and qualitative data collected support the hypothesis that the syllabus does matter: for the most part, students who read a learning-focused syllabus have more positive perceptions of the document, instructor, and course than students who read a content-focused syllabus.

Was this resource helpful?

Study #5: Measuring Student Outcomes

International Journal for Academic Development

In this ambitious study, we were able to connect CDI experiences to Student Outcomes in the form of DFW (students who earn a “D,” “F,” or withdraw from a course) rates.

Headshot of Michael PalmerHeadshot of Lindsay Wheeler
Michael Palmer, Lindsay Wheeler

This work won the IJAD 2021 Article of the Year award, and the commendation sums up its contribution to the literature nicely: "The paper has a great deal to offer academic development practice and offers a way forward for practitioners to develop rigorous evaluations of the initiatives we champion, which can be used to justify those initiatives and garner institutional support—especially important given the performance regimes that are increasingly becoming prevalent across the world. Using a range of statistical measures, the authors provide insights that will prove helpful to the academic development community, both in terms of findings and methods, and present a robust, comprehensive, and convincing study of the effects of academic development interventions on instructor practices and student learning."

View excerpt

Educational developers are increasingly asked to demonstrate the efficacy of their work and to provide evidence that the investment in educational development (ED) is worthwhile. This study contributes to research focused on establishing the link between participation in ED programs, changes in teaching practices, and improved student learning. Specifically, drawing on research on intensive course design workshops and faculty learning communities, it examines the impact of different ED intervention types and durations. Unique in its design, this study uses a multi‐indicator strategy and direct measures such as syllabi and classroom observations to assess changes in teaching as well as institutional grade data to approximate student success.

Was this resource helpful?

Want to recommend a resource to add to this collection? Send us an email.