AI literacy is a hard thing to teach students about for a variety of reasons. First, AI literacy is hard to teach because, frankly, we are not experts at it yet; we are lucky if we are just barely running ahead of the students, but in most cases they are running a bit ahead of us. Beyond that, it’s all so new. It’s not totally clear what skills we are even supposed to teach students, what will be valuable, and so on. That being said, over the past few months the literature has increasingly pointed toward teaching practical skills (such as prompt engineering, the differences between tools, etc.), critical inquiry skills (how to fact-check GenAI output, how to integrate it into other work, etc.), and relevant context of these tools (climate impact, how these tools work, data protection, etc.). Finally, AI literacy is hard to teach to nonsubject-matter experts.

Generative AI is poised to revolutionize how humans work, and has already demonstrated promise in significantly improving human productivity. A key question is how generative AI affects learning—namely, how humans acquire new skills as they perform tasks. Learning is critical to long-term productivity, especially since generative AI is fallible and users must check its outputs. We study this question via a field experiment where we provide nearly a thousand high school math students with access to generative AI tutors. To understand the differential impact of tool design on learning, we deploy two generative AI tutors: one that mimics a standard ChatGPT interface (“GPT Base”) and one with prompts designed to safeguard learning (“GPT Tutor”). Consistent with prior work, our results show that having GPT-4 access while solving problems significantly improves performance (48% improvement in grades for GPT Base and 127% for GPT Tutor). However, we additionally find that when access is subsequently taken away, students actually perform worse than those who never had access (17% reduction in grades for GPT Base)—i.e., unfettered access to GPT-4 can harm educational outcomes. These negative learning effects are largely mitigated by the safeguards in GPT Tutor. Without guardrails, students attempt to use GPT-4 as a “crutch” during practice problem sessions, and subsequently perform worse on their own. Thus, decision-makers must be cautious about design choices underlying generative AI deployments to preserve skill learning and long-term productivity.

The rapid popularization of generative AI has fostered much unease in higher ed. On the CU Boulder campus, educators have implemented various generative AI policies, from banning this technology to integrating its use into activities and assessments. While many initiatives exist on campus to engage educators in conversation around these varied approaches, the Center for Teaching and Learning saw a need to understand the CU Boulder undergraduate student perspective on this emerging technology. The Educational Technology Research Assistant (ETRA) program was created to fill this gap. Specifically, our team of ETRAs conducted a mixed-methods research project to answer the following questions:

• What are CU Boulder students’ attitudes toward and knowledge of generative AI?
• How and why do CU Boulder students use generative AI in an educational context?
• What generative AI training and policies do CU Boulder students want to see going forward?
• How do CU Boulder students’ usage, knowledge, and attitudes toward generative AI vary by college, year in school, age, modalities of classes taken, international status, gender, race, and degree of comfort using English to communicate?