Tutor Bot. When I talk to STEM faculty, this is by far the most common use case of custom AI chatbots I hear about. The idea is that if we can “train” a custom chatbot on learning materials relevant to our course, we can create a helpful tutor that’s available 24-7 to our students. In episode 34 of my podcast, Sravanti Kantheti talked about the use of a tutor bot in her anatomy and physiology course. She pointed out that you can teach A&P at a 3rd grade level, a med school level, and anywhere in between. She appreciated that her tutor bot, which was trained on her course materials, would answer questions at a level that worked for her community college students. And in that podcast conversation with University of Sydney faculty, biochemistry instructor Matthew Clemson shared about his Cogniti-powered tutor bot which he dubbed “Dr MattTabolism.” He wanted students in his large classes to have a place to go to ask content questions that was available when he wasn’t (e.g. 2am) and would provide better answers than a Google search.

Danny Liu: 5:30

Yeah. We see Cogniti as a way for educators to take control of AI. It's a place where they can go and, in a very easy, people-friendly way, be able to build what we call AI agents, which are basically just AIs which can do something on your behalf. And you'll hear from Matthew and Isabelle today the different kinds of agents that they've worked with and designed themselves just using plain language. And for us, it's about giving educators that kind of control and visibility over what the AI is doing so that they can trust it and also so that students can trust it as well.

Over the course of the following spring semester, we developed 10 chatbots with different purposes. The results varied widely, but we learned as much from the failures as from the successes, and this hands-on experimentation allowed us to begin identifying some patterns that could inform and inspire future experiments.

These four chatbot examples represent a first attempt at capturing what we’ve learned so far. We think these delineations make it easier to see how chatbots can align with specific aspects of the learning process, even if they might have more than one role at a time (such as with the Stuckbot, acting as both an assignment tutor and reflective guide, for example).

We hope they will provide you with a starting point as you imagine roles for AI that reflect your particular contexts and purposes, and we look forward to learning from what you discover in the process.

Custom chatbots created with ChatGPT are called Custom GPTs or simply GPTs. Custom GPTs are easy to build because no coding is required. This functionality of building Custom GPTs is reserved for paid users. However, users without a paid subscription can now use Custom GPTs that others have created. This means that if you have the paid ChatGPT version, you can create Custom GPTs for your students to use even if they don’t have a paid account. Step-by-step instructions for building Custom GPTs can be found in Section 2 below, while examples are provided in Sections 3 and 4.

ChatGPT agrees with you 58% of the time. Claude hits 60%. Gemini tops out at 62%. These numbers come from recent academic research [1]. The AI systems millions of people use daily have learned to tell us what we want to hear.

Ask any of these systems whether your controversial opinion has merit.

Watch how they find ways to validate your thinking rather than challenge it. Ask for feedback on your work presentation. Notice how they emphasise strengths while glossing over obvious problems.

You're not imagining this pattern. The technical term is "sycophancy"—the tendency to flatter rather than inform. Your AI assistant has become a digital yes-person.

We found that human raters, at least the well-trained, paid, and relatively time-rich evaluators in this sample, provided higher quality feedback in four of five critical areas: clarity of directions for improvement, accuracy, prioritization of essential features, and use of a supportive tone. The impressive skills of experienced and resourced human educators to provide quality formative feedback were notable. However, the most important takeaway of the study is not that expert humans performed better than ChatGPT—hardly a surprising finding—but rather that ChatGPT's feedback was relatively close to that of humans in quality without requiring any training. To our knowledge, no previous study has compared automated and human feedback on writing because the quality of automated feedback has been so poor that such a study would be futile. Presently, the small differences between the two modes of feedback suggest that feedback generated by ChatGPT can likely serve valuable instructional purposes, particularly in the early stages of writing to motivate revision work by students in a timely fashion.

We have found that when students interact with our AI tutor, at home, on their own, they learn significantly more than when they engage with the same content during an in-class active learning lesson, while spending less time on task. This finding underscores the transformative potential of AI tutors in authentic educational settings. In order to realize this potential for improving STEM outcomes, student-AI interactions must be carefully designed to follow research-based best practices.