AI Community Discussion Session Report: Case Studies to Inform Curriculum
In March, The Carpentries hosted a pair of AI and The Carpentries sessions, the last we currently have planned in this series. These sessions, titled Case Studies to Inform Carpentries Curriculum, were an opportunity to hear directly from community leaders and learn from their teaching experience with and about large language models (LLMs). After several open discussion sessions facilitated by members of the Core Team, we took a deeper dive into the perspectives and expertise of particular community members.
In the first of the two calls, we heard from Lex Nederbragt about his experience teaching Python for biological modelling to students at the University of Oslo. He described how students can use a GDPR-compliant instance of ChatGPT provided by the University. He summarised some challenges and opportunities he sees with novices using this LLM while learning to program. Lex’s overview was followed by an engaging group discussion that touched on points about student motivation, helping them judge when to use the LLM as a learning assistant and a code generator, and what the future of programming education might be.
The second call featured talks from Brian Ballsun-Stanton and Yanina Bellini Saibene. Brian summarised some of the work he and colleagues have been doing around methods for enhancing learning with LLM assistance and other uses he has found for LLMs in his work. He observed that LLMs and an introduction to “vibe coding” should be considered a topic at least as complex as the extant Python and R lessons. And clumsily inserting in LLM use or debugging into existing lessons is unwise. He also noted that the quality of results obtained varies considerably depending on which model is used, with those available via a paid subscription performing better than those accessible without cost, which has implications for equity and any policy on LLM usage in Carpentries workshops.
Yanina discussed her experience with novice learners trying to use LLMs like this anyway. She shared examples of exercises and activities she sets for students and the guidance she gives them around LLM usage. Yani’s experience – one probably shared by every Carpentries Instructor! – is that learners do not always follow instructions, for example, by trying to use LLMs to generate solution code even when asked not to do so.
We are so grateful to the three presenters for their contributions: thank you, Brian, Lex, and Yanina! These presentations and the following discussions helped participants better understand educators’ different approaches to address an increasingly inescapable technology.
What’s Next?
There are no more AI and The Carpentries sessions planned. If you would like to propose a topic or present something to the community, please contact us.
Since a pull request adding guidance on generative AI was merged into the Software Carpentry Plotting and Programming with Python lesson, community members have proposed similar changes to the other Data Carpentry, Library Carpentry, and Software Carpentry lessons. There is still time to get involved, and we would love your help! Join the #genai-lesson-updates channel on Slack (join the Slack workspace) or contact Toby Hodges to learn more about the process.
The Curriculum Team will soon invite developers of ML/AI-related lessons in The Carpentries Incubator to join calls and discuss the possibility of coordinating their efforts to form curricula for new workshops.