Measuring the Impact of Our Workshops and Trainings
We conduct pre and post workshop surveys for each Carpentries Workshop We Run
Find Copies of our Current Surveys
The Carpentries Pre and Post-Workshop Surveys
Instructor Training Pre and Post Surveys
Long-Term Impact Survey
Additionally, we began collecting data on the long-term impact our workshops are having on Carpentries learners. The Carpentries Long-Term Impact Survey was launched in March 2017, and data is collected every six months.
We release regular reports on the work and impact of The Carpentries. These reports are published through Zenodo and can be accessed through the links below.
Instructor Training Report
Analysis of Software and Data Carpentry’s Pre- and Post-Workshop Surveys
Analysis of Software Carpentry’s Post-Workshop Surveys (2017) . The data used in this report is available in this repository.
Long-Term Survey Reports
Analysis of The Carpentries Long-Term Surveys (2020) These results are also available on GitHub.
Measuring The Carpentries Reach
The following reports use data from our AMY database to describe the reach of The Carpentries. This includes number of Instructors certified, number of workshops held, number of learners taught, and number of countries that held Carpentries workshops.
Other Assessment-Related Publications
When Do Workshops Work? A Response to the ‘Null Effects’ paper from Feldon et al.(2017) Author: Karen R. Word. Contributors: Kari Jordan, Erin Becker, Jason Williams, Pamela Reynolds, Amy Hodge, Maxim Belkin, Ben Marwick, and Tracy Teal.
This was a collaborative response to the paper: Feldon, David F. et al. Null effects of boot camps and short-format training for PhD students in life sciences Proc Natl Acad Sci U S A. 2017 Sep 12; 114(37): 9854–9858. doi: 10.1073/pnas.1705783114. Our data suggest that we are having a positive impact, and we expect that other short-format programs can be similarly effective. Read our full response.
The Assessment Network held a webinar with Rochelle Tractenberg. Dr. Tractenberg directs the Collaborative for Research on Outcomes and Metrics at Georgetown University, where she is a tenured professor in the Department of Neurology. Our starting point was the controversy about short-format training which arose last year, following the publication of a 2017 PNAS paper titled “Null effects of boot camps and short-format training for PhD students in life sciences”. The Carpentries design and deliver short-format training for working with software and data; trainees are researchers from various fields. The Carpentries’ initial response to the paper discussed many ways in which we have been successful with respect to our goals for Software Carpentry and Data Carpentry workshops. However, given that short-format training is a known challenge for generating sustainable content learning, we hoped that Dr. Tractenberg’s expertise might shed some light on areas with room for improvement. See the full webinar.