Reflections on Assessment

This post originally appeared on the Data Carpentry website As I approach my one-year anniversary on staff with Data Carpentry, I’m reflecting on what we’ve accomplished this year. I came into this role with no data science experience whatsoever: self-efficacy had been the focus of my doctoral dissertation and post-doctoral work. That background knowledge made for an exciting time exploring our workshop survey data and letting the community know what our learners think about our workshops.

I’ve come a long way this year, partly because of my ability to transfer learning, but mainly because of our community. Looking back at the first report I wrote about Data Carpentry’s workshop impact, compared to my recent report about the impact of Software Carpentry’s workshops, you’ll see that I’ve learned a lot about using R. For that, I owe thanks to all the community members who responded to my tweets for help, hopped on Zoom calls with me, and contributed to my GitHub repositories. I truly feel that I belong to this community, and am living proof that having a growth mindset can lead to success in whatever field you’re passionate about.

Thanks to that help from this great community, I was able to develop a long-term assessment strategy that includes the launch of our new skills-based pre- and post-workshop surveys, and our joint Data and Software Carpentry long-term survey.

Let me stress the importance of our pre- and post-workshop surveys. We know anecdotally that our workshops help learners improve their skills and confidence. We see it when the faces of learners light up during workshops. We hear it during instructor discussion sessions when participants can’t hide their excitement about what they’ve achieved and how much they look forward to teaching. We know that new instructors are fielding requests to run workshops. However, without hard evidence that proves we do indeed enable learners to quickly apply new skills to their daily research practice, we run the risk of missing out on opportunities to effect change in the data science space.

Survey links have been pre-loaded into Data Carpentry’s workshop template, but we need you, our instructors, to encourage learners to complete those surveys. Even the best-designed survey is useless without responses. So, how can you help?

• Include the survey links in both your introductory and post-workshop follow-up e-mails.
• Ask learners to arrive 15 minutes early to complete the pre-survey, and stay a few minutes after to complete the post-survey (This is not ideal, as it doesn’t allow instructors time to calibrate the workshop to attendees’ skill levels, but it is better than nothing).
• Add the survey links to your workshop Etherpad or Google Doc.
• Explain how these surveys not only push learners to take responsibility for their own learning, but also help them track their progress towards their learning goals.
• Tweet the survey links before and after your workshop.

These are just a few ways you can support our assessment efforts. We also invite you to join the assessment network. We created this network to provide a space for those interested in assessment in data science to collaborate and share resources. We meet quarterly - our next meeting is in November. Check out what we’ve already achieved, and feel free to join us!

I am very proud of the work we’ve done, and excited about what’s to come. I’d love to hear from you. Tweet your thoughts to @datacarpentry and @drkariljordan.