Training Lessons

This post originally appeared on the Software Carpentry website.

I wrote about our experiments with the format of instructor training back in May. At that time, we had run the class as:

1. a multi-week online class,
2. an in-person two- or three-day class, and
3. a mixed mode with the trainees physically together for two days with the trainer coming in via teleconference.

We have since tried the mixed mode twice with the trainees at three different sites (three universities in Arizona for one run, and universities in Cape Town, Sheffield, and Ann Arbor for the other). We've also gathered a lot of feedback on what people want from instructor training and what its prerequisites should be. Here's what we've learned.

The two-day format seems to work as well as the online multi-week format, but in a different way.

The multi-week format gives people more time to read and reflect, but squeezing everything into two days is more engaging. I also think that people enjoy recording themselves and watching those videos more (or dislike it less) when they're together, though that may actually reflect the fact that they're doing training with people they know when they're co-located.

The online two-day format seems to work as well as the in-person two-day format.

Again, I'm judging by end-of-course feedback—it hasn't been long enough for us to measure follow-through. I think that having people co-located is the key: we haven't tried it, but I'm certain that having everyone teleconference for two days will be a lot less enjoyable. (It will probably also suffer from a lot more technical issues.) It also makes a big difference if at least one person at each site has been through a workshop as a learner, helper, or instructor—so much so that we may make this a requirement in future.

One other observation about the online two-day format is that it's important to have all the learners in the same time zone (or nearly). South Africa is six hours ahead of Michigan; a four o'clock finish for Ann Arbor was a sleepy ten o'clock wrap in Cape Town.

Our syllabus for the two-day format is working well.

Last week's two-day class for trainers from Harvard and the ACI-REF consortium was one of the best yet. I still hope to turn the training into a proper Software Carpentry-style lesson by the end of August, but in point form, the curriculum these days is:

• Day 1:
  • Introductions: explain why they're important and get everyone to do their first video recording and review.
  • The differences between novices, competent practitioners, and experts
  • Reverse instructional design, leading into design of a good multiple-choice question and another video recording exercise
  • Concept mapping, leading into the third video recording exercise of the day
  • Motivation and demotivation, leading into an overnight homework exercise
• Day 2:
  • Review
  • An overview of Software Carpentry and Data Carpentry's content
  • A demo of live coding, followed by a video recording exericse in which trainees teach the same lesson to each other
  • An overview of Software Carpentry and Data Carpentry content
  • Creating a website for a workshop
  • The problem of assessment, followed by small-group critique of our existing pre- and post-workshop questionnaires
  • Small-group discussion of what should go into a new lesson (e.g., a half-day introduction to HPC)
  • Wrapping up

We need to do a better job of helping instructors finish and follow through.

For the past two years, we've required people doing the multi-week online version to submit a pull request and teach a five-minute lesson online in order to graduate. We've relaxed the lesson requirement for people doing the live version, since they do 3-5 recorded lessons during their training, but have kept the pull request requirement. About half do it within 2-4 weeks, but real life gets in the others' way, and it seems that people who don't complete promptly are unlikely to complete at all. We need to do better, but I'm not sure that requiring the pull request within (say) two weeks is either fair or realistic. (I hope that providing an archive of old assessment exercises for people to recycle will help.)

We also need to do a better job—a much better job—of engaging instructors once they have qualified. More than half of the people who have completed training in the last year have yet to teach for us. I don't think that's indifference on their part, but rather reflects the fact that we don't have enough small, easy-to-join events for them to participate in. Suggesting that they organize a workshop wherever they are isn't going to solve this: it takes a fair bit of time and effort to get one's first workshop going, and teaching for the first time without someone more experienced beside you can be pretty scary.

And finally, we need a better way to keep instructors engaged long-term. Many people complain that they can't keep up with changes to Software Carpentry's lessons and procedures, only a handful of people (almost all male) are active in our discussion list, and most of what we learn individually from teaching workshops either stays locked up between one pair of ears, or goes no further than a debriefing blog post. This isn't really an instructor training issue, but whatever changes we make to instructor training have got to lead people toward whatever we do about long-term engagement.

We need a better way to prioritize access to training.

Over 330 people are currently in queue for instructor training from 37 countries. Some of them have been waiting more than a year for a spot, and while we're training more instructor trainers, demand is always going to exceed capacity. The Leveller in me thinks that "first come, first served" is the fairest solution, but (a) we have promised seats to partners and affiliates as part of their agreements, (b) some of those 330 people are unlikely to ever teach for us specifically, and (c) a tenth or twentieth instructor at Euphoric State University seems less useful than a first or second in some region we haven't been to yet.

One option I'm personally opposed to is charging for instructor training. As several people observed, it seems unfair to ask someone to give us money so that they can then volunteer their time. On the other hand, we charge $20-50 per person for our workshops primarily because it cuts the no-show rate from 25-30% down to 5-10%. In that light, a similar threshold for instructor training doesn't seem unreasonable...

The biggest lesson of all, at least for me, is summed up in something that Adina Chuang Howe said at last week's Steering Committee meeting. Increasingly, Software Carpentry's mission isn't to teach scientists basic computing skills: it's to teach people how to do that, and to make sure that they have the lessons they need to do it without heroic effort. I'm not sure where this realization leads, but the change itself is a great testimonial to all the hard work that hundreds of you have put in over the last five years.

So much for lessons: what about plans? The first is to take a break until September: we've been running hard for the past few months, and need some time to rest and reflect. What happens after that depends on your comments on this blog post, and the solutions you can help us devise for the issues discussed above.