Software Carpentry Workshop Attendance: a New Zealand Perspective

This post originally appeared on the Software Carpentry website.

Having taught and helped at a series of workshop over the past few months Tom Kelly, PhD Candidate in Genetics at the University of Otago, wrote up some of his reflections on the issues related to workshop attendance. This spurred further discussion via email among the New Zealand instructors. We decided to put these thoughts together in hope that this could help other sites struggling with the attendance problems.

Please note that these are the authors’ views and thus they should not be treated as representative for their home institutions.

Tom based his opinions after having taught at various workshops in Australia and in New Zealand, including Research Bazaar 2015 at the University of Melbourne, Research Bazzar 2016 at the University of Otago, University of Otago, University of Canterbury, and NeSI. There have been several other workshops in New Zealand facilitated by NeSI, in Auckland, Wellington, and Palmerston North over the past year.

Main point

Over the course of several workshops we’ve had relatively minor problems with “no-shows” (people signing up and not attending) or “drop-offs” (people not returning for future days or sessions). However, in the case of the oversubscribed workshops it was still somewhat frustrating. This has led to discussions about how we may address the issues related to the attendance to ensure that others who would have attended for the entire workshop, but ended up on the waiting list, do not miss out on places.

Issue 1: No-shows

At our most recent workshop in the University of Otago we had 21/25 attendees who signed up attending. At previous workshops this had also been a bit of an issue, being as high as 25% of no-shows in Christchurch in February 2015. I know that this issue is not specific to these sites or to New Zealand itself. Shortly after I got involved in Software Carpentry, I had a chance to talk to Bill Mills who was visiting from Canada to help boot-up the workshops and train some instructors. Bill did say that they usually have 25-30% no show in US/Canada so our attendance figures are not too bad compared to other free Software Carpentry events.

Issue 2: Departures over time

A larger concern to me is the number of participants who attend for the beginning of a multi-day workshop and do not return for the final sessions. Some participants may be leaving midway because it just doesn’t work for them (thankfully, this is rare). Some will be interested only in a particular session, such as biologists who may attend only for the R module, even if we are encouraging them to attend the full course.

With others it may be difficult to address, particularly if they don’t leave any feedback on reasons why they left. Though, we can assume that some participants may have to leave early due to other committments such as running lab experiments or childcare responsibilities. So at our recent workshop at the University of Otago we tried splitting it into 3 shorter days, rather than 2 full ones.

Approach 1: Registration Fee

We discussed further no-shows with Bill Mills when we were doing ResBaz/SWC in Melbourne and Christchurch. He mentioned a solution suggested by Software Carpentry of applying a small registration fee to make sure those who register actually attend or cancel giving the organisers some notice. Based on the experiences of several hosts accross this usually results in no-show numbers dropping to below 5%.

Whilst this is certainly an option to consider, in many local contexts this would not be possible. There are complications with the university local regulations. Some universities charge venue fees unless the event is free, run at cost, or for the benefit of staff and students. Another problem with charged events (even with a small fee) is that it may create disparity between research groups where some are funded from the lab and others need to foot the bill themselves due to financial or adminstrative constraints. Eventbrite makes it easier for the hosts in terms of handling payments and registrations but within the University system it would create issues for labs that want to pay (through Eventbrite) for their members to attend - not insurmountable, but just extra hassle.

There are also some cultural aspects. For example, New Zealand may differ to other places where ticketed events have been tried. We don’t have a tipping culture, one of the largest home-supplies supermarket chains has the slogan “Where Everyone Gets a Bargain”, and another grocery supermarket chain proudly announces that it has “NZ’s lowest prices”. These stores are widely successful. It can be said that many of us see this as a good deal rather than appearing cheap, particularly among the university student population. Many people here view a bargain or freebie positively, so I don’t think the event is under-valued being free. However, it would be interesting to see if any other NZ sites have tried a paid ticketed event to boost attendance rates and how this compares to other countries.

Approach 2: Catering

Another suggestion to raise numbers is providing catering to boost numbers (possibly registration fees can be used for that) which we tried at Dunedin Research Bazaar last February. However, we had issues with overcatering for those did not stay for lunch and we still had dwindling numbers by the last day. I think the “come back for day 3” rate was higher in our most recent Otago workshop due to combined Git+Bash sessions on days 1 and 3. Unfortunately some participants did still give the impression of only wanting to attend the R session (or Python) but most seemed to give the rest a shot. And even the catering was not enough of an attraction.

Dwindling numbers seems to be a bigger problem with longer (3 day) events but there are higher costs for catering an event this long. Reducing the length of each day was another approach we’ve tried as discussed here.

Approach 3: Blacklist

Another approach could be the hosts checking the actual attendance and keeping a record of people who habitually don’t show up without giving notice. They are then only able to sign up if there are places left right before the event.

At the recent University of Otago workshops no one missed out due to no shows. Generally, we manage to let most of our waiting list in with cancellations anyway. It would be interesting to know if particular people (groups, or institutions) are signing up and not coming recurrently, but a blacklist (as some SWC sites have done) may be overreacting.

This appears to be a rather drastic solution and thus needs to be treated with care. There are many other understandable reasons why a participant may not be able to attend at the last minute which would be difficult to monitor, such as illness or bereavement. There may be students who choose (or feel pressured) to prioritise their experimental research over the workshop on the day. It’s likely preaching to the choir to even mention how counterproductive this lack of training can be in the long-run. However, a punitive approach such as a blacklist is not an appropriate way to encourage engagement in our workshops over research activities.

We consider a blacklist a last resort over the current first-come-first-served sign up system to consider only if people are repetively missing out. Perhaps a whitelist of people who missed out last time would be less punitive? We could either bump them up to the top of the waiting list or email them about the workshop in advance of public announcement. This would give potential participants more incentive to sign up even if the current workshop is full and may give a better indication of how much interest there would be in a future workshop.

Approach 4: Overbooking

This approach is notoriously used by some airlines. Many of you might have experienced a frustrating time at the gate when it has turned out that you actually don’t have a seat even though you do have the ticket. Then it gets to an exciting action when the airline tries to bribe the passengers with the allocated seat to give it up (for cash) and take one of the subsequent flight (possibly next day). Neither Software nor Data Carpentry are aiming to go that way but it may be tempting for hosts to allow for a high number of sign ups (say 45) with an assumption that there will be 20-30% no-show rate, particularly if a larger venue and additional helpers are available.

In larger (parallel session) events, such as ResBaz, feedback has been overwhelmingly positive for the inclusion of ‘helpers’. They can somewhat mitigate issues with a larger group offering one-on-one assitance when needed and getting participants back on track so they can follow along after falling behind of technical problems. We encourage helpers to be proactive at larger events, or those covering more advanced content, checking on participants when they get withdrawn or quiet rather than waiting for the sticky notes. The larger a group, the wider the range of pace and learning styles will be there. If participants have raced ahead of the content this is also a good opportunity to encourage them to work with their neighbours, try out extension challenge questions, or discuss how the tools involved could be applied to their work.

However, one problem that Wolfgang Hayek, NZ Instructor based at NIWA and NeSI, has seen, is that venues get very crowded if turnout is large, with attendees complaining accordingly in their feedback. Sticking to the recommended number of attendees is definitely a good idea. For example, in Wolfgang’s experience, the Wellington Victoria University ResBaz was a very relaxed event, at least partly due to its lower attendance. The Git session that Wolfgang has taught there was a lot more interactive than sessions that he had taught at other events, which made it quite enjoyable for everyone (many questions were asked and issues discussed, attendees participated more in the hands-on sections). While it is clear that we want to maximise efficiency of these events, there is also a positive side of having lower attendance, too.

Approach 5: Establishing rapport with the participants

Another alternative to the carrot and the stick is trying to establish close communication with the participants. Mik Black of the University of Otago said that him being a very hands-on person also helped with attendance numbers: particularly when co-ordinating the larger ResBaz event with parallel sessions. He sent repeated emails to registrants with reminders to tell the hosts if they couldn’t come as there was a waiting list. That was somewhat effective but there were still had no-shows (plus drop-offs after the first day). It also worked at the time because Mik needed to email about other ResBaz details at the same time (venue, schedule, laptop setup, etc) - he wasn’t just spamming them with “are you still coming?” every two days.

Sung Bae of the University of Canterbury (and previously NeSI) who has hosted and taught at a number of workshops across New Zealand developed a habit of going around the participants with the guest list and making them a name tag on the spot (and checking the attendance at the same time). Sung found it was helpful to build up personal connections with them (that helped him to remember their names too) and he also produced attendance lists from events he led. It possibly could help mitigate the number of drop-offs on the subsequent days of the workshop.

We recognise that there is no silver bullet to help us sort out the attendance issues. However, there may be various ways these problems can be mitigated. The experiences from Software and Data Carpentry workshops can also possibly translate to other training that many members of this community run.