Why This Is Hard (Part Deux)

This post originally appeared on the Software Carpentry website.

I pointed a grad student at the IPython tutorial and made some notes as she started to work through it. (She has used the vanilla Python interpreter before, but is still a novice.) Here's what happened.

The first thing the tutorial mentions is tab completion, which she's used in other tools, so she tried that:

>>> x = 'hello'
>>> x.<TAB> # shows methods of the string --- cool!

Does it work directly with strings?

>>> 'abc'.<TAB> # shows --- what?

It took me a moment to figure out it was showing us all the files in the current working directory whose names begin with a '.' character. Since these files are normally hidden, I don't think the grad student had ever seen them; I also don't think she'd have figured out what they were on her own.

>>> x? # gets help for the string referred to by x --- cool!

The information itself isn't very helpful for novices, but she was pleased it was there. Now what about this?

>>> 'abc'?

Uh oh—it tells her "Object `` not found." Given that we've been using:

>>> name = 'Perry'
>>> len(name)

and:

>>> len('Perry')

interchangeably, this confused her. She moved on, but I think it got filed as "weird arbitrary computer stuff", which is exactly what we're trying to clear up.

Now, coming from RStudio, one of the things she wants is to be able to edit while in the interpreter. And woo hoo, there's an '%edit' magic in IPython. But '%edit' brought up Vim, which may be the hardest editor in the world to get out of if you've never seen it before. There's nothing in the tutorial to tell her how to change the editor, and the word '%edit' doesn't link to any docs either. If I hadn't been there, she would have been toast. I'm not sure what to recommend, but unless you think ':wq' is intuitive or discoverable, this is a problem.

And here's something even harder: "%%edit x = 5" opened Vim with the line:

get_ipython().magic(u'%edit x = 5')

When she quit with 'q!', Vim was immediately reopened with the same line. And again, and again, until she killed the window. I opened a fresh window and restart IPython for her. We then typed:

>>> %%edit x = 5

and got a traceback starting with the error message:

WARNING: Argument given (x = 5) can't be found as a variable or as a filename.

Er, what? We then assigned 5 to x and tried the '%%edit x = 5' line again—same traceback. Hm. What if we try:

>>> %edit

all by itself, quit from Vim, and then try:

>>> %edit x = 5

Yup, that reproduces the "Vim forever!" problem.

At this point, we're five minutes in, and has spent almost all of those five minutes wrestling with things that seem arbitrary to her, and that she wouldn't have been able to fix on her own. She has also asked what "cell magic" is, and I've punted by saying, "Oh, it's just multi-line stuff." I don't think that clarified anything for her, but luckily she was immediately distracted by Vim rising from its grave and forgot the question.

She then tried playing around with shell commands from inside Python at my request (since we'd like to stop teaching the shell), and again there was some confusion:

>>> %cd tmp

works fine, but:

tells us that the magic function `ls` is not found. She can use the shell escape '!ls' instead, but she'll have to keep a cheat sheet to remember which are '%' and which are '!', and it'll be a while before she understands why some require one prefix and some require the other, but at least when it doesn't work, it doesn't work.

But then she tried this:

>>> !cd ..

to get back to her home directory from the 'tmp' directory she changed into earlier. It looks like it worked, but '!ls' shows that it hasn't. I understand what's going on—the 'cd' happened in a sub-shell that's immediately discarded, rather than in the interpreter—but she's completely confused at this point.

None of this is intended as a criticism of IPython: in the hands of someone who has mastered the underlying concepts, it's a great power tool, and I'm not going to switch back to the generic Python interpreter any time soon. But as a novice, this grad student doesn't have a clear mental model of how the operating system, the shell, and the Python interpreter relate to each other, so anything that moves back and forth between levels leaves her bewildered. Explaining things categorically therefore doesn't work (yet), because she doesn't have the categories. Equally, more documentation isn't a solution, because looking something up (i.e., adding information to existing concepts) and forming or internalizing concepts are different cognitive processes.

What about putting safeties on the tool itself, i.e., taking features out of IPython to reduce the number of "mistakes" novices could make, or the number of times they'd say "huh?" The problem with that approach is that every one of those features is there for a reason: people really do want to run arbitrary shell commands from inside IPython, and change its own notion of the current working directory, and so on. As Mark Chu-Carroll pointed out, some of this stuff really is irreducibly hard, because some of the things we're trying to do really are complex. Finding ways to "cook" it to make it more digestible is the heart of instructional design, and perhaps the greatest challenge we now face.