Friday, November 30, 2012

Teaching with Unit Tests

Unit tests, it seems, are good for more than just building robust software. I recently taught a Twisted class at Cisco, which included both a lecture and hands-on exercises. Some of the exercises were simple coding exercises: a specification the students needed to implement. One of the larger exercises, however, came with pre-written unit tests. The students needed to make all the tests pass, one by one, and the solution was finished when all the tests are done. Based on both my own observations and some feedback from the students, providing unit tests greatly improves the educational value of an exercise.

Creating The Exercise

Before explaining the benefits, I think it's worth explaining how I created the tests and what I provided the students. The exercise involved implementing a toy HTTP server, sufficient to answer simple browser requests. In order to create the tests I implemented a sample solution using test-driven development: first writing a test, then making it pass. I implemented the tests and corresponding code starting from basic features, and working up to more sophisticated features. Because I was using test-driven development, I was fairly confident (and the results from the students confirmed this) that the unit tests provided sufficient coverage to recreate my implementation. The students received only the tests, and just enough of the solution module to make it importable (some class definitions with no actual code).

In addition to unit tests, which the students were asked to solve in the given order, I also provided functional tests at appropriate points in the sequence of unit tests. For example, after implementing the minimal functionality necessary to parse and respond to a HTTP request, I provided a functional test where a real HTTP client talked to the server implementation. This often caught problems in the students implementation where their implementation was too fragile. I could have provided even more extensive unit tests, but functional tests did turn out to be a good way to catch coding issues I was unable to predict.

Once a functional test passed, the students could then run a demo server which they could point a browser at. After implementing half the tests, they were able to run a simple demo HTTP application, and when they finished all the tests, another more complex and interesting demonstration.

The Benefits to Students

  • Obvious Criteria for Success: Passing unit tests are a clear indication that the exercise had been solved correctly.
  • Guidance: Rather than having to design and structure an application with a new, unfamiliar API, the design is implicitly included. Plus, it's always clear what the next step is: make the next test pass.
  • Debuggability: Without unit tests, it's not clear how to approach a broken solution (assuming one can even recognize the brokenness). A failing unit test, on the other hand, exposes the interaction pattern with the code, and how it ought to work. A failing assertion points at what needs fixing.
  • A sense of achievement: Knowing you've gone from 2 passing tests to 3 passing tests gives a sense of accomplishment, and the knowledge that you are making progress. Without unit tests, it's hard to tell if you're getting anywhere at all; maybe the code you wrote is necessary, maybe it's isn't, but you won't really know until the full exercise is done. This is especially important for students who are taking longer to finish the exercise. Rather than feeling discouraged, as they would if they had no idea how much progress they had made, they know they are actually achieving something.

All this is based on observing the differences in students' learning and reactions between the exercises with and without unit tests. However, it's worth noting the unit test exercise was the fourth, so by that point the students did have more knowledge than when they started.

You Should Do It Too

The technique was successful enough that I am thinking that all coding exercises that teach a new library or API should include unit tests. When I have some time, I may write a post about some lessons for teachers that I learned watching students solve my exercise. And I would love to hear suggestions from people already using this technique. Test First has some information, but seems to mostly focus on teaching programming languages, as opposed to libraries or frameworks. I haven't yet had a chance to watch the videos, though. I know many others are using this technique as well.

2 comments:

  1. I've been using this technique to great success at the tutorials I've given at PyCon and other conferences for the past couple of years. Highly recommended.

    ReplyDelete
  2. Hi,

    That's the way I made the coding exercices in my 4 days Python training for beginners. All 12 coding exercices are guided by a README.txt that tells a story with doctests, and one or two Python files with bugs or missing code.

    ReplyDelete