Armed with this impression and your Python-foo, you set off to write a
unittest extension that will let you mark certain tests as “TODO”. You want the test harness to count these tests differently than normal tests: TODO tests are supposed to fail, and you want to be notified they start unexpectedly passing.
You tinker a bit, you poke and you prod, and you wind up with your extension, and the whole thing works great. You just wish you hadn’t had to subclass
TextTestRunner to get the job done. You feel like it could have been easier, but you don’t pay it much mind. After all, you only needed the one extension.
A few months later, a different project has a need to run reference-count checks around each test case for a C extension module. Confident from your first experience extending
unittest, you head back into the code. A little later, you emerge, bearing the shiny new reference count-checking extension to
unittest. You again ended up subclassing
TextTestRunner, but again, it’s just one extension.
An hour later, your boss walks by and says that the ref-counting extension and the TODO extension need to be combined so they can be used together on a new project. No problem, you say; composing the two should be cake.
That thought lasts about as long as it takes to load the extensions in your editor of choice.
unittest might have been intended to be extended, but only in simple ways, and only by one extension at a time. I’ll save you the suspense; to combine the above extensions, you have to write a completely different third extension, which attempts to merge the two functionality sets as much as possible. You want to incorporate another extension, say one that logs the test results to a database? Tough luck.
unittest’s design is fundamentally broken. Little or no attempt was made to separate the different concerns at work here:
TestCase instances can determine what result logger to use and how exceptions are to be interpreted. Making
TextTestRunner use a subclass of
_TestTextResult means subclassing the runner object.
TestResult is responsible for converting tracebacks to a textual representation, even though this means that any result classes that want to do introspect the tracebacks end up completely rewriting much of
TestResult in the process.
That’s the problem; next time, the solution.