Python Nose: Speed up the runner.

This brief article is a continuation on the series Starting with Nose and Nose: Extending and customizing.

As your test base starts growing, also the time spent running tests grows up.

Fortunately nose provides some mechanisms to divide and conquer your run plan and speed up the running time.

Test Attributes

The usage of attributes could be something pretty useful to split and accelerate tests.

Test attributes decorated is located on the nose.plugins collection. The attr decorator can be used on any testeable function. The following example declares the speed attribute.

from nose.plugins.attrib import attr

def test_slow():  
    # slow test

Once you have all your slow tests marked you can filter this attribute and nose will run only the test cases flagged as slow.

$ nosetest -x -v -a speed=slow

If you want to specify multiple attribute filters, you can use a comma separated list.

Reference: Nose attributes

Parallel testing

Nose offers the nose.plugins.multiprocess plugin that allows you to run test concurrently.

Please make sure that your test suite is ready for run in parallel and all of your tests don't depends on re-entrant context variables or global allocated resources, otherwise your tests will fall in unexpected behaviors.

If you have a setup method that can be executed on every separated test you can use the _multiprocess_can_split_ = True option.

class TestClass:  
    _multiprocess_can_split_ = True

    def setup_class(cls):

This means that the fixtures will execute multiple times, typically once per test, and concurrently.

If you have a setup method that must be executed once and can be shared among the other processes you can use the _multiprocess_shared_ = True option.

class TestClass:  
    _multiprocess_shared_ = True

    def setup_class(cls):

Then for running this suite from the command line you can specify something like:

$ nosetests --processes=NUMBER_OF_PROCESSORS

This will create separate processes for each test suite, depending on the sharing type that you selected. Every test case will have a context for fixtures and shared resources.

Reference: Nose in parallel

Problems with parallel running

Number one issue with parallel test suites is related to non self contained pure unit tests that depends on external context resources that can fail into race conditions and other unexpected behaviors.

Please remember:

"But the biggest issue you will face is probably concurrency. Unless you have kept your tests as religiously pure unit tests, with no side-effects, no ordering issues, and no external dependencies, chances are you will experience odd, intermittent and unexplainable failures and errors when using this plugin. This doesn't necessarily mean the plugin is broken; it may mean that your test suite is not safe for concurrency."

Also because tests are splitted among processes, the results are not natively merged for coverage and reporting.

To merge the coverage reports on parallel tests you can use the nose-cov plugin running:

$ nosetests --with-cov --processes=4 tests/
$ coverage combine
$ coverage report

For XUnit reports is the same problem. So a few months ago some good hackers started coding on a multiprocess compatible XUnit plugin and there is a beta plugin called nose_xunitmp

The basic usage of this plugin:

$ nosetests --with-xunitmp
$ nosetests --xunitmp-file results.xml

Next steps

This article concludes the series of nosetests related ones. After reading this articles you should be able to write a simple test suite, add coverage and unit reporting capabilities, creating your own filters and plugins , add attributes to filter for and run them in parallel! Now is your turn to start exploring the python unit testing universe.

Good luck and have fun.

Jorge Niedbalski

Dev and Ops , and might be the opposite.

View or Post Comments