Python Nose: Speed up the runner.
As your test base starts growing, also the time spent running tests grows up.
nose provides some mechanisms to divide and conquer your run plan and speed up the running time.
The usage of attributes could be something pretty useful to split and accelerate tests.
Test attributes decorated is located on the
nose.plugins collection. The
attr decorator can be used on any testeable function. The following example declares the
from nose.plugins.attrib import attr @attr(speed='slow') def test_slow(): # slow test
Once you have all your slow tests marked you can filter this attribute and
nose will run only the test cases flagged as slow.
$ nosetest -x -v -a speed=slow
If you want to specify multiple attribute filters, you can use a comma separated list.
Reference: Nose attributes
Nose offers the nose.plugins.multiprocess plugin that allows you to run test concurrently.
Please make sure that your test suite is ready for run in parallel and all of your tests don't depends on re-entrant context variables or global allocated resources, otherwise your tests will fall in unexpected behaviors.
If you have a
setup method that can be executed on every separated test you can use the
_multiprocess_can_split_ = True option.
class TestClass: _multiprocess_can_split_ = True @classmethod def setup_class(cls): ...
This means that the fixtures will execute multiple times, typically once per test, and concurrently.
If you have a
setup method that must be executed once and can be shared among the other processes you can use the
_multiprocess_shared_ = True option.
class TestClass: _multiprocess_shared_ = True @classmethod def setup_class(cls): ...
Then for running this suite from the command line you can specify something like:
$ nosetests --processes=NUMBER_OF_PROCESSORS
This will create separate processes for each test suite, depending on the
sharing type that you selected. Every test case will have a context for fixtures and shared resources.
Reference: Nose in parallel
Problems with parallel running
Number one issue with parallel test suites is related to non self contained pure unit tests that depends on external context resources that can fail into race conditions and other unexpected behaviors.
"But the biggest issue you will face is probably concurrency. Unless you have kept your tests as religiously pure unit tests, with no side-effects, no ordering issues, and no external dependencies, chances are you will experience odd, intermittent and unexplainable failures and errors when using this plugin. This doesn't necessarily mean the plugin is broken; it may mean that your test suite is not safe for concurrency."
Also because tests are splitted among processes, the results are not natively merged for coverage and reporting.
To merge the coverage reports on parallel tests you can use the nose-cov plugin running:
$ nosetests --with-cov --processes=4 tests/ $ coverage combine $ coverage report
XUnit reports is the same problem. So a few months ago some good hackers started coding on a multiprocess compatible
XUnit plugin and there is a beta plugin called nose_xunitmp
The basic usage of this plugin:
$ nosetests --with-xunitmp $ nosetests --xunitmp-file results.xml
This article concludes the series of
nosetests related ones. After reading this articles you should be able to write a simple test suite, add coverage and unit reporting capabilities, creating your own filters and plugins , add attributes to filter for and run them in parallel! Now is your turn to start exploring the python unit testing universe.
Good luck and have fun.