unit test: class level setup


Microsoft Visual Studio Premium 2013 Version 12.0.21005.1 REL
Python Tools for Visual Studio 2.1.20211.00
Python 2.7.6

It appears that setUpClass() tearDownClass() as described here https://docs.python.org/2/library/unittest.html#setupclass-and-teardownclass are not working as expected. They seem to behave exactly like SetUp() TearDown() methods.


Zooba wrote Apr 17 at 3:40 PM

This is probably because we run each test in a separate instance of Python (which is also why running all tests is so slow). Each test really does need the class to be set up and torn down around it, because it's the only test being run.

If you have unittest.main() in your test file, you can right-click and choose "Start with/without Debugging" to run the whole file much more quickly, though without the nice UI. You can also put lots of tests into a separate project and give it a startup file that will run all of them.

We really need to write our own test runner instead of relying on the standard one, since we currently don't have a reliable way of extracting individual results out of the normal output. Using a custom test runner would let us write them to a TRX file (basically XML following the usual VS schema for test results) and then we can load them from that, though we'd probably also want some sort of socket/pipe connection to update the UI while it's running.

floristic wrote Apr 28 at 12:37 PM

Thanks for the hint with running tests via the context menu ( without UI ). We also really appreciate your plans concerning the custom test runner and I am looking forward for it to become part of the 2.1 Release - that would be awesome :)
I think for anybody with an expensive test environment setting it up on class level is indispensable.

Zooba wrote Apr 28 at 3:37 PM

Sorry to give you the wrong idea - there won't be a custom test runner for 2.1 unless someone outside of our team contributes one (along with all the integration tests we need to ensure it is reliable, or at least good enough code that it doesn't take us long to set it up). Some work in C# would be required too - it's not a trivial task.

These contributions are very welcome, especially since we have a very small team (compared to say, the C# team, or the teams working on other Python IDEs).