disposer on testclass not run

I have a base class for all my database tests. It will provide the test with namespaced collection (mongo), and delete the collections afterwards.

It doesn't seems like the cleanup code is run when ncrunch is running my tests, though. The result is mongodb running out of namespaces.

I'm using MsTest for these tests.

[TestClass]
public class MyDbTest : MsDbTest {

  [TestMethod]
  public void MyTest() { }

}

public class MsDbTest : IDisposable {


        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        protected virtual void Dispose(bool disposing)
        {
            TeardownDatabase();
        }

} 

I've also tried adding a `TestCleanup` attribute on the `TeardownDatabase` method but that didn't do any difference.

Any tips?

Hi, thanks for posting!

As with standard MSTest, NCrunch's MSTest adapter executes the TestCleanup and ClassCleanup methods on completion of the test and fixture accordingly (assuming these follow standard MSTest syntax). However, I don't recommend using these for critical cleanup steps. This is because there is no way to guarantee that these cleanup steps will always be executed. For example:

- The process may explode with a stack overflow or outofmemory exception
- The test code may malfunction and cause the process to hang, requiring it to be terminated automatically (via timeout) or manually
- If NCrunch gets suddenly shut down, all runner processes will be immediately terminated regardless of their status
- If your PC loses power, the tests may run partway but never complete

With a standard end-to-end test runner, these sorts of use cases aren't a big deal, because most hours of the day you won't be running your tests. However, under NCrunch, your tests are run an order of magnitude more frequently and very often on half finished code, making unexpected interruption much more common.

A better way is actually to clean up at the start of the test run. A test should check for leftover state from a previous run, and deliberately remove it at the beginning of its run. In this way, there is no risk of the state being left behind and interfering with future runs.

Thanks!

I would still love to understand why it fails to clean up as often as it does under what appears as normal circumstances.

But your advice on cleaning up before executing instead of after was interesting. Of cause that means I have to find a different strategy for namespaces. Currently I use guids.

Can you check whether the cleanup methods get executed during a debug run? NCrunch should hit breakpoints in these if you run the tests with a debugger. Note that MSTest is quite particular about syntax and structure with these sorts of methods.

Guids are fine for namespaces as long as you have a way to find them at the start of the next test run. I'm not sure if your DB system has a way to scan over databases or if parallel execution is something you're doing at the moment. Setting the database name or namespace equal to the ID of the currently executing process is often a reliable way to find databases that don't have an associated process executing tests over them.

The challenge about namespaces is that multiple test runs on multiple machines uses the same hosted mongodb.
But pod was a good idea. In combination with machine name it should be a pretty safe strategy I guess - cause all tests run by ncrunch runs by same process?

I get 'black dots' on the cleanup code, and breakpoints aren't hit. This didn't use to be a problem though, so that's why I thought it was ncrunchs fault. I'll fiddle a little with the attributes and see if I find a reason, but most likely I'll rather fix the problem your way :-)

vegar wrote:The challenge about namespaces is that multiple test runs on multiple machines uses the same hosted mongodb.
But pod was a good idea. In combination with machine name it should be a pretty safe strategy I guess - cause all tests run by ncrunch runs by same process?

NCrunch will run tests using multiple processes simultaneously, but only when set up to use parallel execution. When using parallel execution, you'd want to use a different namespace anyway so the tests don't interfere with each other. Including the machine name in the namespace is a very good idea.

vegar wrote:
I get 'black dots' on the cleanup code, and breakpoints aren't hit. This didn't use to be a problem though, so that's why I thought it was ncrunchs fault. I'll fiddle a little with the attributes and see if I find a reason, but most likely I'll rather fix the problem your way :-)

I haven't had any reported problems in this area and there are a good number of tests over the behaviour of MSTest in this area, but I'm not going to draw any conclusions. If you find anything here that you're sure isn't right, do let me know.

I've not been able to make the disposer run, but the TestCleanup method started doing its job as soon as I made it public :-)

Even parallel test runs want run the same test concurrently? Using the TestContext I can combine the machine name and the test name to get a pretty accurate / safe namespace, I think.

Thanks!

vegar wrote:I've not been able to make the disposer run, but the TestCleanup method started doing its job as soon as I made it public :-)

Interesting. I wasn't aware that IDisposable was used by MSTest during cleanup. A quick check now confirms that MSTest does call this method. I'll need to update NCrunch to align with this feature.

vegar wrote:
Even parallel test runs want run the same test concurrently? Using the TestContext I can combine the machine name and the test name to get a pretty accurate / safe namespace, I think.

Parallel execution under NCrunch goes across multiple processes. The docs give some in depth detail on how to use it and control it. Your use case is not unusual and could be made to work very well under NCrunch.