Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
disposer on testclass not run
Options
Previous Topic Next Topic
vegar
#1 Posted : Monday, June 19, 2017 1:57:46 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/19/2017(UTC)
Posts: 5
Location: Norway

Thanks: 1 times
I have a base class for all my database tests. It will provide the test with namespaced collection (mongo), and delete the collections afterwards.

It doesn't seems like the cleanup code is run when ncrunch is running my tests, though. The result is mongodb running out of namespaces.

I'm using MsTest for these tests.

Code:


[TestClass]
public class MyDbTest : MsDbTest {

  [TestMethod]
  public void MyTest() { }

}

public class MsDbTest : IDisposable {


        public void Dispose()
        {
            Dispose(true);
            GC.SuppressFinalize(this);
        }

        protected virtual void Dispose(bool disposing)
        {
            TeardownDatabase();
        }

}


I've also tried adding a `TestCleanup` attribute on the `TeardownDatabase` method but that didn't do any difference.

Any tips?
Back to top
Remco
#2 Posted : Monday, June 19, 2017 10:19:15 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for posting!

As with standard MSTest, NCrunch's MSTest adapter executes the TestCleanup and ClassCleanup methods on completion of the test and fixture accordingly (assuming these follow standard MSTest syntax). However, I don't recommend using these for critical cleanup steps. This is because there is no way to guarantee that these cleanup steps will always be executed. For example:

- The process may explode with a stack overflow or outofmemory exception
- The test code may malfunction and cause the process to hang, requiring it to be terminated automatically (via timeout) or manually
- If NCrunch gets suddenly shut down, all runner processes will be immediately terminated regardless of their status
- If your PC loses power, the tests may run partway but never complete

With a standard end-to-end test runner, these sorts of use cases aren't a big deal, because most hours of the day you won't be running your tests. However, under NCrunch, your tests are run an order of magnitude more frequently and very often on half finished code, making unexpected interruption much more common.

A better way is actually to clean up at the start of the test run. A test should check for leftover state from a previous run, and deliberately remove it at the beginning of its run. In this way, there is no risk of the state being left behind and interfering with future runs.
Back to top
vegar
#3 Posted : Monday, June 19, 2017 10:35:21 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/19/2017(UTC)
Posts: 5
Location: Norway

Thanks: 1 times
Thanks!

I would still love to understand why it fails to clean up as often as it does under what appears as normal circumstances.

But your advice on cleaning up before executing instead of after was interesting. Of cause that means I have to find a different strategy for namespaces. Currently I use guids.
Back to top
Remco
#4 Posted : Monday, June 19, 2017 10:39:28 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Can you check whether the cleanup methods get executed during a debug run? NCrunch should hit breakpoints in these if you run the tests with a debugger. Note that MSTest is quite particular about syntax and structure with these sorts of methods.

Guids are fine for namespaces as long as you have a way to find them at the start of the next test run. I'm not sure if your DB system has a way to scan over databases or if parallel execution is something you're doing at the moment. Setting the database name or namespace equal to the ID of the currently executing process is often a reliable way to find databases that don't have an associated process executing tests over them.
Back to top
vegar
#5 Posted : Monday, June 19, 2017 10:46:40 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/19/2017(UTC)
Posts: 5
Location: Norway

Thanks: 1 times
The challenge about namespaces is that multiple test runs on multiple machines uses the same hosted mongodb.
But pod was a good idea. In combination with machine name it should be a pretty safe strategy I guess - cause all tests run by ncrunch runs by same process?

I get 'black dots' on the cleanup code, and breakpoints aren't hit. This didn't use to be a problem though, so that's why I thought it was ncrunchs fault. I'll fiddle a little with the attributes and see if I find a reason, but most likely I'll rather fix the problem your way :-)
Back to top
Remco
#6 Posted : Tuesday, June 20, 2017 12:13:48 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
vegar;10657 wrote:
The challenge about namespaces is that multiple test runs on multiple machines uses the same hosted mongodb.
But pod was a good idea. In combination with machine name it should be a pretty safe strategy I guess - cause all tests run by ncrunch runs by same process?


NCrunch will run tests using multiple processes simultaneously, but only when set up to use parallel execution. When using parallel execution, you'd want to use a different namespace anyway so the tests don't interfere with each other. Including the machine name in the namespace is a very good idea.

vegar;10657 wrote:

I get 'black dots' on the cleanup code, and breakpoints aren't hit. This didn't use to be a problem though, so that's why I thought it was ncrunchs fault. I'll fiddle a little with the attributes and see if I find a reason, but most likely I'll rather fix the problem your way :-)


I haven't had any reported problems in this area and there are a good number of tests over the behaviour of MSTest in this area, but I'm not going to draw any conclusions. If you find anything here that you're sure isn't right, do let me know.
Back to top
vegar
#7 Posted : Tuesday, June 20, 2017 9:04:24 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/19/2017(UTC)
Posts: 5
Location: Norway

Thanks: 1 times
I've not been able to make the disposer run, but the TestCleanup method started doing its job as soon as I made it public :-)

Even parallel test runs want run the same test concurrently? Using the TestContext I can combine the machine name and the test name to get a pretty accurate / safe namespace, I think.

Thanks!
Back to top
Remco
#8 Posted : Tuesday, June 20, 2017 10:36:02 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
vegar;10663 wrote:
I've not been able to make the disposer run, but the TestCleanup method started doing its job as soon as I made it public :-)


Interesting. I wasn't aware that IDisposable was used by MSTest during cleanup. A quick check now confirms that MSTest does call this method. I'll need to update NCrunch to align with this feature.

vegar;10663 wrote:

Even parallel test runs want run the same test concurrently? Using the TestContext I can combine the machine name and the test name to get a pretty accurate / safe namespace, I think.


Parallel execution under NCrunch goes across multiple processes. The docs give some in depth detail on how to use it and control it. Your use case is not unusual and could be made to work very well under NCrunch.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.063 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download