Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
I want to run tests in parallel at the test class granularity...
Options
Previous Topic Next Topic
oriches
#1 Posted : Monday, October 15, 2012 4:59:44 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/15/2012(UTC)
Posts: 14
Location: United Kingdom

Thanks: 3 times
I have a scenario where I'm self hosting a RESTful web service inside the test class to test a REST client I've written. Each of these test classes creates a self hosted WebAPI service on a unique port and therefore I want to run all of the unit tests for a particular test class in a serial manner - i.e. the test class is only created once per running execution of nCrunch.

I would like to be able to run the tests in parallel at the test class granularity, will this be supported in the future?

Ideally I would like to select the test classes I want to run serially and have every other unit test run in parallel.


I have a blog post describing what I'm attempting to do:

http://awkwardcoder.blog...ervice-inside-test.html

If you need more info let me know...


Great product by the way and I'll be buying a licence :)


ta

Ollie
Back to top
Remco
#2 Posted : Monday, October 15, 2012 8:25:13 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi Ollie,

Thanks for posting, and for offering your support to NCrunch by buying a license!

I may have a magic trick to help you with this problem. It's the NCrunch.Framework.SerialAttribute. Any tests marked with this attribute will be mutually exclusive with everything else, which may give the result that I think you're after.

Still though, I generally try to encourage people to write tests in such a way as to support concurrency. Is the self-host service initialisation time the main constraint you're dealing with? Or is it that the self-host service impacts static data in such a way that it can only be initialised once?

On the side, I have to say that this is a really brilliant way to perform integration testing on REST clients. I didn't know it was so easily possible, otherwise I think I would have put it to use in many a project myself!


Cheers,

Remco
Back to top
oriches
#3 Posted : Monday, October 15, 2012 8:39:22 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/15/2012(UTC)
Posts: 14
Location: United Kingdom

Thanks: 3 times
Total agree with the idea of writing tests to support concurrency - nCrunch has really highlighted...

The issue I have is the fact only one service can use a HTTP port at anyone time, so for my test I have several test for the HTTP GET verb so I run all the tests on the same port by using the TestFixtureSetUp attributed method so the service is setup once for all the tests.

Now I could try selecting an incremental port number from static class so each test could then be run concurrently but this seems a little overkill, but i think I'll give it ago...

'I didn't know it was so easily possible' - good innit :)


O.
Back to top
Remco
#4 Posted : Monday, October 15, 2012 8:50:48 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Something you might find is that if you set the port numbers from a static member, you may still end up with the same problem when NCrunch tries to run the tests in parallel across multiple processes.

As there isn't really an easy way to share state between execution processes (and this would be a bad idea anyway), you may want to look into generating a random port number within a large range. Assuming the range is big enough, it's extremely unlikely that both executing tests will find a way to come up with the same number at the same time.

If you want a more reliable way to come up with a port number, you could also derive it from the currently executing process ID. As mentioned, NCrunch will only perform parallel execution across multiple processes, which means that the process ID of the test runner will always be different for each test being run in parallel. The ports 0-1000 are generally used by other protocols, and it isn't commonplace for a process to be numbered above 64535, so you might just want to do:

System.Diagnostics.Process.GetCurrentProcess().Id + 1000

See if this does the trick :)


Cheers,

Remco
Back to top
oriches
#5 Posted : Monday, October 15, 2012 9:30:06 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/15/2012(UTC)
Posts: 14
Location: United Kingdom

Thanks: 3 times
I'll give it ago and let you know...
Back to top
oriches
#6 Posted : Wednesday, October 17, 2012 4:38:41 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/15/2012(UTC)
Posts: 14
Location: United Kingdom

Thanks: 3 times
Found the issue and it wasn't nCrunch, it was how I was shutting down the WebAPI self hosting instance - I've blogged about it here http://awkwardcoder.blog...us-testing-and-self.html
Back to top
 |  Edit by user
Remco
#7 Posted : Wednesday, October 17, 2012 6:04:52 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Awesome, thanks for letting me know!
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.044 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download