Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Parallel Execution based on Test Framework
michaelsync
#1 Posted : Thursday, October 1, 2015 2:30:54 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 10/1/2015(UTC)
Posts: 1
Location: Australia

Thanks: 1 times
Hello,

According to "Parallel Execution" https://www.ncrunch.net/...pts_parallel-execution, it said that NCrunch will never run tests concurrently within the same process - so you don't need to be concerned about concurrent access to static members, etc.

But we are using xunit version 2.0 and it allows us to run the tests from different class (not just an assembly) run against each other in parallel. Is there any way to make NCrunch respect the parallel execution of test frameworks instead of coming up with its own parallel execution model?

We may use the different test frameworks and those frameworks supports different or same or similar way to execute the tests in parallel. Some frameworks might even allows user to override or configure how we want to execute the tests.

What I found so far with NCrunch is that it asks to set the concurrency options and I think that option might be different from what test framework that we use offer. So, we are able to run our tests from Visual Studio Test Explorer or Reshaper unit test running but some tests are failed randomly in NCrunch tests.

Is there any way to fix that issue? In other words, is there any way to make NCrunch to follow what I configure for xunit?

Thanks!
Remco
#2 Posted : Thursday, October 1, 2015 2:48:37 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for posting.

NCrunch will deliberately disable any and all parallel execution performed by frameworks it invokes. This includes the new parallelisation features in Xunit V2. Under NCrunch, MSTest is actually emulated rather than integrated, and NCrunch's implementation of MSTest won't perform any parallel execution.

The reasons for this behaviour are as follows:
1. It's simpler - In this way, you don't need to consider parallel execution at multiple levels. You can always rely on NCrunch being the only source of parallel execution
2. Resource management - NCrunch needs to strictly control/limit the resources used during build and test execution. There are situations where this just isn't possible (i.e. you may have multithreaded code under test), but the intention is that the engine won't use more threads that you tell it to. This is to prevent the background execution from taking over your system and interfering with the foreground IDE.
3. It's impossible to implement any other way - NCrunch needs to have a distinct synchronous start and end for every test that is run. If tests are being run on multiple threads inside the same process, this is impossible. Let's just say for example you have a test that kicks up a background thread that does some work. NCrunch normally will monitor all code executed by this background thread and mark it as executed by the test it knows is running (because its the ONLY one running). If there were multiple tests running in parallel on different threads, NCrunch would have no way to differentiate which test was actually responsible for the background execution. This would mean that code executed by one test would be considered as executed by other. With the current implementation you will only see this sort of result if you kick off an unbounded thread in one test that continues to execute across other tests executed later in the run.

So in summary, I'm afraid the answer to your question is no. The only way to perform parallel execution under NCrunch is by using the parallelisation features in NCrunch itself.
1 user thanked Remco for this useful post.
michaelsync on 10/1/2015(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.036 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download