Welcome Guest! To enable all features please Login or Register.



Tweak settings to behave like "dotnet test" runner with 1 process ?
#1 Posted : Wednesday, September 27, 2023 9:52:10 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 491

Thanks: 138 times
Was thanked: 64 time(s) in 62 post(s)

I am currently facing a problem in a XUnit project where we have a test collection of 1 time data setup, run multiple tests in different test fixtures, and then 1 time data tear-down.
These tests utilizes XUnit Collection attribute, so when running with dotnet test they run only the setup/teardown once.

However with NCrunch even though I set "Max number of processing threads" to 1, and "Max number of test runner processes to pool" to 1, they tend to end up in multiple processes causing this setup/teardown to be run multiple times making it seem that NCrunch is really slow.

Is there a "trick" in the settings to force NCrunch reuse the same process to simulate same behavior as dotnet test runner?

(I would preferably like an attribute with a Name parameter for this, that forces all tests having that attribute with same name to be run in same process, and in worst case scenario spin up a new process because this will be faster than utilizing 2 processes as long as these exclusively uses same resource, but maybe that's a good candidate for a feature request)

#2 Posted : Wednesday, September 27, 2023 11:11:25 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,866

Thanks: 910 times
Was thanked: 1229 time(s) in 1149 post(s)
Hi, thanks for sharing this issue.

Because of the design of the engine and the limitations of the frameworks we integrate with, we can't have NCrunch simulate the process hosting behavior of a serial test runner without axing many of the features that make it worth using. Marking the tests with ExclusivelyUsesAttribute will help here as it will increase the odds that they share the same process due to the re-use patterns, but there is no specific way to override the manner in which the engine allocates and re-uses processes outside of IsolatedAttribute (which is likely too targeted for your situation).

If you have a way of moving these tests into a separate test assembly, this will help you, as the test assembly/project itself is a determinant in whether the engine can re-use a process when starting an execution task. As you can imagine, each test process launched by NCrunch is set to target a specific project. There is no way the engine can take a test process that was launched for one test project and re-use it for another. If your process pool is large enough and you have all tests in the new test project set to make exclusive use of the same resource (via ExclusivelyUsesAttribute), you can effectively force the engine to only have one test process for these particular tests that cannot be used for any others. In theory, this should create the result you are after.

Note that you may still need to implement a static state flag to prevent the Xunit test collection setup from running multiple times in the same process. I would expect that it probably gets called by Xunit every time NCrunch invokes the test runner to run a new batch of tests, which doesn't seem like something you'd want.
1 user thanked Remco for this useful post.
GreenMoose on 9/27/2023(UTC)
Users browsing this topic
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.028 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download