Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Ncrunch 5 Task Capacity Defect
fidel
#1 Posted : Tuesday, April 23, 2024 12:23:37 PM(UTC)
Rank: Member

Groups: Registered
Joined: 5/15/2023(UTC)
Posts: 11
Location: Ukraine

Was thanked: 1 time(s) in 1 post(s)
Ncrunch 5.6

I use integration tests within Ncrunch grid server.
With Xunit.

I need to run tests exactly one by one. ( on each node)

So on develop machine i disable paralell execution set Paralell Execution enabled = false set up maximum worker = 1

On remote machine which has Task Capacity = 16 it still run tests in parallel.

In my tests i write Start/Finish to text file and i see that it executes all tests in parallel.
Only one thing where i was able to do it one by one is to set on Grid Server on remote Task Capacity = 1.

This works weird, i thought if i use xunit collection attribute then it guarantees that only one execution in the same moment.
Same for ncrunch if i set Paralell Execution enabled = false i expect that even on remote servers it will run tests one by one.

Now I am confused, is that only one way in V5 to execute them one by one?

I found that my tests works when they distributed to 1 remote ncrunch test runner process (in Processing Queue window)
otherwise when they get to 2 processes (despite on option Max Threads = 1 and Allow paralell = False), they are executed simultaneously and fails.
How to avoid that without reducing grid Task Capacity on each grid node?
Remco
#2 Posted : Tuesday, April 23, 2024 12:56:07 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi, thanks for posting.

Disabling parallel execution should prevent tests being run at the same time by NCrunch on a single system. This means you can still end up with parallel execution if you're using a grid (for example, you might have one test running on your local machine, while another test runs on the remote grid node).

NCrunch doesn't integrate with Xunit's collection attributes. If you have xunit features set up to run tests in parallel, this does have the potential to undercut NCrunch's own parallelisation controls and can result in instability, including tests being run on background threads inside the test process.

It is not required to set your max processing threads to 1 to suppress parallel execution. Disabling parallel execution via setting should do the trick.

Something to be aware of is that it's still possible to have multiple processes running at one time, though when this happens NCrunch will only call them one at a time to execute tests. This is because NCrunch pools test processes for optimisation purposes.

If you have multithreaded behaviour in your tests (or xunit is somehow forcing its own parallelisation), this could cause parallel execution across these processes. For example, if you have a test that kicks off a background thread that continues runs after test execution is complete, this can then overlap execution of the next test. NCrunch cannot prevent such behaviour and it should be avoided in your code.

To introduce finer control of parallelisation, I recommend looking into ExcusivelyUsesAttribute.

Edit: I strongly suspect your parallel behaviour is being caused by xunit. I recommend removing/disabling your xunit collection attributes to see if this rectifies the behaviour.
fidel
#3 Posted : Tuesday, April 23, 2024 1:01:10 PM(UTC)
Rank: Member

Groups: Registered
Joined: 5/15/2023(UTC)
Posts: 11
Location: Ukraine

Was thanked: 1 time(s) in 1 post(s)
Thanks for comprehensive answer.

Looks like it was my bad.
So I disabled in Global settings Paralell Execution, but in Shared settings it was enabled.
When i disabled it, All tests now works super well as designed.



I overlooked that
Sorry.
Have a nice day!
1 user thanked fidel for this useful post.
Remco on 4/23/2024(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.030 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download