Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Theory based tests doesn't execute on distributed processing nodes
Options
Previous Topic Next Topic
eriove
#1 Posted : Friday, June 9, 2017 10:37:10 AM(UTC)
Rank: Member

Groups: Registered
Joined: 3/22/2016(UTC)
Posts: 21
Location: Sweden

Thanks: 14 times
Was thanked: 2 time(s) in 2 post(s)
Hi,

After adding a second distributed processing node I've started getting the message "This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures." for all tests with the [Theory] (xunit) attribute when they execute on the nodes and not on my local machine. Unfortunately I upgraded to 3.9.0.1 at the same time I added the second node so I'm not sure if it is a regression or something with my setup.


I've sent a bug report with log files.
Back to top
Remco
#2 Posted : Friday, June 9, 2017 11:21:20 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for sharing this problem. Could you confirm that you also upgraded the version of the grid node so that it is running v3.9? Also make sure that it isn't loading the xunit module from a prior version. You can check this by inspecting the version stamp of the nCrunch.Module.XUnit2.dll file on the grid node.
Back to top
eriove
#3 Posted : Friday, June 9, 2017 11:24:18 AM(UTC)
Rank: Member

Groups: Registered
Joined: 3/22/2016(UTC)
Posts: 21
Location: Sweden

Thanks: 14 times
Was thanked: 2 time(s) in 2 post(s)
I have files from the 26th of May on my local machine and the 1st of June on the grid node. I'll upgrade my machine and get back to you.
Back to top
1 user thanked eriove for this useful post.
Remco on 6/9/2017(UTC)
eriove
#4 Posted : Friday, June 9, 2017 11:39:08 AM(UTC)
Rank: Member

Groups: Registered
Joined: 3/22/2016(UTC)
Posts: 21
Location: Sweden

Thanks: 14 times
Was thanked: 2 time(s) in 2 post(s)
It was the 2015 version which that wasn't upgraded (which I don't use). But I remember that I replaced the xunit files manually a while ago. I removed those and ran a repair. Now it works.

Thanks!
Back to top
1 user thanked eriove for this useful post.
Remco on 6/9/2017(UTC)
eriove
#5 Posted : Friday, June 9, 2017 12:00:06 PM(UTC)
Rank: Member

Groups: Registered
Joined: 3/22/2016(UTC)
Posts: 21
Location: Sweden

Thanks: 14 times
Was thanked: 2 time(s) in 2 post(s)
I didn't check carefully enough. I still have the same problem as in the original post (using the latest version on all machines).

Now they sometimes fail on my local machine too and not just on the nodes.

I've also noticed that the code coverage shows that the test fails on the first line but the other lines are green. In other words, only some instances of the test fail. Not all of them

I'm getting closer. The tests that pass have input values without decimals while the tests that fail have decimal values.

I.e. a test with the attribute [InlineData(12, 217.0)] passes but a tests with the attribute [InlineData(12, 217.2)] fails. The tooltip when hovering over a failing test shows the numbers as 12, 217,2 despite my computer having decimal . and not decimal ,. The machine where it passes have decimal , and not .

If you want to reproduce this I would suggest that you have grid node with decimal comma, one with decimal dor and the client (where you run Visual Studio 2017) should use decimal dot.

I'll try to change all machines to the same settings and see if that is a viable workaround.
Back to top
 |  Edit by user
Remco
#6 Posted : Friday, June 9, 2017 12:31:48 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
I'm impressed you've managed to narrow this down as far as you have. What you're observing makes perfect sense. When NCrunch constructs an identifier for a testcase, it performs a .ToString() on the arguments without specifying any culture preference (oops). So if your decimal separator is different between machines in your grid, the test identifier will be different, and stuff will go wrong.

I'll see what I can do about getting a fix for this in the next build of NCrunch. For the time being, I recommend you use the same culture settings on your grid node as you do for your local dev machine.
Back to top
eriove
#7 Posted : Friday, June 9, 2017 12:45:25 PM(UTC)
Rank: Member

Groups: Registered
Joined: 3/22/2016(UTC)
Posts: 21
Location: Sweden

Thanks: 14 times
Was thanked: 2 time(s) in 2 post(s)
I'm surprised it has ever worked if that's the case (and it has). There seems to be something more to it. I've now set all machines to the same settings but still get "This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures." I've also restarted the grid nodes. Is there any cache I should clear?

I no longer see inconsistent , and . when hovering over the tests but the tests without any decimals are the only ones that work on the grid nodes.

Update: The tests only work on the machine that is mentioned in the "server" column for the project. So if the project has "Server" "local" then only tests without decimals or test running on "local" will pass

Another update: I had to copy the format settings to the system user, the number settings are per user and not machine wide. Now it is working. This also explains why it was working before I added another grid node. Then the service user on the grid node had the same setting as my user on my machine.

Adding the invariant culture in the ToString() will really help us. The grid node is in a data center in Norway (where IT sets Norwegian settings) and we share the grid node between between the Swedish and Lithuanian office.
Back to top
 |  Edit by user
Remco
#8 Posted : Friday, June 9, 2017 11:54:18 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
eriove;10569 wrote:


Adding the invariant culture in the ToString() will really help us. The grid node is in a data center in Norway (where IT sets Norwegian settings) and we share the grid node between between the Swedish and Lithuanian office.


Will do. You can expect this in the next build :)
Back to top
Remco
#9 Posted : Friday, June 16, 2017 4:05:21 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
Back to top
1 user thanked Remco for this useful post.
eriove on 6/16/2017(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.058 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download