Hello,
In my project I have several test projects. If I use "local" NCrunch, then everything works great, as expected. However, I am trying to run these tests on a grid node and I am having all kinds of trouble there.
It seems that the errors I receive are related to System.Runtime.dll. I'm not sure what data to provide so here are a couple of different files.
This is the output from NCrunch test window inside Visual Studio for one of the test projects: https://gist.githubusercontent.com/rog1039/e8472973e5720f1527dfff021afc6e60/raw/c86247ab0d193675fac189472ec21037285f65d0/gistfile1.txt
I turned on fuslogvw logging and there are many (perhaps 20) or so logs in the list related to ncrunch. I picked one in particular that looked possibly interesting:
https://gist.githubusercontent.com/rog1039/1bf62608707be8011e8f67fc523c2d1f/raw/10b54cfc897a7a37b8b4efda349b9150dffd2a55/Fuslogvw
I have tried running this same tests on the grid node computer using locally installed Visual Studio and NCrunch that works great as well.
Both computers are running VS 15.9.0 and NCrunch 3.22.0.1 on Windows 10 1803 with .net 4.7.2 installed.
Thanks,
Paul
Remco NCrunch Developer
#12815
15 Nov 2018 00:16 UTC
Hi Paul,
Thanks for sharing this problem. Troubleshooting assembly resolution issues like this one in .NET core is a horrific nightmare because assembly resolution has become very complex under .NET core.
Could you check whether the grid node has the same versions of the .NET core runtime and Visual Studio 2017 installed as you have on the client connecting to it?
Something to try also is making sure that the Nuget packages on the grid node are an exact match for those on your client machine. Clearing out and copying over the Nuget user packages directory is often a good way to do this. Often the grid node will store its Nuget packages under your snapshots directory, depending on your configuration.
VS 15.9 is a very new drop of VS. We're doing some testing on this side to see if it has any new issues in it that require handling from our side.
Thanks for sharing this problem. Troubleshooting assembly resolution issues like this one in .NET core is a horrific nightmare because assembly resolution has become very complex under .NET core.
Could you check whether the grid node has the same versions of the .NET core runtime and Visual Studio 2017 installed as you have on the client connecting to it?
Something to try also is making sure that the Nuget packages on the grid node are an exact match for those on your client machine. Clearing out and copying over the Nuget user packages directory is often a good way to do this. Often the grid node will store its Nuget packages under your snapshots directory, depending on your configuration.
VS 15.9 is a very new drop of VS. We're doing some testing on this side to see if it has any new issues in it that require handling from our side.
Edited 15 Nov 2018 00:17 UTC
The test projects in question are actually .net Framework projects.
In the solution there are (10) .net Framework projects and (2) .net Core projects.
For .NET Core: there is a WebApi project and a test project for the WebApi project. These are not working on grid nodes either. However, I'm mostly concerned with the .NET Framework projects and test projects.
Here is the output from the command line on the two computers:
Grid Node:
C:\Users>dotnet --list-sdks
2.1.202 [C:\Program Files\dotnet\sdk]
2.1.500 [C:\Program Files\dotnet\sdk]
C:\Users>dotnet --list-runtimes
Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
On my local PC:
C:\Users>dotnet --list-sdks
1.1.0 [C:\Program Files\dotnet\sdk]
1.1.6 [C:\Program Files\dotnet\sdk]
1.1.7 [C:\Program Files\dotnet\sdk]
1.1.8 [C:\Program Files\dotnet\sdk]
1.1.9 [C:\Program Files\dotnet\sdk]
1.1.10 [C:\Program Files\dotnet\sdk]
1.1.11 [C:\Program Files\dotnet\sdk]
2.0.2 [C:\Program Files\dotnet\sdk]
2.1.2 [C:\Program Files\dotnet\sdk]
2.1.4 [C:\Program Files\dotnet\sdk]
2.1.100 [C:\Program Files\dotnet\sdk]
2.1.102 [C:\Program Files\dotnet\sdk]
2.1.103 [C:\Program Files\dotnet\sdk]
2.1.104 [C:\Program Files\dotnet\sdk]
2.1.200 [C:\Program Files\dotnet\sdk]
2.1.201 [C:\Program Files\dotnet\sdk]
2.1.202 [C:\Program Files\dotnet\sdk]
2.1.300 [C:\Program Files\dotnet\sdk]
2.1.400 [C:\Program Files\dotnet\sdk]
2.1.402 [C:\Program Files\dotnet\sdk]
2.1.403 [C:\Program Files\dotnet\sdk]
2.1.500 [C:\Program Files\dotnet\sdk]
C:\Users>dotnet --list-runtimes
Microsoft.AspNetCore.All 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 1.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Both computers, my PC and the grid node, are on Visual Studio 15.9.
I copied the user .nuget folder over to the grid node, however, that did not have any noticeable effect.
In the solution there are (10) .net Framework projects and (2) .net Core projects.
For .NET Core: there is a WebApi project and a test project for the WebApi project. These are not working on grid nodes either. However, I'm mostly concerned with the .NET Framework projects and test projects.
Here is the output from the command line on the two computers:
Grid Node:
C:\Users>dotnet --list-sdks
2.1.202 [C:\Program Files\dotnet\sdk]
2.1.500 [C:\Program Files\dotnet\sdk]
C:\Users>dotnet --list-runtimes
Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
On my local PC:
C:\Users>dotnet --list-sdks
1.1.0 [C:\Program Files\dotnet\sdk]
1.1.6 [C:\Program Files\dotnet\sdk]
1.1.7 [C:\Program Files\dotnet\sdk]
1.1.8 [C:\Program Files\dotnet\sdk]
1.1.9 [C:\Program Files\dotnet\sdk]
1.1.10 [C:\Program Files\dotnet\sdk]
1.1.11 [C:\Program Files\dotnet\sdk]
2.0.2 [C:\Program Files\dotnet\sdk]
2.1.2 [C:\Program Files\dotnet\sdk]
2.1.4 [C:\Program Files\dotnet\sdk]
2.1.100 [C:\Program Files\dotnet\sdk]
2.1.102 [C:\Program Files\dotnet\sdk]
2.1.103 [C:\Program Files\dotnet\sdk]
2.1.104 [C:\Program Files\dotnet\sdk]
2.1.200 [C:\Program Files\dotnet\sdk]
2.1.201 [C:\Program Files\dotnet\sdk]
2.1.202 [C:\Program Files\dotnet\sdk]
2.1.300 [C:\Program Files\dotnet\sdk]
2.1.400 [C:\Program Files\dotnet\sdk]
2.1.402 [C:\Program Files\dotnet\sdk]
2.1.403 [C:\Program Files\dotnet\sdk]
2.1.500 [C:\Program Files\dotnet\sdk]
C:\Users>dotnet --list-runtimes
Microsoft.AspNetCore.All 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.All 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.All]
Microsoft.AspNetCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.AspNetCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.AspNetCore.App]
Microsoft.NETCore.App 1.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.11 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.12 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.0.13 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.8 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 1.1.10 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.3 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.7 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.0.9 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.0 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.2 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.4 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.5 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Microsoft.NETCore.App 2.1.6 [C:\Program Files\dotnet\shared\Microsoft.NETCore.App]
Both computers, my PC and the grid node, are on Visual Studio 15.9.
I copied the user .nuget folder over to the grid node, however, that did not have any noticeable effect.
Remco NCrunch Developer
#12823
15 Nov 2018 23:04 UTC
Thanks for these extra details. We're doing some testing here to see if we can find a way to reproduce this issue. I'll let you know as soon as we have more information.
Remco NCrunch Developer
#12826
18 Nov 2018 22:52 UTC
I think I may have managed to pin this down. Would you like to try the build below to see if this solves the problem for you? Note that you'll need to update both the clients and grid nodes.
NCrunch_Console_3.23.0.4.msi
NCrunch_Console_3.23.0.4.zip
NCrunch_GridNodeServer_3.23.0.4.msi
NCrunch_GridNodeServer_3.23.0.4.zip
NCrunch_LicenseServer_3.23.0.4.zip
NCrunch_VS2008_3.23.0.4.msi
NCrunch_VS2010_3.23.0.4.msi
NCrunch_VS2010_3.23.0.4.zip
NCrunch_VS2012_3.23.0.4.msi
NCrunch_VS2012_3.23.0.4.zip
NCrunch_VS2013_3.23.0.4.msi
NCrunch_VS2013_3.23.0.4.zip
NCrunch_VS2015_3.23.0.4.msi
NCrunch_VS2015_3.23.0.4.msi.7z
NCrunch_VS2015_3.23.0.4.zip
NCrunch_VS2017_3.23.0.4.msi
NCrunch_VS2017_3.23.0.4.msi.7z
NCrunch_VS2017_3.23.0.4.zip
NCrunch_Console_3.23.0.4.msi
NCrunch_Console_3.23.0.4.zip
NCrunch_GridNodeServer_3.23.0.4.msi
NCrunch_GridNodeServer_3.23.0.4.zip
NCrunch_LicenseServer_3.23.0.4.zip
NCrunch_VS2008_3.23.0.4.msi
NCrunch_VS2010_3.23.0.4.msi
NCrunch_VS2010_3.23.0.4.zip
NCrunch_VS2012_3.23.0.4.msi
NCrunch_VS2012_3.23.0.4.zip
NCrunch_VS2013_3.23.0.4.msi
NCrunch_VS2013_3.23.0.4.zip
NCrunch_VS2015_3.23.0.4.msi
NCrunch_VS2015_3.23.0.4.msi.7z
NCrunch_VS2015_3.23.0.4.zip
NCrunch_VS2017_3.23.0.4.msi
NCrunch_VS2017_3.23.0.4.msi.7z
NCrunch_VS2017_3.23.0.4.zip
Hi Remco,
Thanks a lot for the updated build! Almost all the errors are fixed. The .net core projects are working correctly now. Additionally, all the .net Framework projects except for 1 is working correctly.
I did upgrade both grid nodes and they are both working identically--all projects but one are now working.
On the project that is experiencing errors, it is still related to System.Runtime. There are two fuslogvw outputs that appear relevant which I linked to below as well as the output from the ncrunch test window.
Fuslogvw log #1: https://gist.githubusercontent.com/rog1039/05e832a3428e83f5dd056787cd57b928/raw/3feb3328970206d6f1bbebe4c75ed7a536de2d82/fuslogvw-log1.txt
Fuslogvw log #2: https://gist.githubusercontent.com/rog1039/c7680954f7406d4374a792c76f49d9fa/raw/12a234766f101a7bb903c0e46a9d2bbbb1ff0184/fuslogvw-log2.txt
NCrunch output: https://gist.githubusercontent.com/rog1039/474d8ec5b9241c7c0ec4c7b3d0a6de88/raw/61cf6eabda3e59c5564025e60d9aff4749dc50c8/ncrunchoutput.txt
Thanks for all your help,
Paul
Thanks a lot for the updated build! Almost all the errors are fixed. The .net core projects are working correctly now. Additionally, all the .net Framework projects except for 1 is working correctly.
I did upgrade both grid nodes and they are both working identically--all projects but one are now working.
On the project that is experiencing errors, it is still related to System.Runtime. There are two fuslogvw outputs that appear relevant which I linked to below as well as the output from the ncrunch test window.
Fuslogvw log #1: https://gist.githubusercontent.com/rog1039/05e832a3428e83f5dd056787cd57b928/raw/3feb3328970206d6f1bbebe4c75ed7a536de2d82/fuslogvw-log1.txt
Fuslogvw log #2: https://gist.githubusercontent.com/rog1039/c7680954f7406d4374a792c76f49d9fa/raw/12a234766f101a7bb903c0e46a9d2bbbb1ff0184/fuslogvw-log2.txt
NCrunch output: https://gist.githubusercontent.com/rog1039/474d8ec5b9241c7c0ec4c7b3d0a6de88/raw/61cf6eabda3e59c5564025e60d9aff4749dc50c8/ncrunchoutput.txt
Thanks for all your help,
Paul
OK, so I have been able to get the last project to work. I noticed that this last project would not work correctly in Resharper's unit test runner either.
So the library under test would build without problem, but this test project would not. I noticed the library had a nuget package 'System.Reactive' whereas the test project did not have this package. I added System.Reactive 4.1.2 to the test project and the test project then started to work just fine in both Resharper and NCrunch. I do not know why this made a difference but it did fix the issue.
So, I think everything is working great now. Thanks for looking into this issue with me.
Paul
So the library under test would build without problem, but this test project would not. I noticed the library had a nuget package 'System.Reactive' whereas the test project did not have this package. I added System.Reactive 4.1.2 to the test project and the test project then started to work just fine in both Resharper and NCrunch. I do not know why this made a difference but it did fix the issue.
So, I think everything is working great now. Thanks for looking into this issue with me.
Paul
It might be worth noting that VS 15.9 and onwards fixes a problem with binding redirects and System.Runtime, so it may be worth upgrading to that (if possible) and removing the binding redirect from your config altogether
Hi Matthew,
Thanks for the info. I did upgrade my machine to 15.9 and tried removed the binding redirect, however, that did not have any effect on the remote NCrunch grid node. Resharper and VS were able to run the tests without issue, and the NCrunch on my local machine could in fact run the tests as well.
I'm not sure what change Remco made to NCrunch on the build he posted, but that did take care of the issue fortunately.
I'm really looking forward to the future where (hopefully!) we can switch over to .net Core 3.0 with WPF support. That would be great since it can be a pain trying to work with both .net Framework and .net Core projects in the same solution.
Thanks for the info. I did upgrade my machine to 15.9 and tried removed the binding redirect, however, that did not have any effect on the remote NCrunch grid node. Resharper and VS were able to run the tests without issue, and the NCrunch on my local machine could in fact run the tests as well.
I'm not sure what change Remco made to NCrunch on the build he posted, but that did take care of the issue fortunately.
I'm really looking forward to the future where (hopefully!) we can switch over to .net Core 3.0 with WPF support. That would be great since it can be a pain trying to work with both .net Framework and .net Core projects in the same solution.
Remco NCrunch Developer
#12834
20 Nov 2018 01:14 UTC
I can confirm that in this case, the problem and fix for the problem were both in the NCrunch codebase.
A recent VS release (I think it was 15.7?) introduced a new build-time redirection system to try and retroactively add support for .NET Standard 2.0 in earlier versions of the .NET Framework. This change had a pretty serious effect on NCrunch's assembly resolution system but only under very targeted circumstances. We implemented a fix for this in NCrunch v3.22, but the fix was never tested properly with the distributed processing, so a defect slipped through it. The build above resolves this problem.
A recent VS release (I think it was 15.7?) introduced a new build-time redirection system to try and retroactively add support for .NET Standard 2.0 in earlier versions of the .NET Framework. This change had a pretty serious effect on NCrunch's assembly resolution system but only under very targeted circumstances. We implemented a fix for this in NCrunch v3.22, but the fix was never tested properly with the distributed processing, so a defect slipped through it. The build above resolves this problem.
Post a reply
Log in to reply.