Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Improving grid node IL reporting.
Options
Previous Topic Next Topic
Cjdawson
#1 Posted : Saturday, April 25, 2026 8:34:24 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/27/2015(UTC)
Posts: 6
Location: United Kingdom

Thanks: 1 times
Hi, I've got 3 machines setup with ncrunch, my local machine that I write code on, and two VM's that I use as build agents for TeamCity. Since most of the time those are sitting idle, I figured I'd set them up as grid nodes so they can help with running tests.
This has actually be great for reasons that I did not expect. Some of the Builds on the Grid nodes failed, and I was able to figure out that there was some minor config issues in the projects that I needed to fix.

The other thing that I'm getting is

NCrunch has detected that compiled assemblie
Quote:
s produced by different machines in the grid are not identical at IL-level. This is usually caused by grid machines using different compiler versions.

Microsoft very regularly release updates to the .NET compilers in updates to Visual Studio, and possibly with hot fixes. Ensure all machines in the grid have the same updates installed.

IL differences between assemblies can trigger NCrunch's IL-level impact detection features (enabled by default) as the assemblies have physical differences that the engine detects as code changes. Compiler version inconsistencies also have the potential to produce erratic test and runtime behaviour in extreme situations.

Changing your 'Impact Detection Mode' configuration setting to 'Watch Text' will set NCrunch to avoid using IL comparison for impact detection, reducing the severity of this issue. Note that this will also result in significant loss of impact detection accuracy.

IL contents produced the following hashes across the grid:

(local): 12869121715097317820
WIN11PROBUILD02: 16115415983637522274
WIN11PROBUILD01: 16115415983637522274


I like that the two grid nodes are giving the same result, but my local machine is doing something else. All three are running NCrunch 5.21 (VS extension on local, and Grid on the other two)
All three machines have VS Community 18.5.1 installed.

But I cannot understand why the hash for my local machine is different to the two grid nodes.


And here's why I've put this in feature suggestions rather than general help......

I would love for NCrunch to report all the versions of everything that was used in the build process, so that I can figure out what's different between the machines. It would be even better if NCrunch would be able to tell me what the difference is, but I'm not expecting that. Just some more of a clue as to what is causing the difference.
Back to top
Remco
#2 Posted : Sunday, April 26, 2026 1:56:38 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,540

Thanks: 1026 times
Was thanked: 1374 time(s) in 1275 post(s)
Hi, thanks for posting.

Yes, I hate this error. I hate that it exists. I hate that it NEEDS to exist ...

And I accept that it's inadequate.

In recent years, it seems that even the smallest update to the platform causes differences in the IL. I find this a bit concerning, since you wouldn't expect that platform changes would affect compiled code to such an extent. It has me wondering if we're somehow including a platform version or signature in the hash itself.

Now here is the kicker: It's disturbingly hard to accurately provide detailed versions on everything used in the build system. Mostly because there's a lot of components to it, and it does tend to quietly change in structure. I think though, it should at least be possible to obtain the major versions of the systems involved (like the VS platform version and dotnet SDK). I'll take a look into this.

Meanwhile, something that might help is to turn on your NCrunch build logging and examine the logs inside the Processing Queue Window for the build tasks. You'll often find version information splattered throughout the logs. At the very least, you can run the log through a diff tool to see why it's different between machines.

My guess would be that it's the versions of the dotnet SDK being different between machines. Check under "Program Files\dotnet\sdk" to see if the directories line up between your machines. Note that it's possible to force NCrunch to use a specific version of these SDKs if there's a common one you want to target ('Build Sdk' config setting). If you don't force a version, MSBuild will just grab the latest one (which might be slightly different). Whenever I get this error, it's usually the dotnet SDKs that are tripping me up.
Back to top
 |  Edit by user
Cjdawson
#3 Posted : Sunday, April 26, 2026 11:38:49 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/27/2015(UTC)
Posts: 6
Location: United Kingdom

Thanks: 1 times
Quote:
Meanwhile, something that might help is to turn on your NCrunch build logging and examine the logs inside the Processing Queue Window for the build tasks. You'll often find version information splattered throughout the logs. At the very least, you can run the log through a diff tool to see why it's different between machines.


Brilliant, that's the clue that I needed to solve one of the issues easily.
Turned out that I'd chosen different install settings when I did the initial install of VS. It's important that they're all exactly the same.

That solve one project, I've still got two that give different IL outputs.

Whilst it might be a real pain to figure out why the IL outputs are different, I think it's actually a fantastic feature. The two grid nodes that I'm using are also my TeamCity Build agents. So knowing that the output from all three installs of VS is the same is actually a very powerful tool, and in this case, knowing where there are differences gives me the opportunity to actually mitigate and differences before they manifest into that extremely nasty un-recreatable bug that plagues systems long term.

The clue that I needed was that it's possible to get access to the build logs, might be worth adding some text to the IL difference message saying how to get the build logs, as this will provide 99% of people the clue that's needed to figure out and resolve what's going on, with minimal change from you guys point of view - especially as I'm sure that you've got much more important things to be getting on with.

Regards

Colin.
Back to top
Remco
#4 Posted : Monday, April 27, 2026 4:39:27 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,540

Thanks: 1026 times
Was thanked: 1374 time(s) in 1275 post(s)
I'm glad that's helped you make some progress in this. The only thing I could suggest as a next step would be to use an IL tool of some kind to produce a full dump of the assemblies produced on each of your machines, then run these through a diff tool. This might also give some clarity as to whether or not NCrunch's algorithm is working as expected here. Note that NCrunch does also do some work with hashes derived from referenced assemblies, so this won't be exactly the same as what it does internally.
Back to top
Users browsing this topic
Guest (2)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.039 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download