Build/Test Issues

Inconsistent build across Grid Nodes

Started by avishnyakov on 6,502 views

It seems that with the recent updates NCrunch gives the following message preventing distributed tests execution:

>>NCrunch: The build results for this project are not consistent across all computers within the grid.
** LOTS OF DETAILS HERE*

It was not like that before, so it is possible to disable that feature somewhere in the setttings?
avishnyakov wrote:
It seems that with the recent updates NCrunch gives the following message preventing distributed tests execution:

>>NCrunch: The build results for this project are not consistent across all computers within the grid.
** LOTS OF DETAILS HERE*

It was not like that before, so it is possible to disable that feature somewhere in the setttings?


The reporting of this isn't a new feature - it's actually been in place since the first beta releases of v2, when the grid processing was first introduced. It's possible that it has started appearing due to a change in your environment or build system.

Generally when the engine reports this, it means that at least one of the nodes in your grid was unable to build a project where other nodes were able to. When NCrunch detects this situation, it tries to provide you with as much information about the build process as possible to better enable you to solve the problem. The best thing to do is take a look at the build result and see how it is different. If the problem appears intermittently, it may well be that something in your build system is intermittently failing.
Thanks Remco, seems to be the case.

Used to have only VS2013 + NCrunch and grid, all was fine.
Got VS2015 installed, here we go, started having all that fun.

The output says only the folllwing:
..\..\..\..\..\Program Files (x86)\MSBuild\12.0\bin\amd64\Microsoft.Common.CurrentVersion.targets (923, 7): The target "AssignLinkMetadata" does not exist in the project.

Not sure why and what, trying to solve that. Found only the following issue at MSBuild github:
https://github.com/Microsoft/msbuild/issues/544

Guess, some changes with the VS2015.
VS2015 introduced a huge range of changes to the build system. This means that if you have VS2015 on your client machine (running MSBuild 14.0, which is standard for VS2015), then you have grid nodes running MSBuild 12.0 (the standard for VS2013), there is a high chance that you'll get inconsistent behaviour.

Grid nodes will generally try to use the version of MSBuild identical to that used on the client machine. Where this isn't available, they will fall back to closest version of MSBuild available.

The AssignLinkMetadata target is a new build target that was introduced with VS2015. It is invoked by MSBuild indirectly through the 'PrepareForBuildDependsOn' property, and is responsible for handling some edge cases around embedded resource links.

Prior to NCrunch v2.23, NCrunch would actually hard-code the value of this property to a fixed list of build targets, which excluded the PrepareForBuildDependsOn target. Under VS2013, this was safe, as all the VS2013 targets were included in the hard coded list. Unfortunately, this approach started introducing problems in VS2015 as the AssignLinkMetadata target wasn't being called and some projects were failing to build correctly. v2.23 introduced a fix in which the PrepareForBuildDependsOn is instead queried from MSBuild when the project is loaded (on the client machine). The value of this property is then fed back into MSBuild when the project is built (along with another target used by NCrunch to override some areas of the build process).

The result is that all VS2015 builds are now running correctly under v2.23, but it seems that it is now not possible to build a VS2015 project on a grid node using the build system from VS2013.

In theory, it should be possible to reverse this change in your projects by overriding the PrepareForBuildDependsOn property and forcing this to its VS2013 equivalent. For example:

<PropertyGroup>
<PrepareForBuildDependsOn>GetFrameworkPaths;GetReferenceAssemblyPaths</PrepareForBuildDependsOn>
</PropertyGroup>

This will skip the AssignLinkMetadata target and embedded resources in the compiled assembly may not be namespaced correctly, but it may allow the grid nodes to continue to work with the projects under VS2013. Of course, the better solution would be to upgrade the grid nodes ... In the long term, poking holes in the build system generally creates new problems over time.
It is not possible even with VS2013 / VS 2015 build tools installed.
Gave up, installed VS2013 on every grid node, seems to fix so it works well for the time being.

Pity, would say soved.

Post a reply

Log in to reply.