Hi, thanks for posting!

I've also seen this happen and have had others report it. It's a mystery bug that appears randomly then disappears just as randomly ... I've been trying to track it down but the inconsistency makes reproduction difficult.

My suspicion is that it's happening when NCrunch is first loading the projects (i.e. after it is first enabled), though this is just a theory and I have nothing to back it up.

If you can share any information about the usage pattern causing this, that would be awesome.


Thanks!

Remco

I've since done more looking into this issue. It's been proving extremely difficult to track down, because it occurs in very few environments and when it does occur, it is always intermittent.

The issue itself seems to be tied to overlapping builds between Visual Studio and NCrunch. Somehow there is state being shared between the builds, and they interfere with each other. v1.40 of NCrunch introduced a feature that deliberately pauses all background building while Visual Studio runs a foreground build, with the intention of reducing (or eliminating) the possibility of these overlaps.

Unfortunately, the overlapping logic is within MSBuild itself, rather than the NCrunch code, which adds to the ambiguity around the issue and makes it that much harder to understand.

Something that may solve the problem in your situation is to look for derived files from your build that trigger NCrunch to start its own build. NCrunch will track all files that are included in projects within your solution, and files that have been included using the 'Additional files to include' configuration settings. If any of these files are changed on disk, NCrunch will kick off a background build of projects that include the changed files. If you have a file that is included by NCrunch and is somehow being changed as a result of your foreground build, then this can cause builds between NCrunch and Visual Studio to overlap. You should be able to spot when this issue exists, as a build of your solution in Visual Studio will visibly trigger NCrunch to start a build.

Hi Gareth -

Unfortunately I've still had no luck in reproducing this issue consistently, but I can give you some advice on how to find out which project is making use of the contents in the references folder.

There are a couple of quick ways you can do this -

1. Examine the NCrunch workspace built for each project in your solution. This workspace will contain all the files NCrunch has identified as being referenced by the project, deposited relative to the project file. If you notice DLLs stored in the workspace (from the references directory), that means that the project file is referencing those files. Check to make sure you don't have an 'Additional files to include' setting that is too broad, and that they aren't included specifically in the project file itself.

2. If you set the Logging Verbosity to 'Detailed', then reset the engine, you should be able to see the trace logs output by NCrunch when it's building the workspaces for each of your projects - if you look under the Processing Queue Window. Simply click on any of the build projects and you'll see a dump from the workspace builder in the bottom pane. This can help with narrowing down which projects are referencing these files

3. It may also be worth temporarily removing the files and performing a clean of your solution, then executing a build just to be sure none of the projects is referencing anything it shouldn't be

Hi James -

I'm not yet sure whether the build problems you've described above are caused by the same problem being reported on this thread. Previously when people have reported this problem, it occurs only very infrequently and doesn't regularly cause build issues to appear. I'm wondering if you might be able to help with narrowing down the problem by answering the following questions -

How often do the build errors appear? Does it occur on most/all of your builds, or just occasionally throughout the day?

When the build error appears, does it require a reset before it goes away, or can you just continue working in your solution with any subsequent rebuild working correctly?

When you experience the build problem, is this in the Visual Studio build or the NCrunch build?

Are you running any integration tests that contain absolute file references back to your original solution path?

Are you running any integration tests that spawn child processes of any kind, or create locks on files that may not be released?

When you build your solution using Visual Studio, does this cause NCrunch to suddenly trigger and start running builds and tests?


Sorry for the barrage. There's a lot going on here, so it may take a little back and forward before we find a key piece of information to help analyse this problem. The documentation also has some great information about workspaces that would be worth reading if you're still becoming familiar with this concept.


Cheers,

Remco

Hi,

Thanks for sharing the details above. Based on the information you've also provided in the other thread (re: conditional DLL/project reference issue) and the last answer you gave (building your solution causes NCrunch to trigger), I'm fairly certain that this problem is caused by your inclusion of build output files in your NCrunch workspace.

The later versions of NCrunch (1.43 and above) will generally give a warning if they detect this, although there are many complex cases that can create it and the warnings won't always show.

When your foreground build (in visual studio) changes files that are being watched by NCrunch (which includes assembly references, project files, and files included using the 'Additional files to include' configuration setting), this can then cause overlapping builds between NCrunch and Visual Studio. Because NCrunch's build is designed to be isolated from your foreground solution (using the workspaces), in theory these builds should not interfere with each other. However, I have observed (and been unable to recreate) situations where these builds DO actually somehow share state and can cause crazy file copying to happen.

It's rare, but when NCrunch is running thousands of builds throughout the day, the probability doesn't need to be very high for the problem to occur at least once.

If NCrunch is giving you warnings about references to files output from the build process - I recommend taking these warnings seriously. They can provide useful information about implicit behaviour within your build process. If you have no warnings showing, it is possible to use NCrunch's trace log to identify which file change is causing the engine to trigger in the following way:

1. Enable the 'log to output window' configuration setting in your global NCrunch config
2. Set the log verbosity to 'Medium'
3. Set the engine mode to run tests manually (to avoid noise)
4. Open the VS output window and choose 'NCrunch Diagnostic Output' in the dropdown list
5. Build one of the projects that you believe is causing NCrunch to trigger
6. Watch for when NCrunch triggers, then look through the log for a 'ComponentMemberUpdatedEvent', or a 'ComponentAssemblyReferenceUpdatedEvent'. These events identify files that have been updated and will therefore trigger the engine to run a build.

It is highly desirable to be in a situation where you cannot cause NCrunch to trigger a build by executing a build in Visual Studio. While there are many solutions that work fine with NCrunch even though their builds trigger in this way, it's always the first thing that should be addressed when experiencing random problems like those described above.