MissingMethodException after moving from VS 2012 to VS 2015

After a somewhat frustrating conversion from VS 2012 to VS 2015 (weird bug with order of interfaces inherited by other interface) I downloaded NCrunch 2.16.0.13 (sorry, latest version requires me to upgrade, I'll do that, but not yet), I've spent several hours trying to get rid of a MissingMethodException on all my tests.

The first thing I checked was proper config settings, but it took them from the original project unchanged (which ran a version of NCrunch of about a year ago, on VS 2012). Just to be sure I added some implicit project dependencies, in particular for the project containing the missing method.

I restarted, reloaded, made sure that in Visual Studio every build configuration worked (x86 Release, x86 Debug, x64 Release, x64 Debug). I resolved any remaining errors I might have had.

Again, just to be sure, I changed the config of NCrunch to use explicit build configurations and for all related projects (15 in this case) set them to x86, Debug. This was the setting used consistently and successfully for the past year.

The tests use NUnit. They are displayed. NCrunch shows no errors, no warnings when building. The error is consistent on the same method, which happens to be the first method called that used a referenced class (project reference). I can debug, but then just hit that point and no other information is available.

This is likely some assembly loading issue (though I do not get an assembly loading error), but I wonder how I can check this. Or how to manually erase all files in the build directory of NCrunch (wherever that is, I don't think there is a menu option like the Clean command in VS).

Is the referenced assembly loaded into the GAC perhaps? GAC resolution takes priority over everything else, and it can result in considerable mess in some situations.

If not, does setting the engine in compatibility mode make any difference?

I found the culprit. It turned out that after conversion to VS 2015, the NUnit test F# project referenced the F# 3 runtime, while other parts referenced the F# 4 runtime. Don't know why it wasn't consistently updated. Strangely, some of the assemblies still target the F# 3 runtime, but since the Unit test project is an EXE, I assume (but can't verify) that it must itself be using the F4 runtime, then it doesn't matter whether some parts reference the same or an older runtime. But the other way around is not possible.

Another issue with the new NCrunch is that for some reason it has extremely many thread or other stalls on tests (many tests run in a few ms, and a few suddenly run 15s or more, but this seems to be random, different tests are slow, and while running the tests, instead of about 100 tests per second, the average is now closer to 5-10 per second, sometimes stalling for a while completely, running the same tests in TeamCity takes for all tests about 9 minutes, against 3 hours (was ~1hr) in NCrunch), but I will report on that separately, it is not related to this issue.

abelb wrote:I found the culprit. It turned out that after conversion to VS 2015, the NUnit test F# project referenced the F# 3 runtime, while other parts referenced the F# 4 runtime. Don't know why it wasn't consistently updated. Strangely, some of the assemblies still target the F# 3 runtime, but since the Unit test project is an EXE, I assume (but can't verify) that it must itself be using the F4 runtime, then it doesn't matter whether some parts reference the same or an older runtime. But the other way around is not possible.

This sounds like it was a 'fun' issue. .NET Assembly resolution is never simple even at the best of times :(

abelb wrote:
Another issue with the new NCrunch is that for some reason it has extremely many thread or other stalls on tests (many tests run in a few ms, and a few suddenly run 15s or more, but this seems to be random, different tests are slow, and while running the tests, instead of about 100 tests per second, the average is now closer to 5-10 per second, sometimes stalling for a while completely, running the same tests in TeamCity takes for all tests about 9 minutes, against 3 hours (was ~1hr) in NCrunch), but I will report on that separately, it is not related to this issue.

The release of v2.17 included a change in the cache file format, so the first time you used this build on a solution, NCrunch would have had no previous test run data to work with. The test run data has a very significant impact on NCrunch's optimisation, as it doesn't know which tests are fast and which are slow. After the engine has spun through and executed all your tests, it should behave normally again.

The release of v2.17 included a change in the cache file format,

It makes sense. They run relatively normal now. And I just realized that I use them with Debug builds, and on the test server I use the Release build, so it is hardly a fair comparison ;).