Cannot debug tests with the 2.17 version

Hi Martin,

You're the first person to have reported this problem specifically in the 2.17 release, although there have been a number of people reporting debugging related problems in VS2015 which don't appear to be caused by NCrunch (though NCrunch does seem to surface them).

Can you share any more details about the problem? Are you receiving an error message of some kind?

Can you confirm 100% that this is only impacting debug sessions, and that running the tests without the debugger (by right clicking and choosing 'Run') works correctly?

If so, does adding a System.Diagnostics.Debugger.Launch(); call into the test allow you to attach a debugger?

Does the problem also appear for you in a simple test project/solution with just one test?

Cokyn wrote:Good morning,

you are right, the same problem is happening during the test run as well so it is not specifically debugging problem.
When I created a brand new project with 1 test only then everything seems to be working :/

Ok - we've now narrowed it down to a solution-specific NUnit3 integration problem then.

The NUnit3 integration is still really new, so some problems are expected. The key thing now will be figuring out how to reproduce the issue. Are there certain tests that are consistently causing the problem? Are you using any new features of NUnit3 such as ParallelizableAttribute?

Thanks Martin. Is there any chance you could share a project or solution with me that can demonstrate these issues? You can submit it via the contact form if you like. If I can get a project that reproduces these issues, there's a good chance I can provide you with a fixed build.

Sorry, is a sample project an option? There must be some feature of NUnit that you're using in your production solution that is causing this failure. Perhaps you can narrow it down to a small purpose-built code sample you can share?

Thanks for sharing this. I've done some basic testing with the NUnit features you're using here (async, descriptions, SetUp, TearDown, etc). I haven't been able to recreate any problems, so I think we need to take a deeper look at what may be happening in your environment.

My first thought was that the problems you're experiencing may be caused by unexpected asynchronous behaviour in the NUnit environment. This was the source of many of the earlier problems in NUnit3 integration, as NUnit3 is heavily multi-threaded and has a number of features that prevent tests from running synchronously. For tests to work in NCrunch's environment, all async behaviour must be disabled so that the coverage tracking and test boundaries can stay consistent.

Something that would be useful is if we could grab a log from a faulty NCrunch test run on this project. If you set your 'Log Verbosity' (global NCrunch configuration setting) to 'Detailed', and inspect the test tasks inside the NCrunch Processing Queue, you'll notice that they emit trace information about the test run. If you can share some of this data with me (either by posting here or submitting through the contact form on this site), there may be some useful data we can use.

I'd also suggest taking a look at anything in your project that might influence NUnit's behaviour outside of the code you've shown here, such as settings inside your app.config that are targeted towards NUnit, or assembly-level attributes. Of particular interest is anything that might cause NUnit to work asynchronously.

Thanks for sharing this log. I couldn't find anything in the actual test execution itself that described abnormal behaviour. Could you confirm which of the tests in this run were giving you an error after the end of the run?

Something worth investigating is this:

Process 26592: C:\Users\martinco\AppData\Local\NCrunch\16092\17\_ncrunchreferences\SpinSport.Services.Framework.ClientLib.DLL was expected to be loaded from C:\Users\martinco\AppData\Local\NCrunch\16092\6\SpinSport.Services.Framework\SpinSport.Services.Framework.ClientLib\bin\Debug\SpinSport.Services.Framework.ClientLib.dll

Is the SpinSport.Services.Framework.ClientLib project a part of the solution you're working in? If so, something is not right here. NCrunch has copied this to a static reference location inside the workspace, which is something it will only do for projects referenced by their DLL instead of their project.

Has the engine given you any warnings worthy of note? Particularly around 'lost' project references?

Cokyn wrote:Update:
I moved the whole solution to the brand new one. everything has been working and then I tried to run parallel tests with this attribute [Parallelizable(ParallelScope.Fixtures)]
from console and I keep receiving the same error as before:

"This test was not executed during a planned execution run. Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures."

I can still run the same test which are failing in Ncrunch in classic Nunit runner or nunit console.
I haven't tried to uninstall/install Ncrunch again since It did not help me last time. It is very frustrating.

Ok - the good news here is that this is an issue I can give you a fix straight away for.

NCrunch 2.17 had an issue where it didn't disable NUnit3's parallelization features. This would mess up any test runs with parallelizable attribute applied.

2.18 has a fix included for this - https://www.ncrunch.net/download.