Incorrectly reporting MSpec test results and icons stay black

Hi Remco! Do you might have any idea why in 1.34.0.0b, NCrunch incorrectly reports MSpec tests as passing (green in NCrunch Tests window)? I have put a big throw exception (confirmed failing in ReSharper's runner) and click Resynchronize... and it comes back as green in NCrunch.

Additionally the icons stay black in the code editor windows if the tests are passing or failing. Using MSpec 0.4.112.0 and VS 2010.

I saw this post and tried turning up the log to Medium but can't see any errors. What should I do?

You got in a few minutes before me :)

I'd just upgraded to 1.34b and noticed the same issue - all the coverage dots were showing black, as 'no tests are covering this code'. I checked the NCrunch Tests panel, and noticed that many of my specifications were showing as failing, however the individual assertions within that spec all passed. The error in the test panel was saying 'Assembly not found' for the specification constructor. I opened up the Configuration panel and turned 'CopyReferencedAssembliesToWorkspace' to True, and that seemed to fix the issue - now in the Tests panel, all the specifications were showing as passing along with the individual assertions. However, the coverage dots still stayed black.

At this point I purposefully introduced an error in the code to cause failing tests. NCrunch happily reported all tests passing. Yet when I ran the MSpec tests via ReSharper, it reported numerous test failures.

I've rolled back to 1.33b and this functionality is all working as expected - tests that should be failing, are. I'm seeing proper coverage dots now. And I'm also able to turn off 'CopyReferencedAssembliesToWorkspace' and still have full functionality.

I'm pretty busy at the moment but I'll try and replicate this later and submit a bug report.

I am using MSpec 0.4.21.0

Hi guys - thanks for posting.

I think you're both experiencing different issues here.

Simon, your issue sounds like an assembly resolution problem - perhaps related to how your project is referencing the MSpec assembly.

Richard, I think your issue may be related to the version of MSpec you're using.

Would you guys be able to push through bug reports? I'd really like to know what the engine is trying to do :)

Sure, will do send one in on Monday when I'm back at work.

I'm ashamed to say it's the first time I've actually used NCrunch (why!!) but so impressed so far. As a user who doesn't know much about anything, I found that wizard really seamless to get up and running quickly. You just need to get your userbase well-established before Jetbrains release something!

Hi Remco,

I've submitted a bug report. Plus a little more detail here just in case.

After having everything in a working state with 1.33b (all unignored tests running and passing, coverage dots being shown correctly), I installed 1.34b over the top (VS 2010).

On re-opening visual studio (I closed it to upgrade NCrunch):

- The Code Coverage dots are all black (no tests covering this line)
- The Risk/Progress bar shows OK/Green (all passing)
- The NCrunch Tests window shows numerous failures - but only for the 'main' specification. By that I mean something like the following:
--- MyProject.Tests.MyClassSpecs.Calling_MyMethod (failed)
--- --- should_return_5 (pass)
--- --- should_raise_an_event (pass)

The error against the spec is is a 'Could not load file or assembly..' error for the assembly containing the subject under test (in my example it would be MyProject.dll - the thing that contains MyClass)

Hi Simon -

I've had a go at fixing this issue in the 1.35b build that went out this morning. Are you interested in giving it a spin and letting me know if this works for you?

Cheers,

Remco

Hi Remco,

I'm running 1.35.0.16b and my problem seems to be resolved. The coverage dots are showing up correctly now (and with the correct status too). My tests are running without exceptions, and the various windows are all in sync. Overall I'm happy :)

Thanks!

Simon