Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Incorrectly reporting MSpec test results and icons stay black
Options
Previous Topic Next Topic
rdingwall
#1 Posted : Friday, October 7, 2011 1:21:43 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 10/7/2011(UTC)
Posts: 2
Location: London, UK

Was thanked: 1 time(s) in 1 post(s)
Hi Remco! Do you might have any idea why in 1.34.0.0b, NCrunch incorrectly reports MSpec tests as passing (green in NCrunch Tests window)? I have put a big throw exception (confirmed failing in ReSharper's runner) and click Resynchronize... and it comes back as green in NCrunch.

Additionally the icons stay black in the code editor windows if the tests are passing or failing. Using MSpec 0.4.112.0 and VS 2010.

I saw this post and tried turning up the log to Medium but can't see any errors. What should I do?
Back to top
WWW
simon
#2 Posted : Friday, October 7, 2011 1:44:57 PM(UTC)
Rank: Member

Groups: Registered
Joined: 6/17/2011(UTC)
Posts: 10

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
You got in a few minutes before me :)

I'd just upgraded to 1.34b and noticed the same issue - all the coverage dots were showing black, as 'no tests are covering this code'. I checked the NCrunch Tests panel, and noticed that many of my specifications were showing as failing, however the individual assertions within that spec all passed. The error in the test panel was saying 'Assembly not found' for the specification constructor. I opened up the Configuration panel and turned 'CopyReferencedAssembliesToWorkspace' to True, and that seemed to fix the issue - now in the Tests panel, all the specifications were showing as passing along with the individual assertions. However, the coverage dots still stayed black.

At this point I purposefully introduced an error in the code to cause failing tests. NCrunch happily reported all tests passing. Yet when I ran the MSpec tests via ReSharper, it reported numerous test failures.

I've rolled back to 1.33b and this functionality is all working as expected - tests that should be failing, are. I'm seeing proper coverage dots now. And I'm also able to turn off 'CopyReferencedAssembliesToWorkspace' and still have full functionality.

I'm pretty busy at the moment but I'll try and replicate this later and submit a bug report.

I am using MSpec 0.4.21.0
Back to top
Remco
#3 Posted : Saturday, October 8, 2011 9:57:17 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,974

Thanks: 929 times
Was thanked: 1256 time(s) in 1169 post(s)
Hi guys - thanks for posting.

I think you're both experiencing different issues here.

Simon, your issue sounds like an assembly resolution problem - perhaps related to how your project is referencing the MSpec assembly.

Richard, I think your issue may be related to the version of MSpec you're using.

Would you guys be able to push through bug reports? I'd really like to know what the engine is trying to do :)
Back to top
rdingwall
#4 Posted : Saturday, October 8, 2011 10:05:06 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 10/7/2011(UTC)
Posts: 2
Location: London, UK

Was thanked: 1 time(s) in 1 post(s)
Sure, will do send one in on Monday when I'm back at work.

I'm ashamed to say it's the first time I've actually used NCrunch (why!!) but so impressed so far. As a user who doesn't know much about anything, I found that wizard really seamless to get up and running quickly. You just need to get your userbase well-established before Jetbrains release something!
Back to top
WWW
1 user thanked rdingwall for this useful post.
Remco on 10/8/2011(UTC)
simon
#5 Posted : Monday, October 10, 2011 4:56:28 PM(UTC)
Rank: Member

Groups: Registered
Joined: 6/17/2011(UTC)
Posts: 10

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
Hi Remco,

I've submitted a bug report. Plus a little more detail here just in case.

After having everything in a working state with 1.33b (all unignored tests running and passing, coverage dots being shown correctly), I installed 1.34b over the top (VS 2010).

On re-opening visual studio (I closed it to upgrade NCrunch):

- The Code Coverage dots are all black (no tests covering this line)
- The Risk/Progress bar shows OK/Green (all passing)
- The NCrunch Tests window shows numerous failures - but only for the 'main' specification. By that I mean something like the following:
--- MyProject.Tests.MyClassSpecs.Calling_MyMethod (failed)
--- --- should_return_5 (pass)
--- --- should_raise_an_event (pass)

The error against the spec is is a 'Could not load file or assembly..' error for the assembly containing the subject under test (in my example it would be MyProject.dll - the thing that contains MyClass)
Back to top
1 user thanked simon for this useful post.
Remco on 10/10/2011(UTC)
Remco
#6 Posted : Tuesday, October 18, 2011 6:27:51 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,974

Thanks: 929 times
Was thanked: 1256 time(s) in 1169 post(s)
Hi Simon -

I've had a go at fixing this issue in the 1.35b build that went out this morning. Are you interested in giving it a spin and letting me know if this works for you?

Cheers,

Remco
Back to top
simon
#7 Posted : Wednesday, October 19, 2011 12:18:03 PM(UTC)
Rank: Member

Groups: Registered
Joined: 6/17/2011(UTC)
Posts: 10

Thanks: 1 times
Was thanked: 4 time(s) in 4 post(s)
Hi Remco,

I'm running 1.35.0.16b and my problem seems to be resolved. The coverage dots are showing up correctly now (and with the correct status too). My tests are running without exceptions, and the various windows are all in sync. Overall I'm happy :)

Thanks!

Simon
Back to top
1 user thanked simon for this useful post.
Remco on 10/20/2011(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.050 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download