Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Coverage shows black when tests cover the code
Options
Previous Topic Next Topic
samholder
#1 Posted : Friday, May 11, 2012 8:43:28 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/11/2012(UTC)
Posts: 94

Thanks: 28 times
Was thanked: 12 time(s) in 12 post(s)
I have just started using NCrunch and have some classes where the markers show there is no coverage, but I have tests which NCrunch is running successfully.

When I edit the class (just add a space) the markers go green for a short while, then return to black. Why might this be?
Back to top
Remco
#2 Posted : Saturday, May 12, 2012 12:25:46 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,143

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi Sam -

Can you share any more details about the structure of these tests? Note that NCrunch's code coverage is very physical in its nature - if you have static constructors or load-on-demand resources they will usually be considered as covered only by the first test that touches them in the test runner application domain.
Back to top
1 user thanked Remco for this useful post.
samholder on 5/15/2012(UTC)
samholder
#3 Posted : Tuesday, May 15, 2012 1:56:19 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/11/2012(UTC)
Posts: 94

Thanks: 28 times
Was thanked: 12 time(s) in 12 post(s)
Hi Remco,

thanks for the reply sorry I was late to respond. The solution has 72 projects in it some of which are silverlight projects. The project in question The class I looked at (though I think there are others with the same issue) is in a project that is right at the root (ie it doesn't depend on any other projects in the solution) and has several references which are all dlls, and the test project has the project it is testing as a project reference, not a dll. The class in question and the tests are pretty basic, it doesn't have any static constructors etc. In fact it only has a constructor, 2 auto properties and overrides of equals and GetHashCode.

As I said the strange thing is that if I modify the file, then everything goes green for a second, then back to black again.

Sam
Back to top
Remco
#4 Posted : Tuesday, May 15, 2012 9:57:06 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,143

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks - I'm wondering if you wouldn't mind answering a few more questions to help me narrow down where it's going wrong.

- Do you have anywhere in your solution multiple tests under the same physical name?
- How many tests are covering the class in question? Are they all in the same test assembly?
- Have you experienced this problem with any other classes or code within your solution?
- When the markers in the class turn to black, do you still see green markers for the test that should be covering it over other parts of your code?
- Would you be able to submit a bug report straight after you see this happen? It sounds like it may be a mapping exception of some kind.


Thanks a lot!

Remco
Back to top
samholder
#5 Posted : Tuesday, May 22, 2012 11:37:55 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/11/2012(UTC)
Posts: 94

Thanks: 28 times
Was thanked: 12 time(s) in 12 post(s)
Remco, sorry not to have got back sooner, seem to be snowed under at the moment. I wanted to try and create a slimmed down solution which showed the problem but have not had the time.

We have a .NET project (lets call it A) and some tests for that (A.Tests). We also have a silverlight project (SilverlightA) which contains shared files with A, so it doesn't have any fioles of its own we just simply do Add existing Files, browse to the files for A then choose 'Add as Link' rather than Add. so both projects have the same files in them.

Maybe this is causing the issue?

Anyway, when I get time I'll try a simple solution set up like this to see if it shows the same issues, otherwise I'll try and submit a bug report for this.

Hope this helps in the meantime.

Sam
Back to top
Remco
#6 Posted : Tuesday, May 22, 2012 10:59:46 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,143

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi Sam,

There definitely will be an issue with sharing the same source file between two differently tested projects. This kind of setup will cause NCrunch to become confused between the two projects holding the same file, as the open document in the VS IDE is physically pointing to two logically different files.

I'll make a note to see what I can do about this in future, though I don't think it's an easy fix. I hope this situation isn't very common in your solution.


Cheers,

Remco
Back to top
samholder
#7 Posted : Wednesday, May 23, 2012 7:44:33 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/11/2012(UTC)
Posts: 94

Thanks: 28 times
Was thanked: 12 time(s) in 12 post(s)
Thanks Remco,

Unfortunately this setup is common. We currently have 71 projects in our solution, about 15 of them are silverlight builds of another .NET project. The silverlight projects do not have any tests, and are not considered by NCrunch, so I would have thought that as far as NCrunch is concerned there should be only 1 project which is being tested, the .NET one.

Anyway thanks for letting me know your thoughts on the issue. I'll still aim to get you a small test solution which demonstrates the issue, when I get a spare 5 mins, to see if there is the chance of a solution.

And thanks for an excellent, revolutionary tool. Its awesome!

Sam
Back to top
1 user thanked samholder for this useful post.
Remco on 5/23/2012(UTC)
Remco
#8 Posted : Tuesday, June 19, 2012 4:39:18 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,143

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
For anyone interested, 1.40b (just released) includes a fix that should allow the engine to function correctly with source files that are shared between projects. This should hopefully resolve the above issue.
Back to top
myogui
#9 Posted : Tuesday, August 7, 2012 2:55:16 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 8/7/2012(UTC)
Posts: 3

Was thanked: 1 time(s) in 1 post(s)
I'm at version 1.40B and still have the issue. I only have 1 test project and 1 project. I have green markers on my tests but black ones where their should be coverage.

thanks
Back to top
 |  Edit by user
Remco
#10 Posted : Tuesday, August 7, 2012 9:49:31 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,143

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi Myogui,

Thanks for posting! Based on your description I have a feeling that you are the victim of a very different issue.

Can you answer some of the following questions? This will help me to better establish where the issue is:

- Do you have any green/red markers on your non-test project anywhere at all?
- Do you experience this problem also with other solutions?
- How does your test project reference your non-test project? Is this done using a project reference or an assembly reference?


Cheers,

Remco
Back to top
myogui
#11 Posted : Wednesday, August 8, 2012 1:13:22 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 8/7/2012(UTC)
Posts: 3

Was thanked: 1 time(s) in 1 post(s)
Hi!

thanks for the reply.

- Do you have any green/red markers on your non-test project anywhere at all?
Nope

- Do you experience this problem also with other solutions?
Yes

- How does your test project reference your non-test project? Is this done using a project reference or an assembly reference?
Yes, tried with both. First the assembly then project reference. All of the solutions have one non-test project and one test project. All assemblies are GACed, stored on a server and they're versionned (most have mapped versions in the machine.config file)

thanks
Back to top
Remco
#12 Posted : Wednesday, August 8, 2012 9:52:38 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,143

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for the info. The problem you're experiencing will be due to the GAC registration of the assemblies under test. The CLR will always load an assembly from the GAC before touching an assembly on disk, and I've yet to find a clean way around this.

The best way to work around this issue is to make sure that the projects you are testing with NCrunch have a different strong name to their cousins installed in the GAC. Probably you could do this with custom build configuration (perhaps using the $(NCrunch) build environment variable) or by making use of an #IF NCRUNCH directive in the AssemblyInfo.cs file. It's probably easier to change the version number of the assembly using this logic than the key used to sign the assembly.

Something that may also do the trick is if you turn on the 'Prevent signing of output assembly' project-level NCrunch configuration option. In theory, this will result in a different strong name for the assembly being built from the project by NCrunch.


Cheers,

Remco
Back to top
myogui
#13 Posted : Thursday, August 9, 2012 3:41:00 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 8/7/2012(UTC)
Posts: 3

Was thanked: 1 time(s) in 1 post(s)
[EDIT] Changing the setting (Prevent signing of output assembly) did work!!!! Thanks a lot!!!!
Back to top
 |  Edit by user
1 user thanked myogui for this useful post.
Remco on 8/9/2012(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.086 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download