Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Static initialization code isn't considered covered
Strilanc
#1 Posted : Sunday, November 11, 2012 8:19:13 AM(UTC)
Rank: Member

Groups: Registered
Joined: 4/3/2012(UTC)
Posts: 17

Thanks: 1 times
Code that is invoked when initializing static fields of classes is not considered covered by tests. Although there are cases where this makes sense, I'm not so sure it makes sense in general.

I have an AnonymousComparer class. It happens that the only place it is instantiated is to initialize a few static fields (representing different ways of ordering a particular type). I expected to see the anonymous comparer class as 'covered', because I was using its results as part of other tests. However, it isn't considered covered.

Perhaps a more serious issue, that is probably related, is that static fields seem to flip/flop between being marked as covered and being marked as not covered. But I haven't figured out exactly what triggers that yet.

[DebuggerStepThrough]
public sealed class AnonymousComparer<T> : IComparer<T> {
private readonly Func<T, T, int> _compare;
public AnonymousComparer(Func<T, T, int> compare) {
if (compare == null) throw new ArgumentNullException("compare");
this._compare = compare;
}
public int Compare(T x, T y) {
return _compare(x, y);
}
}
Remco
#2 Posted : Sunday, November 11, 2012 8:07:25 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Hi Strilanc -

Generally when people observe issues like this, the cause often comes from misalignment between the perceived code coverage and the way in which NCrunch actually collects it. I'm busy updating the NCrunch documentation at the moment with a mention about why this sort of thing can happen. I'll copy paste the explanation into here in the hope that you'll find it useful:

Code coverage in a conceptual sense can be interpreted in different ways. Broadly there are two ways to look at code coverage:

Logical Code Coverage: This school of thought identifies a line of code as being covered if it would be executed at any time in the pursuit of the execution of a test within the suite. The theory is that if you were to set a debugger breakpoint on a line of code, and the debugger broke into the code when you tried to run a specific test, then this code is considered covered by the test.

Physical Code Coverage: This approach to code coverage looks specifically at which lines of code are executed after a test has started executing. Lines are only considered to be covered if they are physically executed while the test is executing.

As an example, consider a static constructor. Code within the constructor would always need to be executed before any code in the class can run. This means that the static constructor can be logically considered covered by every test that makes use of code within its class. However, as the static constructor can only be executed once for an entire test run that may involve multiple tests, it will only ever be physically covered by the first test to make use of any of the code within its class.

NCrunch's code coverage is physical, not logical.


... If you're not sure about why a particular piece of static code isn't being reported as covered by NCrunch (but you know that it IS executed), it's worth setting a breakpoint on this code then kicking off the test in NCrunch with the debugger attached. When the code hits the breakpoint, examine the stack trace. Where is the entry point to your code? Is it coming from a test within your codebase, or perhaps some kind of setup code that runs before the test? (i.e. TestCaseSource).
GreenMoose
#3 Posted : Wednesday, October 2, 2013 7:02:31 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 507

Thanks: 145 times
Was thanked: 66 time(s) in 64 post(s)
I have a lot of lines like these in my fixtures (max 1 per fixture):
private static readonly ILog _log =
LogManager.GetLogger(MethodBase.GetCurrentMethod().DeclaringType);

And these keep lowering code coverage metrics. If I run tests in a new test runner, they are reported as covered by NCrunch but then as I continue work and tests are re-run, they are yet again marked as non-covered. Can it be because NCrunch reuses the process and simply not sees this line as being executed again when running same test within same process?

It would be nice to have some [ExcludeFromCoverage] attribute valid for fields as well, as a workaround.
Remco
#4 Posted : Wednesday, October 2, 2013 11:26:13 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
This would be happening because the static members are being initialised during the test discovery step, which happens before any tests begin executing. You're correct in your understanding that NCrunch will re-use the dynamic analysis processes for test execution. If you set the 'Framework utilisation type' for your test framework to 'StaticAnalysis', it's possible you won't have this problem - although such a configuration change may impact the structure of tests you have in your codebase, so it should be done carefully.

To work around this, at the end of each line, place: //ncrunch: no coverage
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.054 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download