Why does the Metric Window include Tests Projects?

Two Questions:

1. Why does NCrunch shows the Test Projects within the Metrics window that shows the code coverage?

2. What benefit or purpose does it serve to include Test Projects or even show them in this window?

To me it seems very odd as the Code Coverage is supposed to tell me what lines of code are covered by tests, meaning the tests should be excluded form any code coverage metrics.

I know I can go through an Excude from Metric Aggregation, however it doesn't remove it from the window.

If there is a really good reason then great, however I would assume if there is it is for a different purpose that what unit tests lines of code are covered by tests, and if so maybe this window should separate Unit Tests from Non-Unit Tests projects so that they each have their own report that better explains what you are seeing for each.

Hi, thanks for posting.

NCrunch itself does not discriminate in this situation, largely because it cannot make safe assumptions about how the test code or the solution is structured. Nor can it safely make decisions on behalf of the user about they should interpret or make use of metrics provided to them.

For example, let's just say you have a utility project included in the solution that contains some code that is used by your tests to help them do their thing. This utility project may not contain any production code and isn't deployed with the application, nor does it reference any test libraries. By the logic you imply, this project should be excluded from metrics aggregation. NCrunch has no way of knowing this project should be excluded.

In another example, an early way that some people wrote automated tests within .NET was to embed the tests themselves into the production projects surrounded by compiler directives to exclude this code for release builds. I won't try to argue for the logic of this approach today, but in this situation NCrunch would have no way to classify projects in order to automatically exclude them.

Some people also like to have the test projects included in this view as it can help them to locate 'dead' test code.

The simple option is exactly as you've described - exclude the projects you don't want to see. The tool gives you the information and the power to choose which elements are irrelevant to you.

Those are rare corner cases for today, especially the "early way that some people wrote automated tests". It seems rather easy to figure out a test project, as you have to run the tests, so if a project has any Unit Test class then they are probably a Unit Test project in todays world of unit testing, also you could use a configurable threshold setting to determine it. I think identifying Test Projects is not a difficult thing. Worse case is to add a flag to set on all projects marking it as a Test project or not.

I guess I can see some may like looking at them to see what test code is "Dead" however that is not a code coverage issue, it is an active or inactive test code issue. I think have two sections in the metrics window that separated the Test Code from the Code Under Test, with the Code Under Test section being Test Code Coverage, and the Test Code section being Active Test Code. It is really confusing and misleading having the percentage of active test code rolled up in to the test code coverage percentages. It is also a pain to always have to go and set them to be excluded from the roll up. I would be find with the set it and forget it setting to mark all projects as a test project via an NCrunch configuration setting on each project.

I really think separating them makes more sense to the developer/user of NCrunch, as everyone I work with who uses NCrunch has asked me why this screen is doing this, without any prompting from me.

Anyway, thanks for answering the question about why it is the way it is. I guess now I am asking for a feature request, it could even be a configuration switch to allow both views of the data so as not to upset anyone who likes it the way it is. :)

You're welcome to vote for it as a feature request if you like. If enough people say they want the auto detection, it will be hard for me to argue against it :)

Thanks I created a feature request on the NCrunch UserVoice:

Separate Test Projects from Non-Test Projects in Metrics Window

CreepyGnome wrote:Two Questions:

1. Why does NCrunch shows the Test Projects within the Metrics window that shows the code coverage?

I for one am very happy it does since I use that to find dead test code such as obsolete setups never executed on mock objects, incorrect assumptions regarding to behavior etc. :).

And the metrics windows in this case is useful in caes where you have a 100% covered test projects since then you quickly spot if you introduce redundant test code (or if it becomes redundant due to production code changes etc.).

*Edit: Never mind above, I should've read entire thread before replying. Your feature suggestion seems good :).