Not showing Ignored NUnit tests

NCrunch doesn't appear to be telling me how many tests I have ignored. The test runner in Resharper does, as does running the NUnit console both locally and on the build server.

The 'Show Ignored Tests' button does not show them either.

NCrunch 2.15.0.9
NUnit 2.6.4
VS 2013 Pro

Hi, thanks for sharing this.

NCrunch actually has two concepts of ignored tests:

- Tests that are ignored through the 'Ignore test' options inside the Tests Window. This results in expressions being written into the NCrunch settings files for the applicable projects so that the tests are ignored just by NCrunch and not by other tools.
- Tests that are ignored by test framework [Ignore] tags. NCrunch will simply avoid discovering these tests entirely - as far as NCrunch is concerned, they don't exist at all.

At the moment, because NCrunch technically doesn't discover the second type of ignored test, these tests won't be counted in the Tests Window summary. I realise the limitations of this as it is useful to be able to know if there are ignored tests in the codebase. A problem that would be encountered by adding these tests to the summary would be the confusion coming from needing to find two potential sources of the value.

Probably the better option would have been to avoid using the term 'Ignored' for the NCrunch ignore feature and use something else .. like 'Suppressed'. But it's now too late to change it as so many users are familiar with the term :(

Thanks for the clarification.

Might I suggest a small update to http://www.ncrunch.net/documentation/concepts_ignored-tests with this information.

sgrassie wrote:Thanks for the clarification.

Might I suggest a small update to http://www.ncrunch.net/documentation/concepts_ignored-tests with this information.

Yes, this is a good suggestion. I'll see that this page gets updated.

Remco wrote: I realise the limitations of this as it is useful to be able to know if there are ignored tests in the codebase.

This is exactly our problem. We run our tests with NCrunch on dev machines but with other runners in the build pipeline. So we want to standardize on the NUnit Ignore tag, but then we lose all visibility on these tests in NCrunch.

The most common reason for an Ignore for us is if a test becomes broken and needs additional consideration to fix. We sometimes choose to temporarily disable it, but with NCrunch we lose visibility on that test and it can get disabled indefinitely. We're using an NUnit category as a workaround, but then we have to manually configure other test runner configs to ignore this category, which gets a little kludgey. We also lose the ability to directly associate a comment with the Ignore tag, which NUnit's tag supports.

So I wondered if there's been any thought to revisiting this. It seems like framework-ignored tests could behave in NCrunch like NCrunch's own Ignored tests, just that the franework-ignored would be configured declaratively in code.

Unfortunately I can't provide a built-in solution for this at the moment, but maybe I can provide some help on how to achieve a similar result to what you're after without any NCrunch changes. The following code uses a compiler condition to implement alternative 'Ignored' behaviour under NCrunch by using a custom attribute. This will allow you to assign your ignored tests to a specific category under NCrunch (which you might exclude via engine mode), and still have them physically ignored under other runners:

using NUnit.Framework;

public class Fixture
{
    [CustomIgnore("Requires attention")]
    [Test]
    public void Test()
    {
        // This test will be marked with an 'Ignored' category under NCrunch and will be physically ignored under other runners
    }
}

#if NCRUNCH
public class CustomIgnoreAttribute : CategoryAttribute
{
    public CustomIgnoreAttribute(string message) : base("Ignored")
    {
    }
}
#else
public class CustomIgnoreAttribute : IgnoreAttribute
{
    public CustomIgnoreAttribute(string reason = null) : base(reason)
    {
    }
}
#endif

Edit: Just realised I had the condition round the wrong way. It's been adjusted.