Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Not showing Ignored NUnit tests
Options
Previous Topic Next Topic
sgrassie
#1 Posted : Thursday, June 25, 2015 2:29:58 PM(UTC)
Rank: Member

Groups: Registered
Joined: 6/25/2015(UTC)
Posts: 15
Location: United Kingdom

Thanks: 2 times
Was thanked: 3 time(s) in 3 post(s)
NCrunch doesn't appear to be telling me how many tests I have ignored. The test runner in Resharper does, as does running the NUnit console both locally and on the build server.

The 'Show Ignored Tests' button does not show them either.

NCrunch 2.15.0.9
NUnit 2.6.4
VS 2013 Pro
Back to top
Remco
#2 Posted : Thursday, June 25, 2015 10:14:55 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this.

NCrunch actually has two concepts of ignored tests:

- Tests that are ignored through the 'Ignore test' options inside the Tests Window. This results in expressions being written into the NCrunch settings files for the applicable projects so that the tests are ignored just by NCrunch and not by other tools.
- Tests that are ignored by test framework [Ignore] tags. NCrunch will simply avoid discovering these tests entirely - as far as NCrunch is concerned, they don't exist at all.

At the moment, because NCrunch technically doesn't discover the second type of ignored test, these tests won't be counted in the Tests Window summary. I realise the limitations of this as it is useful to be able to know if there are ignored tests in the codebase. A problem that would be encountered by adding these tests to the summary would be the confusion coming from needing to find two potential sources of the value.

Probably the better option would have been to avoid using the term 'Ignored' for the NCrunch ignore feature and use something else .. like 'Suppressed'. But it's now too late to change it as so many users are familiar with the term :(
Back to top
sgrassie
#4 Posted : Friday, June 26, 2015 7:49:45 AM(UTC)
Rank: Member

Groups: Registered
Joined: 6/25/2015(UTC)
Posts: 15
Location: United Kingdom

Thanks: 2 times
Was thanked: 3 time(s) in 3 post(s)
Thanks for the clarification.

Might I suggest a small update to http://www.ncrunch.net/d.../concepts_ignored-tests with this information.
Back to top
1 user thanked sgrassie for this useful post.
Remco on 6/26/2015(UTC)
Remco
#5 Posted : Friday, June 26, 2015 10:30:12 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
sgrassie;7440 wrote:
Thanks for the clarification.

Might I suggest a small update to http://www.ncrunch.net/d.../concepts_ignored-tests with this information.


Yes, this is a good suggestion. I'll see that this page gets updated.
Back to top
Grendil
#3 Posted : Thursday, December 20, 2018 3:29:03 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2017(UTC)
Posts: 54
Location: United States of America

Thanks: 22 times
Was thanked: 11 time(s) in 10 post(s)
Remco;7436 wrote:
I realise the limitations of this as it is useful to be able to know if there are ignored tests in the codebase.

This is exactly our problem. We run our tests with NCrunch on dev machines but with other runners in the build pipeline. So we want to standardize on the NUnit Ignore tag, but then we lose all visibility on these tests in NCrunch.

The most common reason for an Ignore for us is if a test becomes broken and needs additional consideration to fix. We sometimes choose to temporarily disable it, but with NCrunch we lose visibility on that test and it can get disabled indefinitely. We're using an NUnit category as a workaround, but then we have to manually configure other test runner configs to ignore this category, which gets a little kludgey. We also lose the ability to directly associate a comment with the Ignore tag, which NUnit's tag supports.

So I wondered if there's been any thought to revisiting this. It seems like framework-ignored tests could behave in NCrunch like NCrunch's own Ignored tests, just that the franework-ignored would be configured declaratively in code.
Back to top
Remco
#6 Posted : Thursday, December 20, 2018 10:27:29 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Unfortunately I can't provide a built-in solution for this at the moment, but maybe I can provide some help on how to achieve a similar result to what you're after without any NCrunch changes. The following code uses a compiler condition to implement alternative 'Ignored' behaviour under NCrunch by using a custom attribute. This will allow you to assign your ignored tests to a specific category under NCrunch (which you might exclude via engine mode), and still have them physically ignored under other runners:

Code:
using NUnit.Framework;

public class Fixture
{
    [CustomIgnore("Requires attention")]
    [Test]
    public void Test()
    {
        // This test will be marked with an 'Ignored' category under NCrunch and will be physically ignored under other runners
    }
}

#if NCRUNCH
public class CustomIgnoreAttribute : CategoryAttribute
{
    public CustomIgnoreAttribute(string message) : base("Ignored")
    {
    }
}
#else
public class CustomIgnoreAttribute : IgnoreAttribute
{
    public CustomIgnoreAttribute(string reason = null) : base(reason)
    {
    }
}
#endif


Edit: Just realised I had the condition round the wrong way. It's been adjusted.
Back to top
 |  Edit by user
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.047 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download