Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
NUnit3 TestFixtureSource types have no coverage information associated with them
Options
Previous Topic Next Topic
alastairs
#1 Posted : Tuesday, November 1, 2022 1:51:47 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
Hi all

I think I've found a bug in NCrunch's support for NUnit3: when using the TestFixtureSource attribute with a type as described here, no coverage information is collected for the GetEnumerator() implementation. The below screenshot illustrates this, using the below minimal reproduction:

Code:
[TestFixtureSource(typeof(Source))]
public class Fixture
{
    private readonly string _parameter;

    public Fixture(string parameter) => _parameter = parameter;

    [Test] public void Noop() =>
        Assert.That(_parameter.Clone(), Is.SameAs(_parameter));
}

internal class Source : IEnumerable
{
    public IEnumerator GetEnumerator()
    {
        yield return new object[] {"foo"};
    }
}




Is this a known issue?
Back to top
WWW
Remco
#2 Posted : Tuesday, November 1, 2022 10:30:12 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this.

Although I'll admit that it isn't ideal, this is actually by design. Code that executes inside TestCaseSource is executed at Analysis time, which is when we discover tests. As such, there is no test in the codebase to associate this coverage with.

TestCaseSource is probably the most misunderstood feature in NUnit. When you use this attribute, you are actually extending the logic of the test framework itself and writing user code that will run during test discovery. Clearly, it's a very powerful feature, but it's worth considering that the code running inside TestCaseSource does not have the same liberty as normal test code. It is not executed inside the suite's normal hierarchical structure, does not have the same granularity of error reporting, and we don't have a mechanism for tracking its coverage.

Please take care with this attribute. If you introduce code here that is unreliable or performs badly, you can end up with a very bad experience.
Back to top
1 user thanked Remco for this useful post.
alastairs on 11/2/2022(UTC)
alastairs
#3 Posted : Wednesday, November 2, 2022 9:45:13 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
Understood--many thanks Remco for your prompt reply!
Back to top
WWW
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.027 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download