Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Add ClearCoverageMarkers()
yanglee
#1 Posted : Monday, August 12, 2013 3:43:10 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/25/2012(UTC)
Posts: 89
Location: China

Thanks: 23 times
Was thanked: 19 time(s) in 18 post(s)
"Show coverage for selected test only" is probably my favorite feature. I'd like to suggest an enhancement to this feature: NCrunchEnvironment.ClearCoverageMarkers().

It should work only when "Show coverage for selected test only" is enabled. Below is an example that this feature could be helpful. It should be much more helpful in more complex scenarios.

[img](- BROKEN LINK -)[/img]
Remco
#2 Posted : Monday, August 12, 2013 6:36:22 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 961 times
Was thanked: 1291 time(s) in 1197 post(s)
Hi Yang,

Thanks for the suggestion!

I'm looking at this objectively with the idea that the key problem you're trying to solve is that the setup area of your test is irrelevant to the coverage that you're actually interested in, and the overlapping coverage from the setup is obscuring the much more relevant coverage information from the test (thanks for the image, this really helps a lot).

Before looking into feature solutions to this problem, I'm wondering if I can learn a bit more about how the code coverage is functioning in your above example. Normally, the constructor of the test fixture would be called during the setup for the test fixture (a test which NCrunch would call 'StorageFeature.*'). There is physical separation between the code coverage for this fixture test and the test itself ('add_new_item'). Because of this separation, it should be possible to simply show test coverage for add_new_item only, and the coverage from the constructor should be hidden. Does this match the behaviour you're experiencing? I know this would be the case if the constructor code was implemented under TestFixtureSetUp although I would need to double-check the logic around constructor calls in NUnit to be certain of the entire call sequence.
yanglee
#3 Posted : Monday, August 12, 2013 9:50:44 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/25/2012(UTC)
Posts: 89
Location: China

Thanks: 23 times
Was thanked: 19 time(s) in 18 post(s)
Quote:
Normally, the constructor of the test fixture would be called during the setup for the test fixture (a test which NCrunch would call 'StorageFeature.*'). There is physical separation between the code coverage for this fixture test and the test itself ('add_new_item'). Because of this separation, it should be possible to simply show test coverage for add_new_item only, and the coverage from the constructor should be hidden.

I didn't know that, thanks for the tip!

The example above was for demo purpose. But in my real projects, I put all context setup code into NUnit SetUp method and action attributes, which are executed per test. So "show test coverage for add_new_item only" doesn't work well in this case.

During testing complex acceptance / integration tests, I may want to move ClearCoverageMarkers() around to show more relevant coverages. For example, if an exception throw in the middle of a test, I may only want to see the coverages of a few lines before where exception was throw.
Remco
#4 Posted : Monday, August 12, 2013 11:31:25 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 961 times
Was thanked: 1291 time(s) in 1197 post(s)
That makes sense. If the code is in a standard [SetUp] method, then it will be executed as part of the test .. so there'll be no coverage separation between the test and the setup.

I may be able to suggest a test structure that can help with working around this. It involves treating the test fixture as a scenario, and the tests within the fixture as requirements measured over the results of this scenario. For example:

Code:
public class WhenIClickTheBigRedButton
{
  private FormWithBigRedButton subject;

  [TestFixtureSetUp]
  public void ExecuteScenario()
  {
    subject = new FormWithBigRedButton();
    subject.ClickBigRedButton();
  }

  [Test]
  public void FormShouldTurnRed()
  {
    Assert.That(subject.BackgroundColor, Is.EqualTo(Colors.Red));
  }

  [Test]
  public void FormShouldGrowLarger()
  {
    Assert.That(subject.Width, Is.GreaterThan(300));
    Assert.That(subject.Height, Is.GreaterThan(250));
  }
}


... In this way, the setup code is always separated from the tests - giving much flexibility. You can also further extend the scenario using inheritance, allowing for much more diverse and complex scenario constructions complete with code re-use.

Anyway, I'll need to think more about allowing tests to manipulate their own coverage outside of normal execution flow. There is potential for such a feature to become dangerous (i.e. if you were to accidentally check-in your debug code, this could create some VERY interesting issues for team members working on the same codebase). There may be better ways to peg this down so that such a modification is transient and will only appear for the life of the NCrunch session - rather than being embedded in the code..


Cheers,

Remco
1 user thanked Remco for this useful post.
yanglee on 8/12/2013(UTC)
yanglee
#5 Posted : Monday, August 12, 2013 11:49:00 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/25/2012(UTC)
Posts: 89
Location: China

Thanks: 23 times
Was thanked: 19 time(s) in 18 post(s)
Thanks Remco, will try to use that test structure for some of my tests.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.039 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download