Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
NUnit Testoutput does not contain Lines Written by TestContext.WriteLine()
Options
Previous Topic Next Topic
Paranoid
#1 Posted : Friday, June 9, 2017 8:06:18 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/9/2017(UTC)
Posts: 2
Location: Germany

Thanks: 2 times
Given this Snippet
Code:
[Test]
public void Test()
{
    TestContext.WriteLine("Something");
    Console.WriteLine("foo");
    Assert.Fail();
}

Resharpers TestRunner outputs
Quote:
bei Namespace.Class.Test() in C:\...\Class.cs:Zeile 10.


Something
foo

NCrunch outputs
Quote:
foo
AssertionException : Failure
bei Namespace.Class.Test() in C:\...\Class.cs:Zeile 10.

I'd expect NChrunch to also output the Lines Written by TestContext.WriteLine() as the NUnit Documentation says:
Quote:
WriteLine

Writes text to the current test result, followed by a newline.


What do you think?
Back to top
Remco
#2 Posted : Friday, June 9, 2017 11:31:19 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Hi, thanks for sharing this issue.

This looks like a potential integration issue. NUnit is probably capturing the text within the TextContext, then outputting this in an area that NCrunch doesn't capture. I've made a note to review the integration to see if it can be resolved. For the time being, I recommend using Debug.WriteLine instead.
Back to top
Grendil
#3 Posted : Friday, July 28, 2017 11:22:55 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2017(UTC)
Posts: 54
Location: United States of America

Thanks: 22 times
Was thanked: 11 time(s) in 10 post(s)
I believe in NUnit 3 we've been asked to switch to use NUnit.Framework.TestContext.Progress.WriteLine(message) to ensure messages remain associated with their test thread. As I recall Debug.WriteLine leads to problems under NUnit parallelization, because you'd get interleaved results. I realize NCrunch doesn't use NUnit's parallel capabilities, but it would be helpful for compatibility with existing tests if those progress messages were still picked up in the NCrunch test output.

ReSharper does this, and in fact it writes them to its UI in real time, which can be really helpful for debugging slow running BDD tests. My QA team would like to move to NCrunch when the dev team does, but they write a lot of slow running BDD's. The initial feedback I'm getting is having to wait until execution completes before seeing any of the trace output would be a showstopper. They have tests that might hang early or might just run long, and so they want the interactive feedback to be able to stop the test rather than wait it out. I understand this isn't NCrunch's core audience, but I thought I'd pass it along nonetheless.
Back to top
 |  Edit by user
Remco
#4 Posted : Saturday, July 29, 2017 12:01:24 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Grendil;10880 wrote:
I believe in NUnit 3 we've been asked to switch to use NUnit.Framework.TestContext.Progress.WriteLine(message) to ensure messages remain associated with their test thread. As I recall Debug.WriteLine leads to problems under NUnit parallelization. I realize NCrunch doesn't need NUnit's parallel capabilities, but it would be helpful for compatibility if those progress messages were still picked up in the NCrunch test output. (ReSharper does this, and in fact it writes them to its UI in real time, which can be helpful for debugging slow running BDD tests.)


Yes, it makes sense that a different capturing system would be needed for tests being run over multiple threads.

It looks like this data is reported by NUnit quite differently to the other trace systems. It will require special handling from NCrunch to process. I've made a note to address this.

Considering the wide range of different trace systems that now exist and the changes in this area, it might be sensible to start wrapping up these systems behind your own methods so that you can easily redirect trace information. For this time being, such a structure will let you work with NCrunch too. You can place a '#if NCRUNCH' directive inside the wrapper to redirect it elsewhere under NCrunch until hadnling for Progress.WriteLine is introduced.
Back to top
1 user thanked Remco for this useful post.
Grendil on 7/29/2017(UTC)
Grendil
#5 Posted : Saturday, July 29, 2017 12:37:50 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2017(UTC)
Posts: 54
Location: United States of America

Thanks: 22 times
Was thanked: 11 time(s) in 10 post(s)
Great idea! Thanks.
Back to top
Grendil
#6 Posted : Saturday, July 29, 2017 12:48:21 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2017(UTC)
Posts: 54
Location: United States of America

Thanks: 22 times
Was thanked: 11 time(s) in 10 post(s)
My QA team would like to move to NCrunch when the dev team does, but they write a lot of slow running BDD's. The initial feedback I'm getting is having to wait until execution completes before seeing any of the trace output would be a showstopper. They have tests that might hang early or might just run long, and so they want the interactive feedback to be able to stop the test rather than wait it out. I understand this isn't NCrunch's core audience, but I thought I'd pass it along nonetheless, especially if you're thinking about expanding support there.
Back to top
Remco
#7 Posted : Saturday, July 29, 2017 10:13:49 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Grendil;10885 wrote:
My QA team would like to move to NCrunch when the dev team does, but they write a lot of slow running BDD's. The initial feedback I'm getting is having to wait until execution completes before seeing any of the trace output would be a showstopper. They have tests that might hang early or might just run long, and so they want the interactive feedback to be able to stop the test rather than wait it out. I understand this isn't NCrunch's core audience, but I thought I'd pass it along nonetheless, especially if you're thinking about expanding support there.


Thanks for the feedback here. Because of NCrunch's need to scale much further than serial test runners, it takes some effort to implement this, which is why it hasn't happened yet. You can help here by requesting the feature through uservoice. I'm not sure if there's an existing request for this. If you don't find one, feel free to create it.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.050 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download