NUnit Testoutput does not contain Lines Written by TestContext.WriteLine()

Given this Snippet

[Test]
public void Test()
{
    TestContext.WriteLine("Something");
    Console.WriteLine("foo");
    Assert.Fail();
}

Resharpers TestRunner outputs

bei Namespace.Class.Test() in C:\...\Class.cs:Zeile 10.


Something
foo

NCrunch outputs

foo
AssertionException : Failure
bei Namespace.Class.Test() in C:\...\Class.cs:Zeile 10.

I'd expect NChrunch to also output the Lines Written by TestContext.WriteLine() as the NUnit Documentation says:

WriteLine

Writes text to the current test result, followed by a newline.

What do you think?

Hi, thanks for sharing this issue.

This looks like a potential integration issue. NUnit is probably capturing the text within the TextContext, then outputting this in an area that NCrunch doesn't capture. I've made a note to review the integration to see if it can be resolved. For the time being, I recommend using Debug.WriteLine instead.

I believe in NUnit 3 we've been asked to switch to use NUnit.Framework.TestContext.Progress.WriteLine(message) to ensure messages remain associated with their test thread. As I recall Debug.WriteLine leads to problems under NUnit parallelization, because you'd get interleaved results. I realize NCrunch doesn't use NUnit's parallel capabilities, but it would be helpful for compatibility with existing tests if those progress messages were still picked up in the NCrunch test output.

ReSharper does this, and in fact it writes them to its UI in real time, which can be really helpful for debugging slow running BDD tests. My QA team would like to move to NCrunch when the dev team does, but they write a lot of slow running BDD's. The initial feedback I'm getting is having to wait until execution completes before seeing any of the trace output would be a showstopper. They have tests that might hang early or might just run long, and so they want the interactive feedback to be able to stop the test rather than wait it out. I understand this isn't NCrunch's core audience, but I thought I'd pass it along nonetheless.

Grendil wrote:I believe in NUnit 3 we've been asked to switch to use NUnit.Framework.TestContext.Progress.WriteLine(message) to ensure messages remain associated with their test thread. As I recall Debug.WriteLine leads to problems under NUnit parallelization. I realize NCrunch doesn't need NUnit's parallel capabilities, but it would be helpful for compatibility if those progress messages were still picked up in the NCrunch test output. (ReSharper does this, and in fact it writes them to its UI in real time, which can be helpful for debugging slow running BDD tests.)

Yes, it makes sense that a different capturing system would be needed for tests being run over multiple threads.

It looks like this data is reported by NUnit quite differently to the other trace systems. It will require special handling from NCrunch to process. I've made a note to address this.

Considering the wide range of different trace systems that now exist and the changes in this area, it might be sensible to start wrapping up these systems behind your own methods so that you can easily redirect trace information. For this time being, such a structure will let you work with NCrunch too. You can place a '#if NCRUNCH' directive inside the wrapper to redirect it elsewhere under NCrunch until hadnling for Progress.WriteLine is introduced.

Great idea! Thanks.

My QA team would like to move to NCrunch when the dev team does, but they write a lot of slow running BDD's. The initial feedback I'm getting is having to wait until execution completes before seeing any of the trace output would be a showstopper. They have tests that might hang early or might just run long, and so they want the interactive feedback to be able to stop the test rather than wait it out. I understand this isn't NCrunch's core audience, but I thought I'd pass it along nonetheless, especially if you're thinking about expanding support there.

Grendil wrote:My QA team would like to move to NCrunch when the dev team does, but they write a lot of slow running BDD's. The initial feedback I'm getting is having to wait until execution completes before seeing any of the trace output would be a showstopper. They have tests that might hang early or might just run long, and so they want the interactive feedback to be able to stop the test rather than wait it out. I understand this isn't NCrunch's core audience, but I thought I'd pass it along nonetheless, especially if you're thinking about expanding support there.

Thanks for the feedback here. Because of NCrunch's need to scale much further than serial test runners, it takes some effort to implement this, which is why it hasn't happened yet. You can help here by requesting the feature through uservoice. I'm not sure if there's an existing request for this. If you don't find one, feel free to create it.