Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
NCrunch.exe LogVerbosity Low does not include test error details
Options
Previous Topic Next Topic
GreenMoose
#1 Posted : Tuesday, September 18, 2018 12:15:32 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 503

Thanks: 142 times
Was thanked: 66 time(s) in 64 post(s)
TeamCity server version is 2017.2.3 (build 51047)
NCrunch Console Tool v3.20.0.6 vs v3.17.0.2

2 main issues related to logging:
1) Log verbosity Low does not output test error details in log with 3.20, it did that with 3.17. Using level "Medium" introduces tons of Ncrunch internals and it is very hard to find specific test and its error output. "Summary" on the other hand seems pretty much pointless because that only outputs "start" and "stop" but nothing in between.
With 3.17 I also easily could see the error stack trace in the build log in TC UI.

2) Failed tests are no longer reported with NCrunch 3.20 (not with Low nor Medium verbosity) as they were with 3.17. See below:

3.20

3.17
Back to top
1 user thanked GreenMoose for this useful post.
michaelkroes on 9/18/2018(UTC)
michaelkroes
#2 Posted : Tuesday, September 18, 2018 12:31:20 PM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 277
Location: Netherlands

Thanks: 122 times
Was thanked: 62 time(s) in 59 post(s)
Hi thanks for reporting this!

I'll look into this.
Back to top
GreenMoose
#3 Posted : Thursday, September 20, 2018 1:19:43 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/17/2012(UTC)
Posts: 503

Thanks: 142 times
Was thanked: 66 time(s) in 64 post(s)
Update: I found there was some issues with how I redirect NCrunch stdout in the build powershell script (not sure why it went fine with v3.17 though).
However if I don't redirect stdout the failing tests seems to be reported correctly in TC, but I still don't see the error details (i.e. exception causing test failure) with Low verbosity.

*Edit: Not sure if it is by design or not but if NCrunch execution fails due to ignored test it just reports "Test failure" and the ignored tests are not marked as red in build log (making it kindof hard to spot unless you know ignored test should be failing). In my case they were not run due to required capabilities were not met.
TestFailure due to ignored tests
Back to top
 |  Edit by user
michaelkroes
#4 Posted : Monday, November 5, 2018 8:40:34 AM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 277
Location: Netherlands

Thanks: 122 times
Was thanked: 62 time(s) in 59 post(s)
Sorry for coming back so late to this issue. Again thanks for reporting this in such great detail.

I've looked at reproducing this. Without powershell and executing the console tool directly with LogVerbosity set to Low I do see failed tests being logged with the exception message attached. Could it be that the powershell wrapper is hiding something? What do you see in the raw build log?

We can't mark ignored test as red in the build log, this is handled by TeamCity. (We use service messages)

When tests are ignored, but there are also failed tests the status will be TestFailure. If you increase the log level we report exactly how we determined the exit status. This might help us in debugging this issue.

Don't hesitate to tell me if I misunderstood something from the posts :)
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.031 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download