Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Console tool reports "weird" Build Results
Options
Previous Topic Next Topic
spolonski
#1 Posted : Monday, July 11, 2016 12:21:31 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 7/11/2016(UTC)
Posts: 32
Location: Germany

Thanks: 6 times
Was thanked: 4 time(s) in 4 post(s)
I get "weird" Build Results if i run my test in a console runner. In Visual Studio works all as expected.
Here are screenshots:


Here is an output of a console tool:
Back to top
Remco
#2 Posted : Tuesday, July 12, 2016 1:50:41 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi,

This looks like an installation issue with the console tool.

If I were to guess, I would say that the console tool is loading an older version of its modules when attempting to run the build. Possibly when it was initially installed, the modules were extracted without the directory information, so they were all put in the root directory of the tool. If the tool was later upgraded and the modules were extracted properly, this would give some very strange results.

I'd suggest clearing out the installation of the console tool entirely and reinstalling it from scratch. Make sure you extract all files with full paths if using the ZIP.
Back to top
spolonski
#3 Posted : Tuesday, July 12, 2016 6:05:13 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 7/11/2016(UTC)
Posts: 32
Location: Germany

Thanks: 6 times
Was thanked: 4 time(s) in 4 post(s)
Hi,

Console log looks better


But Results are still not ok. There are all tests now listed, but labeled as "not run". I dont undersand why?
Back to top
Remco
#4 Posted : Tuesday, July 12, 2016 6:18:30 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Are your tests making use of any capabilities?

It looks like the tests are being queued, but the engine is electing not to run them. Are you using the console tool on a dev machine, or on the build server? If you are using it only on the build server, what sort of results do you see if you run it locally on your dev machine?
Back to top
spolonski
#5 Posted : Tuesday, July 12, 2016 7:26:22 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 7/11/2016(UTC)
Posts: 32
Location: Germany

Thanks: 6 times
Was thanked: 4 time(s) in 4 post(s)
Thanks for helping me to diagnose the problem.

For some reason was Grid Node down. After restarting the service, the results are good.

But in our build enviroment we are testing the return code of console tool. And it was 0 althrough there was a connection error.
Could you please change this behaviour?

By the way: One thing was weird too: there was no connecition to the grid server althrough code coverage was computed.
Back to top
Remco
#6 Posted : Tuesday, July 12, 2016 8:23:52 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks, it's great to hear that you managed to identify the problem.

I admit that I'm not sure how to sensibly handle such a scenario using the existing return codes, as there may be perfectly valid reasons for why a test wasn't executed in a run. To expect any unexecuted test to be an error situation, we may end up with false positives for when people are using the tool differently to you.

Perhaps this will need to be done with an extra configuration setting or return code. I'll have a think about it.

The code coverage results for test runs are always remembered using the NCrunch cache file, so the coverage results are you are examining are from the last successful test run.
Back to top
Remco
#7 Posted : Wednesday, July 20, 2016 7:55:33 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
v2.24 has just been released with an extra return code (exitcode 5) for when the tool runs end to end without executing every known test.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.101 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download