I see no\partial coverage info.
To repro the issue:
1) Turn on verbose logging
2) Restart NCrunch with the special "... (Reset)" button in UI
2a) Clear the output window
3) Notice that all tests just ran, all coverage info is in and visible (all bulbs without an exception are green)
4) Take a snapshot of the output window to capture the timestamp when everything was ok
[img=https://cloud.githubusercontent.com/assets/1385977/10919792/c172ed4c-827d-11e5-981c-55e5a177dc7d.png]timestamp[/img]
5) Wait a little bit more. NCrunch seems to run the tests again. Now all (or the most of) the green bulbs become white
[img=https://cloud.githubusercontent.com/assets/1385977/10919832/f9c5cbce-827d-11e5-8fb7-d5931eb7c043.png]bug[/img]
6) Copy the full log
I had the same issue before the last release too. Both on VS2013 and VS2015.
I would be glad for the workaround too, if not for the fix.
Remco NCrunch Developer
#7917
03 Nov 2015 23:13 UTC
Hi Ilya,
Thanks for reporting this problem. Please accept my apologies for the website being down – it seemed to hit some kind of DB related problem overnight and has since recovered.
It looks to me like there is an inconsistency here being caused by a race condition in the test. The test starts a number of different threads which execute in the background while the primary thread waits with a sleep.
When performing multi-threaded testing, it’s extremely important that the execution of all background threads is performed within the boundary of the foreground test thread. Where this isn’t the case, NCrunch will misreport coverage information because the background threads then become bounded by tests executed later in the run, and the coverage data isn’t allocated correctly. See here for more information - http://www.ncrunch.net/documentation/considerations-and-constraints_multi-threaded-tests.
The log file contained no unusual information about engine activity.
Does increasing the size of the Sleep make any difference?
Cheers,
Remco
Thanks for reporting this problem. Please accept my apologies for the website being down – it seemed to hit some kind of DB related problem overnight and has since recovered.
It looks to me like there is an inconsistency here being caused by a race condition in the test. The test starts a number of different threads which execute in the background while the primary thread waits with a sleep.
When performing multi-threaded testing, it’s extremely important that the execution of all background threads is performed within the boundary of the foreground test thread. Where this isn’t the case, NCrunch will misreport coverage information because the background threads then become bounded by tests executed later in the run, and the coverage data isn’t allocated correctly. See here for more information - http://www.ncrunch.net/documentation/considerations-and-constraints_multi-threaded-tests.
The log file contained no unusual information about engine activity.
Does increasing the size of the Sleep make any difference?
Cheers,
Remco
Post a reply
Log in to reply.