Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Test coverage dots show wrong tests
MatthewSteeples
#1 Posted : Wednesday, March 4, 2020 5:50:24 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 10/28/2014(UTC)
Posts: 99
Location: United Kingdom

Thanks: 5 times
Was thanked: 14 time(s) in 12 post(s)
This is going to be a fun one to diagnose! The summary is that if I click the coverage dot in the left hand margin, some of the tests are correctly listed, some are completely missing (even though they have been run) and some are listed as covering that line but they don't (and selecting to run the test and breakpoint at that line confirms it). Manually re-running one of the tests that doesn't cover the line doesn't remove it from the list either.


  • Windows 10 1909
  • Full Framework (version 4.7.2)
  • Visual Studio 16.4.5
  • NCrunch 4.3.0.13
  • Optimised instrumentation mode enabled (will check whether disabling this changes anything, but that will take a few hours to run)


I'm also going to submit a bug report with the details of the test names. Let me know if there are any other details you need
Remco
#2 Posted : Wednesday, March 4, 2020 11:50:59 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,982

Thanks: 781 times
Was thanked: 1008 time(s) in 960 post(s)
Thanks for sending through the bug report. The report was full of internal exceptions indicating a major desync between the internal coverage database and your codebase.

When these desyncs happen, they can be hard to troubleshoot as we can usually can only access the corrupted state long after everything has already gone wrong.

I'm willing to bet that a reset of the engine and full re-run of your tests will resolve the issue. Is this the case for you?

And if so, are you able to make the problem appear again? It should be very noticeable if you turn on the 'Log to output window' setting and keep an eye on the VS/NCrunch output window. Make sure your log verbosity is set to summary.
MatthewSteeples
#3 Posted : Thursday, March 5, 2020 12:26:48 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 10/28/2014(UTC)
Posts: 99
Location: United Kingdom

Thanks: 5 times
Was thanked: 14 time(s) in 12 post(s)
Remco;14501 wrote:
The report was full of internal exceptions indicating a major desync between the internal coverage database and your codebase.


Where is this database is stored? If the answer is where I think it is (the NCrunch Cache Storage Path) then could having 2 different branches of the same solution (so the same sln name) open at the same time with the same path cause the issue?
Remco
#4 Posted : Thursday, March 5, 2020 1:00:58 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,982

Thanks: 781 times
Was thanked: 1008 time(s) in 960 post(s)
MatthewSteeples;14503 wrote:


Where is this database is stored? If the answer is where I think it is (the NCrunch Cache Storage Path) then could having 2 different branches of the same solution (so the same sln name) open at the same time with the same path cause the issue?


In theory, no. In practice, we haven't extensively tested this. It might be a good idea to give each branch its own solution file, so that the cache can be stored separately.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.036 seconds.