Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
2 Pages12>
NCrunch 4 - Optimized mode - Debugging tests is mission impossible?
Options
Previous Topic Next Topic
Advisto
#1 Posted : Tuesday, November 12, 2019 3:51:54 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/15/2018(UTC)
Posts: 11
Location: Netherlands

Was thanked: 2 time(s) in 2 post(s)
In NCrunch 4 with VS 2019, while using the Optimized mode, it seems te be nearly impossible to debug my tests. The debugger starts ok, and breaks on the breakpoint I set. But...
  1. I'm unable to watch my variables (parameters of the method are possible to watch, but variables instantiated within the method-with-breakpoint can't be watched).
  2. I'm can't step into another method, gives me a 'sources not available' page


After setting the mode to legacy debugging works fine.

Any others experienced this issue?
Back to top
Remco
#2 Posted : Tuesday, November 12, 2019 10:49:06 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for sharing this issue.

Can you confirm if your code is using portable PDBs or Win32 ones? Do you notice any difference if you switch between these? Note that you can control this under your advanced build settings in the VS project configuration.
Back to top
warappa
#7 Posted : Thursday, November 14, 2019 8:02:16 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 10/19/2016(UTC)
Posts: 5
Location: Austria

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
I have the same issue concerning debugging: in legacy mode debugging works, in optimized mode only half-way.
Breakpoints are hit but if I try to step into an method, at first I get the "Source Not Available" window and then after another "Step Into" key hit it behaves just like "Step Over" instead of "Step Into" (= staying in the original method and moving to next step instead).

NCrunch 4.1.0.1
VS: 16.3.9
Test project: netcoreapp2.2 with Microsoft.NET.Test.Sdk 16.2.0, MSTest.TestAdapter 1.4.0, MSTest.TestFramework 1.4.0
Tested project: netstandard2.0
Back to top
 |  Edit by user
spolonski
#3 Posted : Thursday, November 14, 2019 8:24:47 AM(UTC)
Rank: Member

Groups: Registered
Joined: 7/11/2016(UTC)
Posts: 27
Location: Germany

Thanks: 6 times
Was thanked: 4 time(s) in 4 post(s)
I have issues too. Value evaluation dosn't work in Optimised mode.

Remco;14062 wrote:
Hi, thanks for sharing this issue.

Can you confirm if your code is using portable PDBs or Win32 ones? Do you notice any difference if you switch between these? Note that you can control this under your advanced build settings in the VS project configuration.

My Code uses Portable pdbs. Which one is win32? i have only None, Full, Pdb-only, Portable and Embeded Items in Debugging information combobox.
Back to top
Remco
#4 Posted : Thursday, November 14, 2019 11:06:12 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
spolonski;14076 wrote:

My Code uses Portable pdbs. Which one is win32? i have only None, Full, Pdb-only, Portable and Embeded Items in Debugging information combobox.


Win32 pdb is 'Full'.


Thanks for sharing these extra details. I'm going to do some experimentation to see if I can get this to appear on my end. I hope I won't need to ask for a sample to reproduce the issue ... I guess we'll see how we go.
Back to top
 |  Edit by user
Remco
#8 Posted : Friday, November 15, 2019 4:42:23 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
I've managed to identify some issues around local variable scoping inside the portable PDBs that we're rebuilding. I'll let you know as soon as a fix is available.
Back to top
1 user thanked Remco for this useful post.
spolonski on 11/15/2019(UTC)
tomtrias
#9 Posted : Friday, November 15, 2019 7:49:53 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/25/2019(UTC)
Posts: 4
Location: United States of America
I have had similar issues with watches / variable or expression evaluation using NCrunch Optimised Mode in Visual Studio 2019 for a .NET 4.7.2 project using Win32 (Full) PDBs. I also noticed that stepping into or out of methods now shows a bit of stepping through external code (instrumentation, if I'm not mistaken). When I switch back to Legacy, these issues disappear.
Back to top
Remco
#10 Posted : Friday, November 15, 2019 10:40:25 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
tomtrias;14086 wrote:
I have had similar issues with watches / variable or expression evaluation using NCrunch Optimised Mode in Visual Studio 2019 for a .NET 4.7.2 project using Win32 (Full) PDBs. I also noticed that stepping into or out of methods now shows a bit of stepping through external code (instrumentation, if I'm not mistaken). When I switch back to Legacy, these issues disappear.


I have a theory about what is causing the external code stepping, though I had hoped the local variable inspection for Full PDBs was OK. Are you able to narrow this to a specific case that you can share with me? This would be a separate problem from the other issues reported in this thread.
Back to top
tomtrias
#11 Posted : Friday, November 15, 2019 11:01:11 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/25/2019(UTC)
Posts: 4
Location: United States of America
I will narrow down a small reproductive case when I get a chance; hopefully it is fairly simple to do. The code base I am currently working on is large and complicated.
Back to top
Remco
#12 Posted : Friday, November 15, 2019 11:28:40 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
tomtrias;14088 wrote:
I will narrow down a small reproductive case when I get a chance; hopefully it is fairly simple to do. The code base I am currently working on is large and complicated.


Thanks for your help!
Back to top
Advisto
#5 Posted : Tuesday, November 19, 2019 4:04:58 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/15/2018(UTC)
Posts: 11
Location: Netherlands

Was thanked: 2 time(s) in 2 post(s)
Remco;14062 wrote:
Hi, thanks for sharing this issue.

Can you confirm if your code is using portable PDBs or Win32 ones? Do you notice any difference if you switch between these? Note that you can control this under your advanced build settings in the VS project configuration.

Sorry for my late reply, it's a netcoreapp2.1 project with portable PDB's. Just tested with Full (win32), which seem a bit better (I can inspect some more variables it seems) but 'step into' still doesn't work (while it works when I set a breakpoint into that method and hit F5).

Hope it helps, and I'm looking forward to the update because I really love the performance!
Back to top
Remco
#6 Posted : Tuesday, November 19, 2019 10:28:48 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
Advisto;14104 wrote:

Sorry for my late reply, it's a netcoreapp2.1 project with portable PDB's. Just tested with Full (win32), which seem a bit better (I can inspect some more variables it seems) but 'step into' still doesn't work (while it works when I set a breakpoint into that method and hit F5).


Thanks! This confirms it's being caused by the two problems we've pinned down. A fix is incoming for these issues soon.
Back to top
1 user thanked Remco for this useful post.
UppSol on 11/20/2019(UTC)
tomtrias
#13 Posted : Monday, November 25, 2019 9:24:10 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/25/2019(UTC)
Posts: 4
Location: United States of America
This is pretty weird; I am getting some really strange behavior inspecting variables when I have a situation like the following (even in Legacy mode):

foreach(var item in collection.Where(item => item.BooleanProperty)) {
...
}
foreach(var item in collection) {
...
}

If I inspect "item" in the second "foreach", I am seeing the value for "item" from the previous "foreach" scope. I have verified that using other unit test runners to debug does not have this issue.
Back to top
Remco
#14 Posted : Monday, November 25, 2019 11:58:56 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
tomtrias;14125 wrote:

If I inspect "item" in the second "foreach", I am seeing the value for "item" from the previous "foreach" scope. I have verified that using other unit test runners to debug does not have this issue.


Thanks for sharing this. This problem is an old existing one caused by the improper scoping of variables. It's existed in NCrunch since V1, and it's one of the problems fixed in the new optimised mode.

It just so happens that at the moment, the optimised mode has some other more serious problems with debugging, but there will be build published very soon with fixes for these.
Back to top
Remco
#15 Posted : Tuesday, November 26, 2019 1:46:41 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
Back to top
warappa
#17 Posted : Wednesday, November 27, 2019 9:42:34 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 10/19/2016(UTC)
Posts: 5
Location: Austria

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
Remco;14128 wrote:
The build below includes fixes for all known debug problems with NCrunch V4 under optimised mode. Can everyone with issues here please give this a try and let me know if all is well?

NCrunch_Console_4.2.0.2.msi
NCrunch_Console_4.2.0.2.zip
NCrunch_GridNodeServer_4.2.0.2.msi
NCrunch_GridNodeServer_4.2.0.2.zip
NCrunch_LicenseServer_4.2.0.2.zip
NCrunch_VS2008_4.2.0.2.msi
NCrunch_VS2010_4.2.0.2.msi
NCrunch_VS2010_4.2.0.2.zip
NCrunch_VS2012_4.2.0.2.msi
NCrunch_VS2012_4.2.0.2.zip
NCrunch_VS2013_4.2.0.2.msi
NCrunch_VS2013_4.2.0.2.zip
NCrunch_VS2015_4.2.0.2.msi
NCrunch_VS2015_4.2.0.2.msi.7z
NCrunch_VS2015_4.2.0.2.zip
NCrunch_VS2017_4.2.0.2.msi
NCrunch_VS2017_4.2.0.2.msi.7z
NCrunch_VS2017_4.2.0.2.zip
NCrunch_VS2019_4.2.0.2.msi
NCrunch_VS2019_4.2.0.2.msi.7z
NCrunch_VS2019_4.2.0.2.zip


This version works for me now in optimized mode ("Step Into" and inspecting variables)!
Thanks for the fix!

VS: 16.3.9
Test project: netcoreapp2.2 with Microsoft.NET.Test.Sdk 16.2.0, MSTest.TestAdapter 1.4.0, MSTest.TestFramework 1.4.0
Tested project: netstandard2.0
Back to top
1 user thanked warappa for this useful post.
Remco on 11/27/2019(UTC)
Advisto
#16 Posted : Wednesday, November 27, 2019 1:05:41 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/15/2018(UTC)
Posts: 11
Location: Netherlands

Was thanked: 2 time(s) in 2 post(s)
Remco;14128 wrote:
The build below includes fixes for all known debug problems with NCrunch V4 under optimised mode. Can everyone with issues here please give this a try and let me know if all is well?


Hi Remco,

When I set a breakpoint in my implementation-method, the local declared variables are not found. I've a variable which I set to an IOrderedEnumerable:
Code:
var x = entity.Details.Where(e=> e.y == z).OrderBy(e => e.DateCreated);


I set my breakpoint just before this line, then step through this line, and try to inspect var x.
- the locals-windows doesn't show my variable
- QuickWatch says: the name 'x' does not exist in the current context

Same result for quite simple assignments:
Code:
            var a = 1;
            int b = 2;



Running VS2019 (16.3.10)
Testproject = netcoreapp2.1
implementationproject = net462;netcoreapp2.1
pdb = Portable (while Full gives the same result)
Back to top
 |  Edit by user
Remco
#18 Posted : Thursday, November 28, 2019 12:30:07 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 930 times
Was thanked: 1257 time(s) in 1170 post(s)
Thanks for sharing this problem.

Do you have a way to run this code without NCrunch, and if so, can you confirm 100% whether the behaviour is different under NCrunch?

I've tried constructing a code sample using the lines you've shared, but I can't seem to get this to fail in the same way ... is there any chance you can reproduce this in a code sample you can share with me?
Back to top
Advisto
#21 Posted : Thursday, November 28, 2019 3:50:23 PM(UTC)
Rank: Member

Groups: Registered
Joined: 10/15/2018(UTC)
Posts: 11
Location: Netherlands

Was thanked: 2 time(s) in 2 post(s)
Currently I don't get it reproduced in a sample. I tried about half an hour, and can't spend more time on it now.
But in my original code it seems to bo only in async methods (with some EF-Core linq queries in it). When I debug the same code from TestExplorer it works fine.
Back to top
EsbjornRedmo
#22 Posted : Friday, November 29, 2019 2:18:37 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 8/17/2017(UTC)
Posts: 3
Location: United Kingdom

Thanks: 3 times
Was thanked: 1 time(s) in 1 post(s)
I had the same issue and found that the combination of 1. old style csproj file & .net framework, 2. ncrunch optimised mode and 3. async unit test (doesn't matter whether nunit or xunit) reliably reproduces in on VS2019 with the new 4.2.2 update. Had the same issue with .net core before but that seems to have been fixed with 4.2.2 (thanks!)
Back to top
Users browsing this topic
Guest
2 Pages12>
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.120 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download