Build/Test Issues

NCrunch 4 - Optimized mode - Debugging tests is mission impossible?

Started by Advisto on 7,709 views

In NCrunch 4 with VS 2019, while using the Optimized mode, it seems te be nearly impossible to debug my tests. The debugger starts ok, and breaks on the breakpoint I set. But...
[list=1]
  • I'm unable to watch my variables (parameters of the method are possible to watch, but variables instantiated within the method-with-breakpoint can't be watched).
  • I'm can't step into another method, gives me a 'sources not available' page


  • After setting the mode to legacy debugging works fine.

    Any others experienced this issue?

    Hi, thanks for sharing this issue.

    Can you confirm if your code is using portable PDBs or Win32 ones? Do you notice any difference if you switch between these? Note that you can control this under your advanced build settings in the VS project configuration.
    I have the same issue concerning debugging: in legacy mode debugging works, in optimized mode only half-way.
    Breakpoints are hit but if I try to step into an method, at first I get the "Source Not Available" window and then after another "Step Into" key hit it behaves just like "Step Over" instead of "Step Into" (= staying in the original method and moving to next step instead).

    NCrunch 4.1.0.1
    VS: 16.3.9
    Test project: netcoreapp2.2 with Microsoft.NET.Test.Sdk 16.2.0, MSTest.TestAdapter 1.4.0, MSTest.TestFramework 1.4.0
    Tested project: netstandard2.0

    Edited

    I have issues too. Value evaluation dosn't work in Optimised mode.

    Remco wrote:Hi, thanks for sharing this issue.

    Can you confirm if your code is using portable PDBs or Win32 ones? Do you notice any difference if you switch between these? Note that you can control this under your advanced build settings in the VS project configuration.

    My Code uses Portable pdbs. Which one is win32? i have only None, Full, Pdb-only, Portable and Embeded Items in Debugging information combobox.
    spolonski wrote:
    My Code uses Portable pdbs. Which one is win32? i have only None, Full, Pdb-only, Portable and Embeded Items in Debugging information combobox.


    Win32 pdb is 'Full'.


    Thanks for sharing these extra details. I'm going to do some experimentation to see if I can get this to appear on my end. I hope I won't need to ask for a sample to reproduce the issue ... I guess we'll see how we go.

    Edited

    I've managed to identify some issues around local variable scoping inside the portable PDBs that we're rebuilding. I'll let you know as soon as a fix is available.
    I have had similar issues with watches / variable or expression evaluation using NCrunch Optimised Mode in Visual Studio 2019 for a .NET 4.7.2 project using Win32 (Full) PDBs. I also noticed that stepping into or out of methods now shows a bit of stepping through external code (instrumentation, if I'm not mistaken). When I switch back to Legacy, these issues disappear.
    tomtrias wrote:I have had similar issues with watches / variable or expression evaluation using NCrunch Optimised Mode in Visual Studio 2019 for a .NET 4.7.2 project using Win32 (Full) PDBs. I also noticed that stepping into or out of methods now shows a bit of stepping through external code (instrumentation, if I'm not mistaken). When I switch back to Legacy, these issues disappear.


    I have a theory about what is causing the external code stepping, though I had hoped the local variable inspection for Full PDBs was OK. Are you able to narrow this to a specific case that you can share with me? This would be a separate problem from the other issues reported in this thread.
    I will narrow down a small reproductive case when I get a chance; hopefully it is fairly simple to do. The code base I am currently working on is large and complicated.
    tomtrias wrote:I will narrow down a small reproductive case when I get a chance; hopefully it is fairly simple to do. The code base I am currently working on is large and complicated.


    Thanks for your help!
    Remco wrote:Hi, thanks for sharing this issue.

    Can you confirm if your code is using portable PDBs or Win32 ones? Do you notice any difference if you switch between these? Note that you can control this under your advanced build settings in the VS project configuration.

    Sorry for my late reply, it's a netcoreapp2.1 project with portable PDB's. Just tested with Full (win32), which seem a bit better (I can inspect some more variables it seems) but 'step into' still doesn't work (while it works when I set a breakpoint into that method and hit F5).

    Hope it helps, and I'm looking forward to the update because I really love the performance!
    Advisto wrote:
    Sorry for my late reply, it's a netcoreapp2.1 project with portable PDB's. Just tested with Full (win32), which seem a bit better (I can inspect some more variables it seems) but 'step into' still doesn't work (while it works when I set a breakpoint into that method and hit F5).


    Thanks! This confirms it's being caused by the two problems we've pinned down. A fix is incoming for these issues soon.
    This is pretty weird; I am getting some really strange behavior inspecting variables when I have a situation like the following (even in Legacy mode):

    foreach(var item in collection.Where(item => item.BooleanProperty)) {
    ...
    }
    foreach(var item in collection) {
    ...
    }

    If I inspect "item" in the second "foreach", I am seeing the value for "item" from the previous "foreach" scope. I have verified that using other unit test runners to debug does not have this issue.
    tomtrias wrote:
    If I inspect "item" in the second "foreach", I am seeing the value for "item" from the previous "foreach" scope. I have verified that using other unit test runners to debug does not have this issue.


    Thanks for sharing this. This problem is an old existing one caused by the improper scoping of variables. It's existed in NCrunch since V1, and it's one of the problems fixed in the new optimised mode.

    It just so happens that at the moment, the optimised mode has some other more serious problems with debugging, but there will be build published very soon with fixes for these.
    The build below includes fixes for all known debug problems with NCrunch V4 under optimised mode. Can everyone with issues here please give this a try and let me know if all is well?

    NCrunch_Console_4.2.0.2.msi
    NCrunch_Console_4.2.0.2.zip
    NCrunch_GridNodeServer_4.2.0.2.msi
    NCrunch_GridNodeServer_4.2.0.2.zip
    NCrunch_LicenseServer_4.2.0.2.zip
    NCrunch_VS2008_4.2.0.2.msi
    NCrunch_VS2010_4.2.0.2.msi
    NCrunch_VS2010_4.2.0.2.zip
    NCrunch_VS2012_4.2.0.2.msi
    NCrunch_VS2012_4.2.0.2.zip
    NCrunch_VS2013_4.2.0.2.msi
    NCrunch_VS2013_4.2.0.2.zip
    NCrunch_VS2015_4.2.0.2.msi
    NCrunch_VS2015_4.2.0.2.msi.7z
    NCrunch_VS2015_4.2.0.2.zip
    NCrunch_VS2017_4.2.0.2.msi
    NCrunch_VS2017_4.2.0.2.msi.7z
    NCrunch_VS2017_4.2.0.2.zip
    NCrunch_VS2019_4.2.0.2.msi
    NCrunch_VS2019_4.2.0.2.msi.7z
    NCrunch_VS2019_4.2.0.2.zip
    Remco wrote:The build below includes fixes for all known debug problems with NCrunch V4 under optimised mode. Can everyone with issues here please give this a try and let me know if all is well?

    NCrunch_Console_4.2.0.2.msi
    NCrunch_Console_4.2.0.2.zip
    NCrunch_GridNodeServer_4.2.0.2.msi
    NCrunch_GridNodeServer_4.2.0.2.zip
    NCrunch_LicenseServer_4.2.0.2.zip
    NCrunch_VS2008_4.2.0.2.msi
    NCrunch_VS2010_4.2.0.2.msi
    NCrunch_VS2010_4.2.0.2.zip
    NCrunch_VS2012_4.2.0.2.msi
    NCrunch_VS2012_4.2.0.2.zip
    NCrunch_VS2013_4.2.0.2.msi
    NCrunch_VS2013_4.2.0.2.zip
    NCrunch_VS2015_4.2.0.2.msi
    NCrunch_VS2015_4.2.0.2.msi.7z
    NCrunch_VS2015_4.2.0.2.zip
    NCrunch_VS2017_4.2.0.2.msi
    NCrunch_VS2017_4.2.0.2.msi.7z
    NCrunch_VS2017_4.2.0.2.zip
    NCrunch_VS2019_4.2.0.2.msi
    NCrunch_VS2019_4.2.0.2.msi.7z
    NCrunch_VS2019_4.2.0.2.zip


    This version works for me now in optimized mode ("Step Into" and inspecting variables)!
    Thanks for the fix!

    VS: 16.3.9
    Test project: netcoreapp2.2 with Microsoft.NET.Test.Sdk 16.2.0, MSTest.TestAdapter 1.4.0, MSTest.TestFramework 1.4.0
    Tested project: netstandard2.0

    Edited

    Remco wrote:The build below includes fixes for all known debug problems with NCrunch V4 under optimised mode. Can everyone with issues here please give this a try and let me know if all is well?


    Hi Remco,

    When I set a breakpoint in my implementation-method, the local declared variables are not found. I've a variable which I set to an IOrderedEnumerable:
    var x = entity.Details.Where(e=> e.y == z).OrderBy(e => e.DateCreated);


    I set my breakpoint just before this line, then step through this line, and try to inspect var x.
    - the locals-windows doesn't show my variable
    - QuickWatch says: the name 'x' does not exist in the current context

    Same result for quite simple assignments:
                var a = 1;
                int b = 2;
    



    Running VS2019 (16.3.10)
    Testproject = netcoreapp2.1
    implementationproject = net462;netcoreapp2.1
    pdb = Portable (while Full gives the same result)

    Edited

    Thanks for sharing this problem.

    Do you have a way to run this code without NCrunch, and if so, can you confirm 100% whether the behaviour is different under NCrunch?

    I've tried constructing a code sample using the lines you've shared, but I can't seem to get this to fail in the same way ... is there any chance you can reproduce this in a code sample you can share with me?
    Currently I don't get it reproduced in a sample. I tried about half an hour, and can't spend more time on it now.
    But in my original code it seems to bo only in async methods (with some EF-Core linq queries in it). When I debug the same code from TestExplorer it works fine.
    I had the same issue and found that the combination of 1. old style csproj file & .net framework, 2. ncrunch optimised mode and 3. async unit test (doesn't matter whether nunit or xunit) reliably reproduces in on VS2019 with the new 4.2.2 update. Had the same issue with .net core before but that seems to have been fixed with 4.2.2 (thanks!)

    Post a reply

    Log in to reply.