Debugger visualizers not working anymore after update to 3.22.1

Hi,

I updated to 3.22.1 and now debugger visualizations aren't working anymore. They do work when e.g. using VS test runner to debug a test, so I assume it's not an obvious VS issue. I nevertheless ran a VS "Repair" installation, which didn't help either. All test were running fine with the last version I had installed (I think it was 3.18, not 100% sure anymore). It is also not happening for each test project in the solution, only a few. I compared csproj files of "working test project" and "not working test project" but could not spot any relevant difference. All of those test projects are on .NET 4.7.2 and most of the projects under test are netstandard2.0.
I tried to downgrad to 3.18 and 3.21, but had the same issue with those versions as well.

Environment:
VS 2017 15.9.3
NCrunch 3.22.1 (also tried 3.21)
OzCode 3.1.0.3913

I can provide further details and/or logs, just tell me what you need.

Thanks
Florian

Hi, thanks for sharing this issue.

There haven't been any changes to NCrunch's debugger integration or anything PDB related for a good many versions now, so this is probably an unrelated breakage.

When you talk about debug visualizations, what do you mean? Is this the analysis of variable tooltips?

Do the intermediate and watch windows work correctly?

Does this seem to only happen when running tests with NCrunch, or does it also happen when you run your software directly?

Hi, thanks for your quick reply.

Talking about debug visualizations I mean the variable tooltips that you get when hovering the mouse over your code.

Adding a quickwatch or a watch for a variable results in "error CS0103: The name 'variableNameHere' does not exist in the current context" being displayed as its value, so those aren't working as well.

When I run those tests with VS test runner, debugger tooltips / watch windows, ... work as expected. As I described in my initial post, this is not happening in all test projects. Some of them work pretty fine debugging them with Ncrunch.

I am happy to provide more information if you require.

Florian

Hi Florian,

Have you noticed that there are certain regions of code that work within the file, but other regions that don't?

It may be something to do with the structure of the PDB file that has been rebuilt by NCrunch.

Which version of .NET are you using in the projects that are failing?

Do breakpoints work correctly within the affected areas?

Hi Remco,

.NET version of test projects is .NET 4.7.2, projects referenced by test project are either 4.7.2 or netstandard2.0.

I tried all tests within the class and none of those showed the tooltips during debugging. I tried another test class from the same project and there it worked. I then added a Dummy-Test to the offending class and it worked for the dummy test.
The dummy test looked like this:

[Fact]
public void Dummy()
{
var id = Guid.NewGuid();
var other = id;
}

What's interesting is, that when I use NCrunch "Break into test at this line" feature on the closing '}', it sets the breakpoint to the line beginning with 'var id...'. On all other lines, it works as expected and sets the breakpoint at the expected line.
I tried this for the other methods and some seem to correctly set the breakpoint, while others differ by a few lines of code.
I also switched NCrunch workspace directory from my ramdrive to a local drive, to make sure files aren't corrupted by the underlying storage. I had the same effect after switching workspace base path.

There is a known problem in the current NCrunch build where the 'Break into test at this line' feature isn't always mapping its target line correctly for files that have just been edited. This will be fixed in the next version and it isn't related to the debug issues.

The problem as you've described it looks like it's originating from the debugger's inability to load variable-based symbols from the PDB file. Where we've seen this before, it's been caused by changes made to the PDB structure to deal with compiler-generated code, such as 'async' and 'yield' structures. Do the broken areas of your test project have anything in common in regards to use of compiler generated code or new C# features? If you're able to produce a sample solution that can reproduce this problem, I should be able to analyse it and maybe produce a fix.

Yes, the broken areas of the test class are all async methods. The dummy test I added stops working, if I turn it into an async method returning a task and adding a Task.Delay(100) to the method body. I tried for 2 hours to create a repro, but I am unable to reproduce the phenomen in the repro project. The strange thing is, that in my test projects it is also only affecting a single class. Other test classes within the test project (even async test methods) work fine. I am not sure, how to come up with a repro for this strange issue. I can't share all the code with you, but we could either setup a remote session, so you can have a look or I could maybe share the VS generated PDB and the NCrunch generated PDB. Not sure, what would actually help you.
Maybe there is also another way. E.g. could I force NCrunch to use the VS generated PDB (to verify its working with that one) or somehow make the VS and NCrunch PDB comparable for you?

Unfortunately, there isn't really any way for me to analyse this issue without having a sample solution that can produce it. This is one of those areas where debugging over someone's shoulder can only yield very limited information, because the problem is actually manifesting in MS's debugger, which is very much a black box to me. Troubleshooting these issues involves very extensive deduction and guesswork to try and establish what the debugger is expecting to find in the PDB file and why it isn't being provided (there are no error messages in this area).

The good news is that in a few months time, the issue may actually be resolved as a result of other development. Right now we're extensively reworking the area of NCrunch that reconstructs the PDB files, mostly because we need to add support for the new portable PDB format that was introduced with .NET Core. The portable PDB format is much cleaner and is an open standard, whereas the legacy format being used right now is an ancient piece of machinery that the world is trying very hard to move away from. It's my hope that when the new portable PDB system is in place, you can simply switch over to it at the problem will be gone. Of course, this speculative and it will probably be 2-4 months before we have anything available in this area.

Ok, I understand. I will try again over the weekend to create a sample that reproduces the issue. If that doesn't work, I will have to workaround the issue until the version arrives, that uses the new pdb format. Thanks for your help so far.

another vote for getting this resolved. it is maddening to not be able to see variable values for anything when debugging.

Upvote for a fix. Some details - NCrunch 3.22.0.1, .NET 4.7.2, PostSharp 6.0.35, VS 15.9.7. When a debug session is started on some tests (480 tests, 2 problems so far) from NCrunch, locals are not available in the debugger window. Running the test from VS Test Explorer does not have same issue. So, yeah I have a workaround but :( Also, you can have access to the Azure project if you need a code base to work with.

Thanks. We've been making progress on the new PDB system. It's still at best a couple of months away, but I'm becoming more convinced that it will solve problems like this.

Hello.
It's an important problem and a fix would be really appreciated.
Take your time of course ;)

Hi @Remco,
in a solution containing only "netstandard2.0" and "netcoreapp2.0" projects, I can debug the tests with NCrunch, but locals are not recognized with CS0103 in watch window (locals work when debugging with Test Explorer). From what I read here it seems like it relates to the issue with the new PDB format.

Is there any update on when support for the new PDB format is available?

Visual Studio 2019 16.0.4
NCrunch 3.27.0.3

Hi, sorry the new CIL based system is still a work in progress. I'd like to give an ETA, but the truth is that with the platform going the way it has been, we're not really in charge of our own schedule at the moment. I can promise you that we're working on it as a matter of priority.

Would you be interested in trying the build below to see if this solves the problem for you?

NCrunch_Console_3.28.0.4.msi
NCrunch_Console_3.28.0.4.zip
NCrunch_GridNodeServer_3.28.0.4.msi
NCrunch_GridNodeServer_3.28.0.4.zip
NCrunch_LicenseServer_3.28.0.4.zip
NCrunch_VS2008_3.28.0.4.msi
NCrunch_VS2010_3.28.0.4.msi
NCrunch_VS2010_3.28.0.4.zip
NCrunch_VS2012_3.28.0.4.msi
NCrunch_VS2012_3.28.0.4.zip
NCrunch_VS2013_3.28.0.4.msi
NCrunch_VS2013_3.28.0.4.zip
NCrunch_VS2015_3.28.0.4.msi
NCrunch_VS2015_3.28.0.4.msi.7z
NCrunch_VS2015_3.28.0.4.zip
NCrunch_VS2017_3.28.0.4.msi
NCrunch_VS2017_3.28.0.4.msi.7z
NCrunch_VS2017_3.28.0.4.zip
NCrunch_VS2019_3.28.0.4.msi
NCrunch_VS2019_3.28.0.4.msi.7z
NCrunch_VS2019_3.28.0.4.zip

Hello again and thanks for the feedback.
I have tried version 3.28 and it seems that debugging is working again(on vs 2017).
Thanks!

tasoss wrote:Hello again and thanks for the feedback.
I have tried version 3.28 and it seems that debugging is working again(on vs 2017).
Thanks!

Awesome, thanks for confirming!

I would like to ask you if it's normal ,that while debugging, i step into nunit's(using step over (F10)) framework code?
Can i somehow disable it?
Thanks!

tasoss wrote:I would like to ask you if it's normal ,that while debugging, i step into nunit's(using step over (F10)) framework code?
Can i somehow disable it?
Thanks!

Hi! This isn't controlled by NCrunch. It's possible you have an IDE/debugger setting turned on that is doing this. Perhaps check the 'Just My Code' debugging setting in VS?