slow typing in VS2015

Hi,

In VS2015 if I keep ncrunch enabled and start modifying the code(especially renaming using F2), the UI freezes for some time (500 ms to 1 sec). If I disable ncrunch altogether, the IDE becomes very responsive again. If I keep ncrunch enabled, but ignore all my tests, the problem still happens. Another thing to note, the problem does not happen when I modify the unit test codes, it only happens with the modification of production code.

Abu

Hi Abu, thanks for sharing this issue.

Would you be able to help with answering the following questions? This should help with narrowing down the issue:

- What are the specifications of the machine you're running on? Is it a high-end system?
- Approximately how large is the solution you're working in? (in terms of lines of code and number of tests)
- Have you used NCrunch before on other versions of Visual Studio with the same solution? If so, did you have the same problem?
- Do you experience this problem when working with other solutions?
- What do you have your 'Engine hosting strategy' (NCrunch global config setting) set to?

I am able to reproduce this. I get this in Test as well as Production code. This occurs in Windows 8.1 and Windows 10 running Visual Studio 2015 with NCrunch. I have this set-up on various machines:

- What are the specifications of the machine you're running on? Is it a high-end system?
-- Intel Core i5 760 @ 2.8GHz, 16Gb RAM, Windows 10, VS2015 Pro
-- Intel Core i7 (don't have the specs to hand), 12Gb RAM, Windows 8.1, VS2015 Pro
-- Windows Azure D12 VM instance (4 cores, 28Gb RAM), Windows 10, VS2015 Pro

- Approximately how large is the solution you're working in? (in terms of lines of code and number of tests)
~19K lines of code, 2.5K tests

- Have you used NCrunch before on other versions of Visual Studio with the same solution? If so, did you have the same problem?
NCrunch works fine with same solution under VS2013 Pro on Windows 8.1, no problems

- Do you experience this problem when working with other solutions?
Not tried with other solutions, will try and comment back

- What do you have your 'Engine hosting strategy' (NCrunch global config setting) set to?
x64SatelliteProcess

Thanks for sharing these details.

Can you try adjusting the Sliding build delay configuration setting to see if this makes any difference?

With the issue appearing on such a wide range of different systems, I wonder if it may be solution-specific. I'm very interested to hear how you go with other solutions if you can find any of a comparable size.

Would you also be able to examine Task Manager (or an equivalent resource monitor) to see what happens with your CPU when you try typing in VS? Are there certain cores that are becoming overutilised? Is the whole CPU maxing out?

Something else that might also be worth trying is to set the 'Ignore this project completely' progressively through the projects in your solution, starting with the topmost in the dependency tree. If there is a specific project causing the issue, this may help to isolate it.

I have tried varying the Sliding build delay with values ranging from 500 to 10,000: this had no affect.

I have tried two other solutions: an ASP.NET MVC app and another Windows 8.1 app (the solution with 19K lines, 2.5K tests is a Windows 8.1 Universal app). Both those solutions (which are tiny, tiny solutions) worked fine, no delay. I'm happy to set up a Skype call to demo this and try some diagnostics to help you to identify the cause of the issue.

One other minor difference between the slow solution and the other two that work fine: the slow solution is using MSTest (not my choice!) and the other two are using xUnit.

willcowell wrote:One other minor difference between the slow solution and the other two that work fine: the slow solution is using MSTest (not my choice!) and the other two are using xUnit.

Did you have much luck with narrowing down the issue through ignoring projects, or examining CPU consumption when the issue appears?

willcowell wrote:I have tried varying the Sliding build delay with values ranging from 500 to 10,000: this had no affect.

I have tried two other solutions: an ASP.NET MVC app and another Windows 8.1 app (the solution with 19K lines, 2.5K tests is a Windows 8.1 Universal app). Both those solutions (which are tiny, tiny solutions) worked fine, no delay. I'm happy to set up a Skype call to demo this and try some diagnostics to help you to identify the cause of the issue.

Do you see the same behaviour for a blank Windows 8.1 Universal App without any tests in it?

Remco wrote:

willcowell wrote:I have tried varying the Sliding build delay with values ranging from 500 to 10,000: this had no affect.

I have tried two other solutions: an ASP.NET MVC app and another Windows 8.1 app (the solution with 19K lines, 2.5K tests is a Windows 8.1 Universal app). Both those solutions (which are tiny, tiny solutions) worked fine, no delay. I'm happy to set up a Skype call to demo this and try some diagnostics to help you to identify the cause of the issue.

Do you see the same behaviour for a blank Windows 8.1 Universal App without any tests in it?

To be clear, I only see this issue on the larger Windows 8.1 Universal app project: it does not appear in the small Windows 8.1 Universal app project (that has xUnit tests).

Ok - I'll try and share some more details with you about what I think is happening, in the hope that some of my questions may make more sense to you. I think we need to check a few things further to narrow this down before there is any certainty on what is happening to your system.

NCrunch itself has a very limited presence inside the IDE when the 'Engine hosting strategy' is set to use a satellite process. Basically it's just a synchroniser that pushes changes through IPC into the satellite process to be handled. The mechanism for this is the same for every type of project or solution, and it's very very lightweight. I guess what I'm trying to say here is that I think it's unlikely this problem is directly being caused by NCrunch's VS integration - though it is still quite possible that the issue is being triggered by NCrunch indirectly.

It's known that NCrunch is able to cause poor performance in the IDE through overconsumption of system resources. There are many mechanisms built into the product to try and prevent this, but it can still happen in the following cases:

1. If NCrunch itself is not configured correctly (for example, 'Max number of processing threads' is set too high or the CPU cores are not assigned correctly)
2. If the IDE is attempting to perform excessive blocking I/O activities on a slow disk, NCrunch can block IDE threads by tying up the I/O system with its background processing. There would need to be something wrong with Visual Studio to cause this (a 3rd party package perhaps), as normally VS doesn't heavily block on I/O on foreground threads.
3. If NCrunch is executing tests that tie up system resources in other ways. For example, you may have tests running against an SQL Server Database. SQL Server runs in a process outside of NCrunch's control, so there is no way for NCrunch to stop the database server from overconsuming resources if the tests are particularly taxing. This basically takes CPU away from the IDE and causes poor performance.

All of the above cases can be identified with close attention to system resource utilisation when the problem presents itself. For example, if there is a background process stealing CPU away from VS, this should be visible with around 100% CPU utilisation. If it's I/O that's the problem, you'll likely see a large disk queue.

Finally, there is one more thing that can cause poor performance in the IDE: Memory traffic. It's worth being aware of this - http://blog.ncrunch.net/post/Engine-Separation-and-VS2015-Integration.aspx. It's entirely possible that the poor performance you're seeing is being caused by a combination of VS, NCrunch, and other installed packages all competing for the same GC. NCrunch should do very little to impact this when it is satellite hosted, but it's not impossible that in this situation NCrunch is the last straw that broke the camel's back.

Hi Ramco,

Here are my answers:

- What are the specifications of the machine you're running on? Is it a high-end system?
i7 3770 CPU. 12 GB RAM. Windows 7.

- Approximately how large is the solution you're working in? (in terms of lines of code and number of tests)
solutions is big but not huge, 24 projects. But number of unit tests are minor only 44 tests.

- Have you used NCrunch before on other versions of Visual Studio with the same solution? If so, did you have the same problem?

Yes, it was fine with VS 2013. As soon as I switched to 2015, problem started appearing.

- Do you experience this problem when working with other solutions?
With a brand new solution, I did not see the problem, at least it was not noticeable enough.

- What do you have your 'Engine hosting strategy'
x64SatelliteProcess

And no, the sliding build delay technique did not help much.

Remco wrote:Hi Abu, thanks for sharing this issue.

Would you be able to help with answering the following questions? This should help with narrowing down the issue:

- What are the specifications of the machine you're running on? Is it a high-end system?
- Approximately how large is the solution you're working in? (in terms of lines of code and number of tests)
- Have you used NCrunch before on other versions of Visual Studio with the same solution? If so, did you have the same problem?
- Do you experience this problem when working with other solutions?
- What do you have your 'Engine hosting strategy' (NCrunch global config setting) set to?

Hi,

A few more questions:

- Under Windows Task manager, when the problem appears, do you see high CPU utilisation? How high and which CPUs?
- Under Windows Task manager, when the problem appears, do you see an excessive disk queue (i.e. heavy I/O activity?)
- What happens when you progressively ignore projects in your solution using the 'Ignore this project completely' NCrunch configuration setting? Start with the top-most projects and work your way down. At which stage does the problem disappear?
- What happens when you set the engine to 'Manual Mode'? Does the problem go away?

I think I found the issue. I had two different solutions open in two different VS 2015 instance (they have some common shared projects). If I close one Visual Studio instance, the problem disappears. I believe I should be given my next year subscription renewal for free to help you find the issue :)

Remco wrote:Hi,

A few more questions:

- Under Windows Task manager, when the problem appears, do you see high CPU utilisation? How high and which CPUs?
- Under Windows Task manager, when the problem appears, do you see an excessive disk queue (i.e. heavy I/O activity?)
- What happens when you progressively ignore projects in your solution using the 'Ignore this project completely' NCrunch configuration setting? Start with the top-most projects and work your way down. At which stage does the problem disappear?
- What happens when you set the engine to 'Manual Mode'? Does the problem go away?

Interesting, now I can't repro the issue even after I opened the other solution. I am keeping an eye to find out when it happens again.

mazher wrote:I think I found the issue. I had two different solutions open in two different VS 2015 instance (they have some common shared projects). If I close one Visual Studio instance, the problem disappears. I believe I should be given my next year subscription renewal for free to help you find the issue :)

Remco wrote:Hi,

A few more questions:

- Under Windows Task manager, when the problem appears, do you see high CPU utilisation? How high and which CPUs?
- Under Windows Task manager, when the problem appears, do you see an excessive disk queue (i.e. heavy I/O activity?)
- What happens when you progressively ignore projects in your solution using the 'Ignore this project completely' NCrunch configuration setting? Start with the top-most projects and work your way down. At which stage does the problem disappear?
- What happens when you set the engine to 'Manual Mode'? Does the problem go away?

I think I'm seeing similar effect when I've got two+ instances of Visual Studio 2013 running ...

The Visual Studio UI locks up ... but other applications not affected, and CPU load is minimal

Again fairly good specification machine ... i7, 16GB memory, running of a SSD

Various tests, some of them are "integration tests", calling web services + other potentially slowish services

Cheers,
SteveC.

I have 8 cores. When this happens, the first CPU gets a huge spike, the other CPUs are mostly idle. I can also see spike on the disk usage too. The problem exists even when I use manual mode.

Remco wrote:Hi,

A few more questions:

- Under Windows Task manager, when the problem appears, do you see high CPU utilisation? How high and which CPUs?
- Under Windows Task manager, when the problem appears, do you see an excessive disk queue (i.e. heavy I/O activity?)
- What happens when you progressively ignore projects in your solution using the 'Ignore this project completely' NCrunch configuration setting? Start with the top-most projects and work your way down. At which stage does the problem disappear?
- What happens when you set the engine to 'Manual Mode'? Does the problem go away?

I also believe it has to do something with multiple instances. The other reason for this doubt is I am having another related problem with VS2015+ ncrunch, which never happened with vs 2013 + ncrunch.
Basically, if I have both solutions opened in 2 vs instances (remember, some of the projects are shared), once I make changes in one solution and then run both solution, the one that is last run gives error saying some of the files are locked by another process.

At this point I am really frustrated with the ncrunch and planning to not use it until they fix the issues.

SteveC wrote:I think I'm seeing similar effect when I've got two+ instances of Visual Studio 2013 running ...

The Visual Studio UI locks up ... but other applications not affected, and CPU load is minimal

Again fairly good specification machine ... i7, 16GB memory, running of a SSD

Various tests, some of them are "integration tests", calling web services + other potentially slowish services

Cheers,
SteveC.

I wonder if this issue may be caused by overconsumption of the CPU cores NCrunch has assigned to Visual Studio.

To prevent obstruction of VS, NCrunch always splits your CPU cores between itself and VS. Behind the scenes it assigns 'affinity' for the VS foreground thread to execute only on a specific CPU core. NCrunch activity then executes on other cores to avoid obstructing it. In this way, it's possible for NCrunch to overload your system with background execution without making the UI unresponsive.

A situation where this can cause problems is if you are working with multiple instances of VS, and VS is particularly hungry. VS2015 introduces quite a lot of background VS processing (through Roslyn), and this would only be added to other demands placed on your system by refactoring tools and other VS packages.

The good news is that NCrunch's handling of your CPU cores is entirely configurable. You can either run the configuration wizard and assign more cores to VS, or you can fine tune the core assignments using this configuration setting. To start with, I would suggest assigning most of your cores to VS to see if this makes a difference. Make sure you make this configuration change with only one VS instance open, so you can be sure the configuration won't be overwritten by the other instance. For the changes to take effect, you'll probably need to restart VS.

Can you tell me anything more about the file locking problems you're experiencing? NCrunch goes through huge trouble to avoid locking files in the foreground solution. Do you have the include static references in workspace setting enabled for any of your projects? Is there any pattern as to which files are being locked?

But I have 7 cores assigned to VS and only 1 core for ncrunch.

Remco wrote:I wonder if this issue may be caused by overconsumption of the CPU cores NCrunch has assigned to Visual Studio.

To prevent obstruction of VS, NCrunch always splits your CPU cores between itself and VS. Behind the scenes it assigns 'affinity' for the VS foreground thread to execute only on a specific CPU core. NCrunch activity then executes on other cores to avoid obstructing it. In this way, it's possible for NCrunch to overload your system with background execution without making the UI unresponsive.

A situation where this can cause problems is if you are working with multiple instances of VS, and VS is particularly hungry. VS2015 introduces quite a lot of background VS processing (through Roslyn), and this would only be added to other demands placed on your system by refactoring tools and other VS packages.

The good news is that NCrunch's handling of your CPU cores is entirely configurable. You can either run the configuration wizard and assign more cores to VS, or you can fine tune the core assignments using this configuration setting. To start with, I would suggest assigning most of your cores to VS to see if this makes a difference. Make sure you make this configuration change with only one VS instance open, so you can be sure the configuration won't be overwritten by the other instance. For the changes to take effect, you'll probably need to restart VS.

Can you tell me anything more about the file locking problems you're experiencing? NCrunch goes through huge trouble to avoid locking files in the foreground solution. Do you have the include static references in workspace setting enabled for any of your projects? Is there any pattern as to which files are being locked?

Has there been any follow up on this issue from ncrunch? I'm experiencing the same issue even with the latest release which said it fixed some vs 2015 performance issues. None of the suggested changes in this thread have helped.