Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Memory Leak in Large Project
Luciferius
#1 Posted : Tuesday, September 27, 2016 8:11:07 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/28/2013(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
I have the following setup:
Visual Studio 2015 Update 3
Solution Framework: 4.6.1
NCrunch Version: 2.23.0.2
Projects Count: 198

The problem is, as soon as I enable NCrunch the engine host process has a memory consumption of 10GB growing. It seems it takes as much memory as available. After some time it even seems to start using the hard drive.

I know that all assemblies are kept in memory. But more than 10GB seems a bit much. Additionally this slows my computer so much that I cannot use NCrunch currently.
Remco
#2 Posted : Tuesday, September 27, 2016 8:19:30 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 4,157

Thanks: 549 times
Was thanked: 606 time(s) in 586 post(s)
Hi, thanks for sharing this issue.

10GB is rather extreme, I haven't heard of the engine reaching so high before.

Can you give me any more statistics on on your solution in terms of code count, number of tests, and code coverage density?

Try deleting the NCrunch cache file to see if this resets the consumption. I'm curious as to whether it will return to 10GB after a full run through of your test suite.

The bulk of NCrunch's memory consumption is normally in its coverage index. Each individual line of covered code by every covered test must be recorded in this index. So if you have 1,000,000 lines of code, with each line covered by 10,000 tests, that will be 1,000,000 * 10,000 index entries. Are you able to construct any kind of rough estimate as to how many covered lines of code (by test) you have in your solution?
Luciferius
#3 Posted : Wednesday, September 28, 2016 12:07:17 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/28/2013(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
Deleting the cache solved the issue. Unfortunately I did not save the cache previously. Therefore I cannot reproduce the issue anymore. When it happens again, I can write you again.

As for the statistics:
Lines of code: 700,000
Tests: 10,300
Remco
#4 Posted : Wednesday, September 28, 2016 12:12:00 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 4,157

Thanks: 549 times
Was thanked: 606 time(s) in 586 post(s)
Interesting. It's definitely a leak then, not simply high memory consumption.

Let me know if you find a way to reproduce. It would be great if you could also keep an eye on your overall memory consumption across sessions to see if this increases over time. I'm wondering if somehow stale coverage data is accumulating across sessions.

Did you perhaps have a chance to see the size of the cache file before you deleted it?
Luciferius
#5 Posted : Friday, October 07, 2016 8:53:07 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/28/2013(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
I can now reproduce it. It seemed, as if the memory consumption grew from day to day. Currently it was again near 10GB. I have now a backup of the cache. How can I help analyse the leak?
The size of the cache file is 650 mb.
Remco
#6 Posted : Friday, October 07, 2016 10:38:21 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 4,157

Thanks: 549 times
Was thanked: 606 time(s) in 586 post(s)
Luciferius;9255 wrote:
I can now reproduce it. It seemed, as if the memory consumption grew from day to day. Currently it was again near 10GB. I have now a backup of the cache. How can I help analyse the leak?
The size of the cache file is 650 mb.


Ok, that's good to know. Unfortunately, analysing this will be quite hard unless you're able to share your source code (which I don't usually expect). The cache file itself does actually contain quite a bit of your source code. It uses this to accurately preserve coverage data between sessions.

In your daily use of the product, do you very frequently do any of the following?

- Move projects around on disk
- Rename namespaces or tests
- Ignore/unignore large numbers of tests
- Add or remove large numbers of tests from the solution/projects
- Perform large scale refactoring on your codebase (esp renames on classes that are heavily covered)
Luciferius
#7 Posted : Friday, October 07, 2016 10:58:14 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/28/2013(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
We have quite a multitude of changes per day from all developers, which I get the next morning with an svn update. I have no actual numbers, but I believe this could be around 3000 lines of code changed/added/removed per day. I usually do not ignore tests but unload unnecessary test projects. I have done this once since the last deletion of the cache - around 4 project with at most 10 tests per project. Locally I do smaller refactorings from day to day. I could try to monitor the memory consumption from day to day and look for peaks or if it is a general trend.
Remco
#8 Posted : Friday, October 07, 2016 11:16:41 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 4,157

Thanks: 549 times
Was thanked: 606 time(s) in 586 post(s)
An interesting test would be for you to unload and reload your test project a few times, and see if this makes a difference in the size of the cache file.

Do you often update from SVN with the NCrunch engine running? Or do you usually disable it when you update?
Luciferius
#9 Posted : Friday, October 07, 2016 11:19:03 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/28/2013(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
I do the update always wihtout the engine running. I will check the loading and unloading of projects.
1 user thanked Luciferius for this useful post.
Remco on 10/7/2016(UTC)
Luciferius
#10 Posted : Thursday, December 22, 2016 1:31:07 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/28/2013(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
After some more observation it seems, that loading and unloading project has an impact on the growth rate of the memory consumption. I started to use the extension Funnel, where you can switch between different loaded solution configurations. This seem to speed up the allocation of memory faster, than it did before.
Remco
#11 Posted : Thursday, December 22, 2016 9:41:01 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 4,157

Thanks: 549 times
Was thanked: 606 time(s) in 586 post(s)
Luciferius;9631 wrote:
After some more observation it seems, that loading and unloading project has an impact on the growth rate of the memory consumption. I started to use the extension Funnel, where you can switch between different loaded solution configurations. This seem to speed up the allocation of memory faster, than it did before.


Thanks for confirming this. Does this mean that you can actually drive the memory up into infinity by unloading and reloading projects using funnel?
Luciferius
#12 Posted : Friday, December 23, 2016 6:23:13 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/28/2013(UTC)
Posts: 22

Thanks: 2 times
Was thanked: 4 time(s) in 4 post(s)
I think it is by unloading and reloading projects, only with funnel this happens a lot faster, as I unload / reload more often.
1 user thanked Luciferius for this useful post.
Remco on 12/23/2016(UTC)
Users browsing this topic
Guest (3)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.055 seconds.