Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Huge Memory usage
Der-Albert.com
#1 Posted : Friday, June 22, 2012 9:37:55 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/17/2011(UTC)
Posts: 209

Thanks: 11 times
Was thanked: 53 time(s) in 50 post(s)
Hi,

1.40 and 1.39 are huge in memory consumption.

After a couple of hours all runners (4) are using each about 1GB RAM and more. Which leads to "not" enough memory in
my 8 GB System.

Runner is mostly MSpec, some NUnit Test.

Best Regards
Remco
#2 Posted : Monday, June 25, 2012 12:38:20 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Hi Albert,

Thanks for reporting this. Can you give me any more details about which .EXEs are giving high memory usage? Is it the TestHost processes or the BuildHost ones?

Does the memory usage spike quickly after a full test run, or does it slowly increase incrementally?

Also - did you notice this problem with any earlier versions of NCrunch?
Rammesses
#3 Posted : Wednesday, July 4, 2012 11:09:00 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 7/4/2012(UTC)
Posts: 2
Location: Reading, UK

I get the same issue with NCrunch for MSTest tests....

NCrunch Memory Usage

It's a real issue that's stopping general adoption at work - the 4Gb of memory in our workstations basically precludes us doing so - which is a shame, because it's a real help for a smooth workflow for developing software of the level of quality that we want.
Remco
#4 Posted : Thursday, July 5, 2012 1:23:43 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Hi Rammesses,

Thanks for posting! Looking at the EXE names of the task runners, it looks as though you aren't making use of the new hosted task runners. Are you running the latest version of NCrunch? 1.40b includes some fixes specifically focused on reducing memory utilisation.
Rammesses
#5 Posted : Thursday, July 5, 2012 7:55:59 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 7/4/2012(UTC)
Posts: 2
Location: Reading, UK

Hi there,

I found v1.40b about 5 minutes after posting. :$

It's a LOT better - to the point of making NCrunch usable (I've already converted one colleague!), but still CAN generate quite high memory usage, particularly during the compilation phase.

From my (initial) testing, the test runners are taking about 60Mb (ish) each, whilst the build runners vary wildly up to 600Mb+ but then seem to settle back after - this if for the same solution as before, with an equivalent NCrunch configuration. I'm going to keep an eye on it tho'

It's not stopping me getting more memory from our Group IT Services!

Cheers
Remco
#6 Posted : Friday, July 6, 2012 3:49:09 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Great to hear!

I have heard reports that the build runners tend to progressively increase their memory utilisation over time on some solutions. If this happens to you, just kill the BuildHost EXEs and they should recycle and carry on. I have a task logged to look into what is causing the leak in this process.


Cheers,

Remco
darrenkopp
#7 Posted : Wednesday, July 25, 2012 8:40:56 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 7/25/2012(UTC)
Posts: 2

Yeah, BuildHost EXE's grow over time because there is some memory that isn't getting released by what looks like MSBuild functionality. it's basically 12 types that account for the majority of the memory, and 90% of those types are xml. Not sure what you are doing, but perhaps if it's something like a managed msbuild extension that is leaking the info due to it not being disposed, or something like that.

Here's an image of the memory dump I took, and as you can see at the bottom, most of the memory in the system is being used by those bottom 12 types, with almost everything else being very low memory usage.

http://i.imgur.com/d2Nl8.png

If you are unfamiliar with windbg, the first column is method table (basically the id for that type), second column is total # of instances, third column is total amount of memory those instances use in bytes, and fourth is of course the type.
Remco
#8 Posted : Thursday, July 26, 2012 12:06:42 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Nice analysis! Thanks for the memory dump. This does help to confirm my suspicions that the leaks are being caused internally in MSBuild. Whether or not NCrunch's manner of use of MSBuild is contributing to this is an interesting question and I'll continue looking into this. The upcoming 1.41b release contains a fix that should periodically cycle the build processes to keep the memory usage in check - which I guess could be considered the ultimate cure.
Remco
#9 Posted : Monday, August 13, 2012 4:20:52 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
For anyone interested, NCrunch 1.41b has just been released including a feature designed to limit the memory consumption by build processes. This should effectively solve the build process memory leaks described above, as NCrunch will automatically recycle these processes if their working memory becomes too large.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.055 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download