Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Out of memory exception during build
Mangust
#1 Posted : Tuesday, February 5, 2013 4:56:12 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/16/2012(UTC)
Posts: 25
Location: Norway

Was thanked: 4 time(s) in 4 post(s)
I have found an error during the build process when the project is big. I have an open source 3d party dll with a lot of code (NCrunch says it's 2mln lines of code). I know it's not very good but it's 3d party. So when I do the build NCrunch fails with the out of memory exception. I have done some investigations and found that the build itself is OK but the ncrunchhost.x86 fails after reaching 2gb virtual memory limit after the csc.exe is closed. I guess it's doing analysis of the code in this phase. I went to project configuration and turned off instrument assembly (set to false). Then the build was completed. I don't know if NCrunch has some memory leaks or it just needs a lot of memory for that kind of project but may be it's worth to make these hosts x64 so that they can use more memory?
Remco
#2 Posted : Tuesday, February 5, 2013 8:31:30 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
As part of its instrumentation, NCrunch needs to load and dissect the built assembly in memory in order to manipulate it. On large assemblies, this can be a memory intensive process.

Is this project being included in your solution as a project dependency, or as a referenced DLL?

To ensure compatibility with all environments, NCrunch always uses an x86 process for building and instrumenting projects. I'll make a note to test the reliability of doing this under x64, as it may perform better this way anyway.
Mangust
#3 Posted : Wednesday, February 6, 2013 8:28:41 AM(UTC)
Rank: Member

Groups: Registered
Joined: 11/16/2012(UTC)
Posts: 25
Location: Norway

Was thanked: 4 time(s) in 4 post(s)
I have it as a project dependency as a few changes are made to the source. May be you can implement at least an option to switch between x86/x64? So that it's possible to work with large projects as well.
Do you need the project to check the memory consumption? Perhaps there is something special apart from the large amount of code that makes it die with out of memory?
Remco
#4 Posted : Wednesday, February 6, 2013 9:17:01 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
A single assembly file with 2M lines of code is fairly unconventional, so I think it's quite a safe bet that the sheer size of the project is causing the memory problems. The best workaround I can suggest is basically what you've already done - setting the 'Instrument output assembly' setting to 'False' will prevent NCrunch from processing the assembly for code coverage. As you've mentioned that this is a 3rd party component, my assumption is that you probably don't spend the bulk of your time working in this code and the code coverage is probably more or a nice to have.

I also hold a concern that the size of this project will likely cause performance problems for NCrunch even if we are able to solve the out of memory problems. If you have any opportunity to split up the project, I would definitely recommend doing so.

Meanwhile, I've made a note in the backlog to investigate adding an x86/x64 configuration option for the build processes.
Mangust
#5 Posted : Wednesday, February 6, 2013 12:06:28 PM(UTC)
Rank: Member

Groups: Registered
Joined: 11/16/2012(UTC)
Posts: 25
Location: Norway

Was thanked: 4 time(s) in 4 post(s)
I don't really need code coverage as you say for that project so the solution works pretty well for me and it's not any critical.
May be a small suggestion can be to have a hint on doing that (instrument output assembly = false) somewhere when there is an out of memory exception saying that project is perhaps too big and you can trying doing this.
Remco
#6 Posted : Wednesday, February 6, 2013 8:58:16 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
Good thinking :) I'll look into it!
Remco
#7 Posted : Saturday, March 30, 2013 9:54:17 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,161

Thanks: 964 times
Was thanked: 1296 time(s) in 1202 post(s)
For anyone interested, a new configuration option has been introduced with the newly released version of NCrunch (v1.45) which can control the bitness of the build process (x86/x64) - Build Process CPU Architecture.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.037 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download