Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Hard Drive usage going up
Options
Previous Topic Next Topic
greba
#1 Posted : Monday, February 6, 2012 1:41:13 PM(UTC)
Rank: Member

Groups: Registered
Joined: 1/27/2012(UTC)
Posts: 13

Was thanked: 3 time(s) in 3 post(s)
It looks like NCrunch is not cleaning up old workspaces when a solution is not in use anymore. I have a large solution that when compiled leaves just over 400 Megabytes on the hard drive after I close Visual Studio. Once I restart Visual Studio and open up that same solution NCrunch will chug through the solution as if it's never seen if before and create another copy of the workspace on my machine. After doing this a bunch of times through a week I end up with a large amount of data, 26 Gigs, stored in the C:\Users\%username%\AppData\Local\NCrunch directory that looks like it is just taking up space. I deleted the entire directory and saw no harm come to NCrunch.

Is there a setting to tell NCrunch to clean up old workspaces, or is this a bug that I've run across?
Back to top
Remco
#2 Posted : Monday, February 6, 2012 9:12:44 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,974

Thanks: 929 times
Was thanked: 1256 time(s) in 1169 post(s)
NCrunch will generally clean up workspaces when the process exits, but sometimes this is impossible (due to locks being held on the files by background processes etc). To prevent the drive from getting cluttered, NCrunch will automatically clean up any leftover workspaces when it is restarted and enabled in a new IDE instance. It's worth noting that currently NCrunch will not re-use workspaces between IDE instances - as soon as you enable NCrunch, it will reconstruct a whole new set of workspaces from scratch. NCrunch can also construct multiple workspaces for a single project so that it can run builds in parallel with tests that are already executing.

Because of the behaviour above, on large solutions (such as yours), you may notice highly elevated build times shortly after enabling NCrunch (while it rebuilds the workspaces). I have some plans to improve this via workspace re-use, and I am working on finding a reliable way to implement this.

So anyway, it's possible that with large solutions you can end up with a lot of disk space in use. 26 Gigs seems a bit extreme to me though (considering you have only 400 Meg in the original solution?), so I'm wondering if you can check the following:

- Load up Visual Studio, let NCrunch run through a full pass of your solution
- Open up task manager and note down the process ID of Visual Studio (i.e. 1438 or something)
- Close down Visual Studio
- Browse to your workspaces directory (usually under your local user account, but this is configurable)
- Look directly inside the workspaces directory - at the top level you may see a directory with the same process ID as the Visual Studio you just closed
- If it's not there, that means NCrunch managed to clean everything up properly while the IDE was terminated... You may be able to trigger it to leave things behind if you repeat the above steps but closing VS while NCrunch is partway through a run
- Assuming you managed to get NCrunch to leave data behind in the directory, try loading up a new instance of Visual Studio with the same solution
- Enable NCrunch
- Browse back to the workspace for the original IDE's workspace and see if it's still there

If the workspace is still there, then something has gone really wrong. Otherwise, we may need to have a look a bit more at the structure of your solution - it's possible that you have it configured in a way that makes NCrunch use HDD space very inefficiently.
Back to top
greba
#3 Posted : Tuesday, February 7, 2012 1:51:10 PM(UTC)
Rank: Member

Groups: Registered
Joined: 1/27/2012(UTC)
Posts: 13

Was thanked: 3 time(s) in 3 post(s)
I went through your steps and here are the results. In my large solution which has 58 projects NCrunch will create a folder for each project in it's workspace. After NCrunch is done processing the whole solution and running all the tests the total size of the workspace ends up at 503MB. When I close down Visual Studio the workspace directory still exists but some of the folders are removed. I am seeing 45 of the 58 original folders being left in the workspace. Starting Visual Studio again and letting NCrunch go through the same solution does not help clean up the original workspace. Once I close down Visual Studio again and check the workspace folder I am left with two workspaces that are both taking up 433MB. This would explain why I am seeing the workspace folder size rise over an entire week to 26 Gigs or so.

I don't think I'd be able to send you a log of what the engine is doing without first sanitizing it, but if you have things that I could look at I'd appreciate that.

In the mean time I will see if I can trim down the solution and reproduce the problem with a smaller number of projects in an attempt to isolate what could be causing the problem.
Back to top
rlarno
#4 Posted : Tuesday, February 7, 2012 2:24:14 PM(UTC)
Rank: Member

Groups: Registered
Joined: 9/27/2011(UTC)
Posts: 27
Location: Belgium

Thanks: 6 times
Was thanked: 5 time(s) in 5 post(s)
FYI: if you have some free memory it really helps to set up a RAMDrive. And configure NCrunch to use that for its 'business'. My machine has 8GB and usually has 4GB that is not in use, so I have set up a RAMDrive of 2GB where NCrunch is storing its data. I reboot my machine (laptop) daily, so I always have a clean environment. Rebuilding the solution each day is also very acceptable as it is all happening in memory.

Ps: I have a pet peeve with large solutions, can you perhaps explain why you have 58 projects in your solution? (PM me)
Back to top
1 user thanked rlarno for this useful post.
DamonOverboe on 10/21/2014(UTC)
Remco
#5 Posted : Tuesday, February 7, 2012 8:40:11 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,974

Thanks: 929 times
Was thanked: 1256 time(s) in 1169 post(s)
Hi Greba,

Thanks for sharing the details. I'm wondering if you might be able to confirm for me whether you're doing any cross process testing or anything that might hold file locks on the workspaces? One way to confirm this is if you go through some of the instructions above, but after closing down your IDE, you try manually deleting the contents of the workspace.
Back to top
greba
#6 Posted : Tuesday, February 7, 2012 9:45:57 PM(UTC)
Rank: Member

Groups: Registered
Joined: 1/27/2012(UTC)
Posts: 13

Was thanked: 3 time(s) in 3 post(s)
I was able to look at this problem in a little bit more detail and was able to trim my solution down to one project that still caused the problem. It looks like adding a reference to a dll from a lib path is causing NCrunch to not be able to clean up the workspace.

I am able to delete the workspaces no problem even while the IDE is open (I disable NCrunch first).
Back to top
Remco
#7 Posted : Wednesday, February 8, 2012 12:52:50 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,974

Thanks: 929 times
Was thanked: 1256 time(s) in 1169 post(s)
Nice testing - that is a very useful piece of information. I'll see what I can do about arranging a fix.
Back to top
greba
#8 Posted : Wednesday, February 8, 2012 1:47:30 PM(UTC)
Rank: Member

Groups: Registered
Joined: 1/27/2012(UTC)
Posts: 13

Was thanked: 3 time(s) in 3 post(s)
That would be great. In the mean time I'll make sure I clear out the workspace folder every once in a while. If you need any more details let me know as I should be able to reproduce the problem in a very simple solution.
Back to top
1 user thanked greba for this useful post.
Remco on 2/8/2012(UTC)
Remco
#9 Posted : Saturday, February 25, 2012 11:15:49 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,974

Thanks: 929 times
Was thanked: 1256 time(s) in 1169 post(s)
Hi Greba,

I've been having a tough time this weekend trying to reproduce the above problem and I'm wondering if you would be able to send me your simple solution?

Also ... do you have a custom workspace base path configured?


Thanks!

Remco
Back to top
Remco
#10 Posted : Wednesday, March 7, 2012 12:25:41 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,974

Thanks: 929 times
Was thanked: 1256 time(s) in 1169 post(s)
For anyone interested, 1.38b (just released) contains a fix for an issue that was preventing workspaces cleaning up if they contained read-only files. Hopefully this should solve the problems described above.
Back to top
jaminto
#11 Posted : Wednesday, March 7, 2012 8:13:29 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 3/7/2012(UTC)
Posts: 1
Location: Austin, TX
I just downloaded NCrunch today (1.38.0.17b), and it looks pretty cool, but i am also having this issue. After using it for a couple of hours, my ncrunch temp folder is 28Gb. It looks like it's copying my entire packages directory and not cleaning it up in every workspace (?) folder.
i was able to delete the oldest 20 of about 35 workspace folders manually through windows explorer. the remaining 15 are locked by something and cannot be deleted right now.
Back to top
Remco
#12 Posted : Wednesday, March 7, 2012 9:44:49 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,974

Thanks: 929 times
Was thanked: 1256 time(s) in 1169 post(s)
Hi Jaminto,

Are you able to help with answering the questions below? This will help me to determine whether what you're experiencing is defective behaviour or behaviour by design:

* How large is your solution with all of its projects and resources? (As required in the workspace by NCrunch)
* Are you making use of any large resource files that are included into several different projects using the 'Additional files to include' setting?
* When your NCrunch workspace directory has filled up, is it doing so with only a single root level folder (i.e. '5195' or something), or are there multiples of these containing the individual project folders?
* After you shut down the IDE and wait for 30 seconds, are all of the workspaces removed?
* What do you currently have your 'process pool size' and 'max number of processing threads' set to?


Thanks!

Remco
Back to top
greba
#13 Posted : Thursday, March 8, 2012 7:51:16 PM(UTC)
Rank: Member

Groups: Registered
Joined: 1/27/2012(UTC)
Posts: 13

Was thanked: 3 time(s) in 3 post(s)
I hadn't received email notifications of replies so I didn't know you needed more information. I'll try the new build out and let you know how it goes.

I did get jaminto's post as an email though.
Back to top
1 user thanked greba for this useful post.
Remco on 3/8/2012(UTC)
greba
#14 Posted : Friday, March 9, 2012 2:39:12 PM(UTC)
Rank: Member

Groups: Registered
Joined: 1/27/2012(UTC)
Posts: 13

Was thanked: 3 time(s) in 3 post(s)
Just did a quick test and this issue looks to be solved! Thanks.

Still checking out the new features in the latest build, but I'm liking what I'm seeing so far. Keep up the great work.
Back to top
1 user thanked greba for this useful post.
Remco on 3/11/2012(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.140 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download