Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Use symbolic links for referenced assemblies in workspace
yanglee
#1 Posted : Monday, June 24, 2013 11:46:04 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/25/2012(UTC)
Posts: 89
Location: China

Thanks: 23 times
Was thanked: 19 time(s) in 18 post(s)
Hi,

Since I installed RavenDB's nuget packages (67MB) for my main project several days ago, I got an IOException every 0.5~2 hours because my 4GB RamDisk is out of space. By using WinDirStat, I found 95% of the disk space is filled with referenced dll files.

I'm not a computer expert, not sure if it's technically a good idea, but is it possible to solve this issue by using symbolic links instead of copying these assembles to the workspace?

Cheers,

Yang
Remco
#2 Posted : Tuesday, June 25, 2013 2:23:07 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,979

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi Yang -

This is something I did look at earlier, as it would reduce the amount of file copying going on.

Unfortunately, it wouldn't solve all the isolation issues that come with the need to separate projects from their base solution. The big reason NCrunch copies these files is because of file locking issues - DLL files are locked by any application domain that makes use of them, which means they cannot be modified or replaced while tests are in progress or their host processes are still running.

There is actually a project-level configuration setting that you might find useful - Include static references in workspace. When turned off, NCrunch will avoid copying DLL files into the workspace and will simply reference them from your foreground solution. As described, a side-effect of doing this is that you'll find you are unable to update any of these DLLs while the NCrunch engine is enabled - but this should greatly reduce disk consumption on your ram disk.


Cheers,

Remco
1 user thanked Remco for this useful post.
yanglee on 6/25/2013(UTC)
yanglee
#3 Posted : Tuesday, June 25, 2013 5:02:03 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/25/2012(UTC)
Posts: 89
Location: China

Thanks: 23 times
Was thanked: 19 time(s) in 18 post(s)
Remco, thanks for your detailed answer.

I've increased ram disk to 5.5GB, now everything back to normal.

Quote:
The big reason NCrunch copies these files is because of file locking issues - DLL files are locked by any application domain that makes use of them, which means they cannot be modified or replaced while tests are in progress or their host processes are still running.

Do you mean NCrunch copies these files, so that:

#1) I can modify the original projects while tests are in progress;

#2) Tests of different sessions can work concurrently (because if they share the same copy of a file, the tests of one session may lock a file, that will cause problems for tests of another session)?

If #2 is not a reason, is it possible to create one copy of a file, and for each test session, create a hard link for the copy. This should greatly reduce disk consumption, and I can modify the original projects while tests are in progress.
Remco
#4 Posted : Tuesday, June 25, 2013 6:30:48 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,979

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
#1 - Definitely.

#2 - Sort of. In the current architecture, NCrunch can run multiple sets of tests concurrently using the same workspace, which technically does mean that tests can already be sharing the same set of DLLs. Extensive real-world testing has shown to be largely safe to do. The problems around concurrency generally occur more around build steps. Build steps have the potential to do some particularly crazy things with files - everything from copying them around to manipulating their physical structure (i.e. PostSharp). NCrunch is able to fire off a new build for a project while this project is already being used for testing tasks being run in parallel - this wouldn't be safe to do without building a completely new workspace with a new set of files.

Having a single file shared between multiple workspaces also introduces new complexities - such as finding a safe time to update the file if it is changed in the foreground solution. Files can only be updated when all test processes using them have been terminated, so by isolating the use of files to individual workspaces, NCrunch always has the flexibility to build a completely new workspace to deal with any concurrency issues. Where resources are shared between workspaces, NCrunch then needs to wait for all shared use of these resources to expire before it can update the resources ... this may include waiting for some very big long tests to finish their execution.

I hope this makes sense :) File locking issues are easily one of the largest constraints on the engine and were a huge source of problems in early releases of the product. It's taken quite a bit of trial and error to find a way to work them out, but the current model works fairly well as long as people don't introduce absolute file path references in their solutions.
1 user thanked Remco for this useful post.
yanglee on 6/25/2013(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.036 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download