hi,

i have a large solution (~70 projects) which is taking a very very long time (for ncrunch) to initialise building of projects.
not sure why this is, couple of thoughts which might be relevant:

1. Each project takes on average, upwards of 1min 30sec to build the assembly. Almost all of this time appears to be in building the workspace.
   
2. I see each project gets its own subdir in the ncrunch working dir. Each project has an entire copy of the 270MB nuget packages dir from the solution. Why is this? This (copying the packages dir) is presumably what is causing building a workspace to take so long. I can see the _ncrunchreferences dir should(?) contain all the required refs for the project - so why are the nuget packages being brought along as well?
   
3. To shorten the build time I've got a custom solution config DebugFast which a) doesn't run the codecontracts rewriter and b) doesn't try to restore nuget packages. Am I right in thinking that ncrunch doesn't try to restore nuget packages anyway?
   
4. I have made some small modifications to the .nuget\nuget.targets solution file. I presume that shouldn't affect ncrunch - although each ncrunch proj working dir does have the .nuget dir (and as mentioned the packages dir) so i wonder if ncrunch is getting mixed up with nuget somehow.
   
5. Once the workspace is set up, building is as quick as I'd expect. It's setting up the workspace that is taking ages.
   
6. copy of a single project ncrunch build log:
   [18:03:09.005-BuildTask-72] Launching task: [BuildTask: [SnapshotComponent: MyProject.Tests, 8, 55829439], BeingProcessed]
   [18:03:09.027-BuildTask-72] Building new workspace d:\ncrunch\10068\121\ for component MyProject.Tests
   Note how long (45s) it takes to build the workspace...
   [18:03:54.961-BuildTask-72] Registered usage of workspace: d:\ncrunch\10068\121 (now 1 usages)
   [18:03:54.97-BuildTask-72] Now building MyProject.Tests
   [18:03:54.972-BuildTask-72] Returning process 10936 from pool with signature: [ProcessSignature:
   Compiler
   nCrunch.Compiler.RemoteBuildRunner
   nCrunch.build.4.0.config
   x86
   
   Framework4_0
   
   ]
   [18:03:59.17-BuildTask-72] Storing process 10936 in pool
   [18:03:59.17-BuildTask-72] Adjusted assembly references for this project are as follows:
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\83\Shared\Common\bin\DebugFast\MyCompany.MySolution.Common.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\86\MyProject\MyProjectService\MyProjectApi.Contracts\bin\DebugFast\MyCompany.MySolution.MyProject.MyProjectService.MyProjectApi.Contracts.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\8\Tests\Tests.Common\bin\DebugFast\MyCompany.MySolution.Tests.Common.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\101\MyProject\MyProjectService\MyProjectService.Client\bin\DebugFast\MyCompany.MySolution.MyProject.MyProjectService.Client.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\85\MyProject\MyProjectService\MyProjectService.Contracts.Data\bin\DebugFast\MyCompany.MySolution.MyProject.MyProjectService.Contracts.Data.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\94\MyProject\MyProjectService\MyProjectService.Contracts\bin\DebugFast\MyCompany.MySolution.MyProject.MyProjectService.Contracts.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\103\MyProject\MyProjectService\MyProjectService.Secure.Contracts\bin\DebugFast\MyCompany.MySolution.MyProject.MyProjectService.Secure.Contracts.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\115\MyProject\MyProjectService\MyProjectService\bin\DebugFast\MyCompany.MySolution.MyProject.MyProjectService.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\121\_ncrunchreferences\Machine.Specifications.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\121\_ncrunchreferences\Moq.dll
   [18:03:59.17-BuildTask-72] C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\mscorlib.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\121\_ncrunchreferences\Ninject.dll
   [18:03:59.17-BuildTask-72] d:\ncrunch\10068\121\_ncrunchreferences\Ploeh.AutoFixture.dll
   [18:03:59.17-BuildTask-72] C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\System.Core.dll
   [18:03:59.17-BuildTask-72] C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\System.dll
   [18:03:59.17-BuildTask-72] C:\Program Files (x86)\Reference Assemblies\Microsoft\Framework\.NETFramework\v4.0\System.ServiceModel.dll
   [18:03:59.17-BuildTask-72] Project configuration is:
   [18:03:59.17-BuildTask-72] IgnoredTests=nCrunch.Core.Configuration.ITestSelector[]
   [18:03:59.17-BuildTask-72] HiddenWarnings=System.String[]
   [18:03:59.17-BuildTask-72] CopyReferencedAssembliesToWorkspace=False
   [18:03:59.17-BuildTask-72] ConsiderInconclusiveTestsAsPassing=False
   [18:03:59.17-BuildTask-72] IgnoreThisComponentCompletely=False
   [18:03:59.17-BuildTask-72] RunPreBuildEvents=False
   [18:03:59.17-BuildTask-72] RunPostBuildEvents=False
   [18:03:59.17-BuildTask-72] PreviouslyBuiltSuccessfully=True
   [18:03:59.17-BuildTask-72] InstrumentAssembly=True
   [18:03:59.17-BuildTask-72] PreventSigningOfAssembly=False
   [18:03:59.17-BuildTask-72] AnalyseExecutionTimes=True
   [18:03:59.17-BuildTask-72] IncludeStaticReferencesInWorkspace=True
   [18:03:59.17-BuildTask-72] DefaultTestTimeout=60000
   [18:03:59.17-BuildTask-72] AdditionalFilesToInclude=System.String[]
   [18:03:59.17-BuildTask-72] UseBuildConfiguration=DebugFast
   [18:03:59.17-BuildTask-72] ProxyProcessPath=
   [18:03:59.17-BuildTask-72] UseCPUArchitecture=AutoDetect
   [18:03:59.171-BuildTask-72] Using config file: d:\ncrunch\10068\121\Tests\MyProject.Tests\bin\DebugFast\MyCompany.MySolution.Tests.MyProject.Tests.dll.config
   [18:03:59.197-BuildTask-72] Build was successful for d:\ncrunch\10068\121\Tests\MyProject.Tests\bin\DebugFast\MyCompany.MySolution.Tests.MyProject.Tests.dll
   [18:03:59.198-BuildTask-72] Task processing complete for [BuildTask: [SnapshotComponent: MyProject.Tests, 8, 55829439], BeingProcessed], processing time: 00:00:22.5660000
   

I'm guessing it's the nuget/packages copy which is blowing processing times out - any ideas why this is?

thanks
justin

Hi,

You're absolutely correct with all of the above. NCrunch is copying all the nuget references into each isolated workspace for each component, and this creates a lot of grinding. It also uses a lot of disk space.

NCrunch won't recopy the nuget packages again once the engine is properly wound up, but it will nuke all the workspaces when the engine is disabled - so every time you restart the IDE you're in for a long wait.

It's difficult to find a correct answer to a problem like this - basically because Nuget needs to keep the packages list up to date, and NCrunch needs isolated workspaces. The main problem comes down to Nuget auto package restore. If NCrunch doesn't copy the nuget packages into the workspace and Nuget package restore is enabled, Nuget will automatically open a connection to its home server in order to refresh and redownload the packages. Naturally this gives a massive increase to initial build times when workspaces are being created.

A solution I am considering right now is as follows:

1. If a user has Nuget auto package restore disabled, NCrunch will avoid copying the Nuget packages to the workspace and simply copy the binaries that are being used instead. This should mean that the problem you are experiencing will not exist.
2. If a user has Nuget auto package restore enabled, NCrunch will copy the Nuget packages to each workspace and present a warning to the user informing them that their initial workspace build times may be higher because of the need to support Nuget auto package restore.

Would this solution work for you?


Cheers,

Remco

so when ncrunch runs msbuild, can't you just pass in a property?
$(RestorePackages) = false

well, that solution would work fine for me. ;)

yeah, i see what you mean though. If the packages aren't there to begin with then you're kind of screwed unless you know to run a full build (to restore the packages) first.
i guess at the very least you (ncrunch) could check to see if the sln packages dir is empty and show a warning - that would pick up the situation where someone new to the team has loaded up the sln for the first time.

what about a simple button in ncrunch to verify all the packages exist?
that could just shell out to the nuget command line - much like we do on our build server...
step 1:
foreach ($project in get-project -all){
# trivial 1-liner to run nuget commandline to verify/restore packages for project into the ncrunch workspace for that project
}
step 2:
build solution with restore packages turned off via msbuild switch

that way users have an easy resolution if ncrunch falls over because nuget packages aren't available in the workspace.

For anyone interested, the recently released v1.39b contains a fix that should greatly reduce the disk consumption of solutions that are using Nuget.