Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Building C++/CLI project is very slow
openshac
#1 Posted : Thursday, March 5, 2015 10:33:12 AM(UTC)
Rank: Member

Groups: Registered
Joined: 1/19/2015(UTC)
Posts: 14
Location: United Kingdom

Thanks: 6 times
Was thanked: 1 time(s) in 1 post(s)
I have a solution with about 25 projects (about half of those being unit test projects). One of these projects is a C++/CLI project which is somewhat slow to build - about one minute in Visual Studio.
However in NCrunch it takes over 20 minutes, which is a much bigger problem. If I open my solution it means I can't run my tests until it this project has built. Also if I modify a project that this is referenced by this C++/CLI project it means

Is there a config change or something else I can do to improve the NCrunch build time for this project?
Remco
#2 Posted : Thursday, March 5, 2015 10:46:15 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this issue.

The first thing I'd try is turning off the 'Instrument output assembly' setting for this project. C++ projects aren't properly supported for NCrunch instrumentation anyway, so this isn't likely to have any downside .. and it will reduce the amount of processing NCrunch does on the assembly.

Something else worth doing would be to set your 'Log Verbosity' to 'Detailed', then examine the build log inside the Processing Queue for the project's build task. The NCrunch build log can sometimes give some useful information on timings. If turning off the instrumentation doesn't have any impact on the build time, then the build log may give us a clue as to where the bottleneck is.
openshac
#3 Posted : Thursday, March 5, 2015 1:12:36 PM(UTC)
Rank: Member

Groups: Registered
Joined: 1/19/2015(UTC)
Posts: 14
Location: United Kingdom

Thanks: 6 times
Was thanked: 1 time(s) in 1 post(s)
Remco,

Thanks for your reply, I turned on the logging and I think I can see the problem...

My C++/CLI project has "Additional Include Directories" specified. NCrunch is making a separate copy of all these files to its local build directory, rather than using the actual directories themselves. Ordinarily I guess this wouldn't be a big problem, however in this case our directory contains the Boost C++ libraries which number 38,000 files. It is this operation which is taking 20mins.

How do I get NCrunch to use the "Additional Include Directories" specified in the C++/CLI project, rather than make it's own copy?
Remco
#4 Posted : Thursday, March 5, 2015 8:31:50 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
This makes sense. Copying 38,000 files into each workspace will make a difference in build time.

I can suggest a solution to this, although it isn't a tidy one ... basically it involves referencing the library files using absolute file paths and removing them from the 'Additional files to include'. In this way they'll be referenced from your original solution directory instead of the workspace they'd otherwise need to be copied to. This is a bit messy because it does break workspace isolation - although for read-only libraries this is probably OK. If you're reluctant to warp your project with the absolute paths (which is messy for VCS systems and fresh checkouts), you may be able to get away with using an environment variable that can be specified on the machine to point to where the files need to be.
openshac
#5 Posted : Friday, March 6, 2015 9:51:57 AM(UTC)
Rank: Member

Groups: Registered
Joined: 1/19/2015(UTC)
Posts: 14
Location: United Kingdom

Thanks: 6 times
Was thanked: 1 time(s) in 1 post(s)
When you say "referencing the library files" what exactly do you mean?
The Boost header files are currently "referenced" from the C++/CLI files using #include statements.

Remco
#6 Posted : Friday, March 6, 2015 8:03:05 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
openshac;7056 wrote:
When you say "referencing the library files" what exactly do you mean?
The Boost header files are currently "referenced" from the C++/CLI files using #include statements.



Sorry, I often have my head stuck in C# land :)

It may be enough to simply adjust the library search paths. Most likely if you are including the headers using #include, then you'll have a relative search path set in the project pointing at the directory of the header files for the boost libraries. Changing this to an absolute file path would make NCrunch use the boost libraries that sit in your foreground solution, instead of trying to search relatively from the workspace and needing to have all the files copied with the project.
openshac
#7 Posted : Monday, March 9, 2015 9:20:42 AM(UTC)
Rank: Member

Groups: Registered
Joined: 1/19/2015(UTC)
Posts: 14
Location: United Kingdom

Thanks: 6 times
Was thanked: 1 time(s) in 1 post(s)
The directory I have specified in the "Additional Include Directories" is

C:\Program Files %28x86%29\Boost\boost_1_51_0

which is an absolute path.

However in the typical NCrunch output I am seeing:

ENGINE - [09:13:31.1332-LocalBuildTask-10] Writing new workspace member C:\Users\username\AppData\Local\NCrunch\9392\4\Program Files (x86)\Boost\boost_1_51_0\boost\mpl\at_fwd.hpp (version 1)

Is there something I can do to ensure NCrunch respects the absolute path?
Remco
#8 Posted : Monday, March 9, 2015 9:34:33 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Interesting ... It must be the case that MSBuild is automatically resolving these dependencies during the NCrunch load, then reporting them to NCrunch as being files inside the project. If this is indeed the case, it may be possible to work around this by hiding them from the NCrunch load.

Inside the project file, try adding a build condition to the declaration for the Additional Include Directory. For example:

<AdditionalIncludeDirectories Condition="$(NCrunch) != '1'">C:\Program Files %28x86%29\Boost\boost_1_51_0</AdditionalIncludeDirectories>
openshac
#9 Posted : Wednesday, March 11, 2015 9:15:20 AM(UTC)
Rank: Member

Groups: Registered
Joined: 1/19/2015(UTC)
Posts: 14
Location: United Kingdom

Thanks: 6 times
Was thanked: 1 time(s) in 1 post(s)
I tried specifying the condition you mentioned however it didn't change the behaviour. And it would also mean the additional include directories would get omitted if $(NCrunch) == '1', but I guess that modification was just to test things.
Remco
#10 Posted : Wednesday, March 11, 2015 9:34:06 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
openshac;7071 wrote:
I tried specifying the condition you mentioned however it didn't change the behaviour. And it would also mean the additional include directories would get omitted if $(NCrunch) == '1', but I guess that modification was just to test things.


Yes. Sorry ... it's likely this wouldn't have actually worked, even if it worked ... if that makes any sense :/

We're running out of options here. The big issue is that there isn't a way for NCrunch to separate the included files from other source files in your solution - MSBuild is simply combining them and returning a unified result set. This means that either the files get copied, or they can't be referenced at all and the build will completely fail.

I think the best option may to be look at ways in which we can exclude this particular project/assembly from NCrunch itself, and use NCrunch for the rest of your solution. Since NCrunch won't support code coverage over a C++ project anyway, there is probably little value in having it as part of the NCrunch build tree.

The way to do this is by building the output assembly from the project, then referencing the built assembly directly (from all depending projects) instead of the C++ project. This will likely make a mess of your build sequence, so the best way to handle this would be to make the references conditional on NCrunch being resident so that the <AssemblyReference> is used for NCrunch, and the <ProjectReference> is used for Visual Studio. This will let you use NCrunch without making a mess of your VS build. The only way you can do this is by modifying the project files manually and adjusting the references.

So the following:

<ItemGroup>
<ProjectReference Include="..\MyCppProject\MyCppProject.vcxproj">
...
</ProjectReference>
</ItemGroup>

Would become:

<ItemGroup>
<ProjectReference Include="..\MyCppProject\MyCppProject.vcxproj" Condition="$(NCrunch) != '1'">
...
</ProjectReference>
</ItemGroup>
<ItemGroup>
<Reference Include="..\MyCppProject\Bin\Debug\MyCppProject.dll" Condition="$(NCrunch) == '1' />
</ItemGroup>


When this change has been made, you'll need to do a rebuild of the C++ project in VS every time you make changes to it. This is because NCrunch will be hooked up to the output of this project, so it won't detect the change until you rebuild manually.

You'll need to turn on the 'Ignore this project completely' NCrunch project-level configuration setting for the C++ project, otherwise it'll still try to build it and consume valuable resources.

NCrunch may give you a warning about a 'lost reference' - just ignore this.
1 user thanked Remco for this useful post.
openshac on 3/31/2015(UTC)
openshac
#11 Posted : Tuesday, March 31, 2015 8:18:44 AM(UTC)
Rank: Member

Groups: Registered
Joined: 1/19/2015(UTC)
Posts: 14
Location: United Kingdom

Thanks: 6 times
Was thanked: 1 time(s) in 1 post(s)
Thanks for that suggestion, it worked. However with the by modifying the references in the way you have you end up getting two references displayed in Visual Studio. One for the assembly that's referenced and another one for the project reference. In addition one of these has a horrible warning icon against it.

To rectify this I modified the config you provided to:

Code:
  <Choose>
	<When Condition="$(NCrunch) != '1'">	
	  <ItemGroup>
		<ProjectReference Include="..\MyCppProject\MyCppProject.vcxproj" >
		  <Project>{6b56558e-42d4-4593-96d9-b446c95ca6d1}</Project>
		  <Name>MyCppProject</Name>
		</ProjectReference>
	  </ItemGroup>
	</When>
	<Otherwise>	
	  <ItemGroup>
		<Reference Include="MyCppProject, Version=1.0.0.0, Culture=neutral, PublicKeyToken=f8f13c26d6a0a8bd, processorArchitecture=x86">
		  <SpecificVersion>False</SpecificVersion>
		  <HintPath>..\MyCppProject\Bin\x86\Debug\MyCppProject.dll</HintPath>
		</Reference>
	  </ItemGroup>
	</Otherwise>
  </Choose>
1 user thanked openshac for this useful post.
Remco on 3/31/2015(UTC)
Remco
#12 Posted : Tuesday, March 31, 2015 8:39:34 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Great suggestion, thanks. I completely forgot about how VS would interpret it. I hope all is well now.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.068 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download