Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Building Large Source Trees Spanning Solutions
kalebpederson
#1 Posted : Monday, October 30, 2017 5:01:37 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/1/2012(UTC)
Posts: 25
Location: US

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
Hi all,

Not too long ago we switched from an extremely large single-solution project to a multiple solution project whose build system is based on the recommendation in "Appendix B, Building Large Source Trees" of "Using MSBuild and Team Foundation Build". The Appendix containing a full description of this build process is available online at http://msbuildbook.com/Appendices.pdf. Appendix B starts at numbered page 579 (page 11 in the PDF).

Here's an example of what that might look like and the problem that I run into with NCrunch:

$BASE/../MSBuild/CommonMSBuild*.Targets
$BASE/build.proj # defines the order of the areas
$BASE/build.root # used to identify the root of the tree
$BASE/Area1/Solution1.sln
$BASE/Area1/dirs.proj # references to the solution or set of projects to build
$BASE/Area1/dirs.props # the properties used when doing the build
$BASE/Area1/Subdir/Solution1Project1.csproj
$BASE/Area2/Solution2.sln
$BASE/Area2/dirs.proj # references to the solution or set of projects to build
$BASE/Area2/dirs.props # the properties used when doing the build
$BASE/Area2/Subdir/Solution2Project1.csproj
...

For this example, let's assume that Area2/Solution2.sln depends on Area1/Solution1.sln. When the build occurs, Solution1 is built and its output is copied into a "libs" directory referenced by Solution2. This allows us to do a command line build, which populates the libs folder, and then open Solution2 in VisualStudio and have all its files present. Note that although my example shows a fairly shallow tree, the reality is that the tree can be much deeper. The common msbuild targets files are used by each Solution file through includes.

The problem that I have encountered in NCrunch is that, despite modifying the *.csproj files in Solution2 to conditionally reference all the content, NCrunch doesn't copy the $BASE/MSBuild/Common*.Targets into the NCrunch build hierarchy. It appears that this is the case regardless of the "Additional Files to Include" setting because the hierarchy of files references appears to be retained which would place the common target files outside of the NCRunch directory.

To put that into a picture, here's what NCrunch *attempts* to create when loading Solution2:

AppData/Local/NCrunch/PID1/1/_ncrunchreferences
AppData/Local/NCrunch/PID1/1/Solution2Project1/*
AppData/Local/MSBuild/CommonMSBuild*.Targets

However, even with "Additional Files to Include" set to include MSBuild/CommonMSBuild*.Targets, NCrunch prevents them from being copied to the filesystem (and rightfully so) because those files would end up outside of the AppData/Local/NCrunch hierarchy... if I'm understanding everything correctly.

Given the above, do you have any ideas on how we can get NCrunch working with the build structure given above?

Thanks!

--Kaleb

Remco
#2 Posted : Monday, October 30, 2017 10:21:00 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi Kaleb,

Thanks for posting!

When NCrunch creates a workspace for your project, it does so with consideration to upwards relative references by padding out the workspace directory. In this way, all dependencies should be contained completely within the workspace.

It's not clear to me why this particular file seems to have somehow exempted itself from this process. There must be something unusual about how the file is being referenced or copied. Do you have any custom code involved in copying this file? An interesting experiment would be to change the permissions on the file in its copied location so nothing can write to it, to see what will blow up. If NCrunch is doing this, it should blow up with an exception that will be visible in the build log while providing a useful stack trace.
1 user thanked Remco for this useful post.
kalebpederson on 11/7/2017(UTC)
kalebpederson
#3 Posted : Tuesday, November 7, 2017 9:47:43 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/1/2012(UTC)
Posts: 25
Location: US

Thanks: 5 times
Was thanked: 3 time(s) in 3 post(s)
Remco;11428 wrote:
When NCrunch creates a workspace for your project, it does so with consideration to upwards relative references by padding out the workspace directory. In this way, all dependencies should be contained completely within the workspace.

It's not clear to me why this particular file seems to have somehow exempted itself from this process. There must be something unusual about how the file is being referenced or copied.


Thanks Remco,

Based on your feedback I dug into this more and was able to figure it out and make everything work. Here's the explanation:

Within our build hierarchy are a number of different "props" files that define properties for that section of the tree. In some cases we will descend up the tree to one of these props files and then go back down the tree to a an overriding props files. These references look like the following:

Code:
  <Import Project="$([MSBuild]::GetDirectoryNameOfFileAbove($(MSBuildThisFileDirectory), dirs.props))\SomeSubDirectories\dirs.props" />


NCrunch ascended the tree looking for highest-level dirs.props file, which wasn't present. Once I explicitly added it as an additional file to include within the NCrunch solution configuration everything worked perfectly. Where I got confused was that the lower level dirs.props files were present (e.g., SomeSubDirectories\dirs.props), but the higher level ones weren't. I'm not sure if I expected NCrunch to automatically know they should be present since it was pulling in some of them, but in retrospect the projects in question don't actually need the intermediate dirs.props as they are only used to locate those deeper in the tree which are actually needed by the csproj files.

Thank you for the wonderful tool that is NCrunch!

--Kaleb

Remco
#4 Posted : Tuesday, November 7, 2017 11:36:26 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
I'm glad to hear you found the problem and were able to resolve it. This particular expression (GetDirectoryNameOfFileAbove) is a frightfully complex thing for NCrunch to handle because of the sequence of evaluation in MSBuild. It's not a surprise to me that this expression has interfered with resolving a critical file in your build. Where this happens, the 'Additional files to include' setting always provides a suitable fix. I'm glad you're up and running now with this solution.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.049 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download