Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

NCrunch + NuGet PackageReference + PostSharp
Amarok
#1 Posted : Thursday, March 08, 2018 2:21:58 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/27/2013(UTC)
Posts: 4
Location: Austria

Thanks: 2 times
Hello!

We are currently in the process of migrating our .NET 4.7.1 projects from NuGet packages.config to the new PackageReference system. This changes a few things. For example, included NuGet packages are no longer located in a packages folder relative to the solution, but in a global location, for example, D:\DATA\packages on my workstation.

Our projects also use PostSharp (https://www.postsharp.net/), which extends the MSBuild-based build process to post-process assemblies. That means the assembly generated by C# compiler is partially rewritten by PostSharp afterwards. You can find indication of this integration in NuGet's artifacts under /src/Lib/obj/Lib.csproj.nuget.g.props and /src/Lib/obj/Lib.csproj.nuget.g.targets. There you can see that a PostSharp-specific targets file PostSharp.targets is included. This file is taken from the global packages location D:\DATA\packages.

/src/Lib/obj/Lib.csproj.nuget.g.props
Code:

<?xml version="1.0" encoding="utf-8" standalone="no"?>
<Project ToolsVersion="14.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <PropertyGroup Condition=" '$(ExcludeRestorePackageImports)' != 'true' ">
    <RestoreSuccess Condition=" '$(RestoreSuccess)' == '' ">True</RestoreSuccess>
    <RestoreTool Condition=" '$(RestoreTool)' == '' ">NuGet</RestoreTool>
    <ProjectAssetsFile Condition=" '$(ProjectAssetsFile)' == '' ">D:\Demo\src\Lib\obj\project.assets.json</ProjectAssetsFile>
    <NuGetPackageRoot Condition=" '$(NuGetPackageRoot)' == '' ">D:\DATA\Packages</NuGetPackageRoot>
    <NuGetPackageFolders Condition=" '$(NuGetPackageFolders)' == '' ">D:\DATA\Packages</NuGetPackageFolders>
    <NuGetProjectStyle Condition=" '$(NuGetProjectStyle)' == '' ">PackageReference</NuGetProjectStyle>
    <NuGetToolVersion Condition=" '$(NuGetToolVersion)' == '' ">4.6.0</NuGetToolVersion>
  </PropertyGroup>
  <PropertyGroup>
    <MSBuildAllProjects>$(MSBuildAllProjects);$(MSBuildThisFileFullPath)</MSBuildAllProjects>
  </PropertyGroup>
  <ImportGroup Condition=" '$(ExcludeRestorePackageImports)' != 'true' ">
    <Import Project="$(NuGetPackageRoot)\postsharp\5.0.45\build\PostSharp.props" Condition="Exists('$(NuGetPackageRoot)\postsharp\5.0.45\build\PostSharp.props')" />
  </ImportGroup>
</Project>


Now, building and testing via NCrunch works well on my local machine, but fails when using a Grid Node Server. On the Grid Node Server the PostSharp-specific post-build targets are not run, which causes the test to fail, because PostSharp didn’t rewrote the NotNullAttribute in my sample code.

I looked at the workspace used on the Grid Node Server and I suspect that there is a small re-configuration missing to make this work correctly. In my understanding NCrunch copies NuGet packages to the Grid Node Server into location <snapshot-dir>\.nuget, this is located at D:\Snapshots\.nuget on our server.

Now, the Grid Node Server workspace containing our project contains everything necessary, even the NuGet specific artifacts project.assets.json, Lib.csproj.nuget.g.targets and Lib.csproj.nuget.g.props. But looking at Lib.csproj.nuget.g.props we can find the culprit for our problem. The file still contains

Code:

<NuGetPackageRoot Condition=" '$(NuGetPackageRoot)' == '' ">D:\DATA\Packages</NuGetPackageRoot>


which refers to the location on my workstation, but doesn’t make sense on the Grid Node Server.

Since third-party (PostSharp) props and targets are included via
Code:

  <ImportGroup Condition=" '$(ExcludeRestorePackageImports)' != 'true' ">
    <Import Project="$(NuGetPackageRoot)\postsharp\5.0.45\build\PostSharp.props" Condition="Exists('$(NuGetPackageRoot)\postsharp\5.0.45\build\PostSharp.props')" />
  </ImportGroup>


using $(NuGetPackageRoot) a wrong folder location is used on Grid Node Servers.

Is my analysis correct, or did you provide MSBuid property NuGetPackageRoot via command line when building?

A sample project can be found here: https://www.zipshare.com...W50b24tcGFhci5jb20ifQ==
Remco
#2 Posted : Friday, March 09, 2018 2:15:15 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,248

Thanks: 691 times
Was thanked: 844 time(s) in 804 post(s)
Hi, thanks for posting! I appreciate the detail you've provided on your investigation into this issue.

When NCrunch transfers solutions over to grid nodes, it needs to send the packages with them.

This somewhat breaks the binding model that MS use for NuGet, which assumes that all machines building software have an open internet connection and can execute a restore step prior to running builds (not necessarily true on all grid nodes).

So to keep things working, NCrunch copies the packages to a known location on the grid node and adjusts the bindings to point to the grid node package locations, which in almost all cases will be different to that on the client. Doing this reliably has been extremely challenging, partly because of the evolving nature of the platform, but also because they are represented in different ways throughout the project structure. In the runtime domain, there is actually the possibility of multiple package root directories that are sequentially searched by the runtime; a concept that doesn't appear to exist presently in the build system.

NCrunch forces the runtime environment to use the new package paths by manipulating the project.assets.json file, and redirects the build system by injecting/overriding properties in MSBuild. Unfortunately, the PostSharp build reference seems to fall through a hole here, because the importation of its .props file happens before any of the overrides can take place. So probably we'll need to change how this works and rewrite the file instead. It will take some effort before we're able to support this use case.

So for the time being, to work around this problem you will need to avoid overriding your default nuget package root directory for this use case. I actually would recommend avoiding changing this directory in future too (unless you have a VERY good reason to do so), because the interactions here can create some extremely confusing scenarios that may not be well tested in the .NET tools space. I don't doubt that this problem has cost you several hours already, and with the platform in this area still being very new (and subject to change), avoiding this complexity would be desirable.
1 user thanked Remco for this useful post.
Amarok on 3/9/2018(UTC)
Amarok
#3 Posted : Friday, March 09, 2018 8:01:20 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 9/27/2013(UTC)
Posts: 4
Location: Austria

Thanks: 2 times
Hi Remco!

Thanks for your response. It's interesting to hear what NCrunch does under the hood to make all work. A really fantastic product! Can't code without it ;-)

Quote:
So for the time being, to work around this problem you will need to avoid overriding your default nuget package root directory for this use case. I actually would recommend avoiding changing this directory in future too (unless you have a VERY good reason to do so), because the interactions here can create some extremely confusing scenarios that may not be well tested in the .NET tools space. I don't doubt that this problem has cost you several hours already, and with the platform in this area still being very new (and subject to change), avoiding this complexity would be desirable.


Unfortunately, not changing the default nuget package folder isn't working in our setup, because other developers also use the same grid node server and everybody has a different root folder by default. BUT, a workaround is to change this nuget package folder on all developer workstations to the same folder that matches the folder on the shared grid node server, in our case we changed it to D:\Snapshots\.nuget and now everything is working again.

As said, this is a workaround. A solution would be nice as I'm not sure whether PostSharp will the only one causing such problems. But for now, we have a working setup.

Thanks
Olaf Kober
Remco
#4 Posted : Friday, March 09, 2018 10:51:08 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,248

Thanks: 691 times
Was thanked: 844 time(s) in 804 post(s)
Hi Olaf,

As long as the package root folder is left at its default (which I realise is different between users), the transfer should still work fine. The problem itself isn't caused by the package root being different, but rather by it being overridden. We have a situation where some parts of the system are considering the override when processing on the grid node, and some parts (PostSharp) are not. I'm glad you've found a short term solution though. We have a task set out to try to find a better way to handle this use-case.
Users browsing this topic
Guest (2)
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.054 seconds.