Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Microsoft.Testing.Platform support
Options
Previous Topic Next Topic
latop2604
#1 Posted : Monday, February 10, 2025 6:56:44 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/10/2025(UTC)
Posts: 7
Location: France

Thanks: 1 times
Hello,

According to a recent blog post from Microsoft, all major testing frameworks now support the new Microsoft.Testing.Platform.

Unfortunately, this does not appear to be the case with NCrunch at present, which has become a blocker for us in migrating to this platform.

Could you kindly clarify if there are any plans to add support for it, or if it will be included in the roadmap in the near future?

For reference, here is the link to the Microsoft blog post:
https://devblogs.microso.../mtp-adoption-frameworks
Back to top
Remco
#2 Posted : Monday, February 10, 2025 11:08:05 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,213

Thanks: 974 times
Was thanked: 1302 time(s) in 1207 post(s)
Hi, thanks for posting.

In .NET, the testing landscape is essentially divided into 3 areas:
- Test frameworks (xUnit, NUnit, MSTest, MSpec, etc)
- Test runners (VSTest, NCrunch, ReSharper, etc)
- Profilers (MSFakes, TypeMock Isolator, etc)

Test frameworks usually ship with their own bare-bones runners to allow them to work in CI systems or in situations where IDE-based runners are not easily available. Test frameworks take responsibility for defining the 'language' used by tests to describe themselves. They also take responsibility for the reflection calls that organise test fixtures and call into user code.

Test runners integrate with test frameworks and tie them into the environment you work in. It's the responsibility of the runner to determine which tests should be run (i.e. with user selection), arrange sub-processes to isolate test execution, call into the frameworks, then present the results of the test run. Test runners also serve as the 'glue' between all other tools involved in the testing system, which makes them responsible for ensuring things like profilers are correctly loaded into the environment so they can do their job.

Microsoft.Testing.Platform is a runner. So is NCrunch. They occupy the same space in the eco-system and as such they are direct competitors with the same features. There is no practical way to integrate them that will add value.

So in summary there are no plans to tie NCrunch into Microsoft.Testing.Platform.
Back to top
1 user thanked Remco for this useful post.
GlobalConcepts on 2/11/2025(UTC)
latop2604
#3 Posted : Tuesday, February 11, 2025 10:48:00 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/10/2025(UTC)
Posts: 7
Location: France

Thanks: 1 times
Ok, but do that mean there is no way to have both working at the same time?

I use xUnit and tried to upgrade one of my test project to xUnit.v3 and the new test platform. Everything work fine. dotnet test/ dotnet run / vs Test Explorer.
But I'm getting an error in NCrunch Tests panel.

Quote:
..\..\..\..\..\..\.nuget\packages\xunit.v3.core\1.1.0\buildTransitive\xunit.v3.core.targets (7, 5): xUnit.net v3 test projects must build an app host (set project property '<UseAppHost>true</UseAppHost>')


Any idea why?
Back to top
GlobalConcepts
#7 Posted : Tuesday, February 11, 2025 11:24:51 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/7/2013(UTC)
Posts: 57
Location: Germany

Thanks: 6 times
Was thanked: 6 time(s) in 6 post(s)
there is simply no need for having both runners active!
use ncrunch or use that testing platform.

regarding xunit.v3, have a look at https://xunit.net/docs/g...ng-started/v3/migration
If you're still expecting issues please open a new thred.

regards
Back to top
Remco
#6 Posted : Tuesday, February 11, 2025 11:47:24 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,213

Thanks: 974 times
Was thanked: 1302 time(s) in 1207 post(s)
latop2604;17910 wrote:
Ok, but do that mean there is no way to have both working at the same time?


As mentioned above, this normally wouldn't be needed, because when things are set up correctly, you can use NCrunch for all your tests and forget about needing another runner.

However, there's nothing in NCrunch that would stop you from using a different runner to execute tests. Certainly, VSTest still works fine even when NCrunch is enabled.

Quote:
..\..\..\..\..\..\.nuget\packages\xunit.v3.core\1.1.0\buildTransitive\xunit.v3.core.targets (7, 5): xUnit.net v3 test projects must build an app host (set project property '<UseAppHost>true</UseAppHost>')


It's a convention that Xunit v3 projects must be compiled as app hosts. You'll need to set this property on your test project otherwise you'll receive this error in NCrunch and any other runner. Actually, it's likely your project won't build at all without it.
Back to top
latop2604
#8 Posted : Tuesday, February 11, 2025 7:08:30 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/10/2025(UTC)
Posts: 7
Location: France

Thanks: 1 times
Thx @Remco

About

Quote:
It's a convention that Xunit v3 projects must be compiled as app hosts. You'll need to set this property on your test project otherwise you'll receive this error in NCrunch and any other runner. Actually, it's likely your project won't build at all without it.


It tried that, but I still get the same error in NCrunch panel. And be the way there is no compilation error using dotnet build/run/test without <UseAppHost>true</UseAppHost>. Only with NCrunch.

I check the xunit v3 migration page, and everything seem ok. The test project is really small.

It look like this (csproj + Directory.Packages.props)

Code:

<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>net9.0</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>

    <IsPackable>false</IsPackable>
    <IsTestProject>true</IsTestProject>
    <UseMicrosoftTestingPlatformRunner>true</UseMicrosoftTestingPlatformRunner>
    <OutputType>Exe</OutputType>
    <TestingPlatformDotnetTestSupport>true</TestingPlatformDotnetTestSupport>
    <UseAppHost>true</UseAppHost>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="coverlet.collector">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="coverlet.msbuild">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
    <PackageReference Include="JunitXml.TestLogger" />
    <PackageReference Include="Microsoft.AspNetCore.Mvc.Testing" />
    <PackageReference Include="Microsoft.NET.Test.Sdk" />
    <PackageReference Include="Microsoft.OpenApi.Readers" />
    <PackageReference Include="Moq" />
    <PackageReference Include="Moq.AutoMock" />
    <PackageReference Include="xunit.v3" />
    <PackageReference Include="xunit.runner.visualstudio">
      <PrivateAssets>all</PrivateAssets>
      <IncludeAssets>runtime; build; native; contentfiles; analyzers; buildtransitive</IncludeAssets>
    </PackageReference>
  </ItemGroup>

  <ItemGroup>
    <ProjectReference Include="..\..\src\MyApp.Api\MyApp.Api.csproj" />
  </ItemGroup>

  <ItemGroup>
    <Using Include="Xunit" />
  </ItemGroup>

</Project>


Code:

<Project>
  <PropertyGroup>
    <ManagePackageVersionsCentrally>true</ManagePackageVersionsCentrally>
  </PropertyGroup>
  <ItemGroup>
    <PackageVersion Include="coverlet.collector" Version="6.0.4" />
    <PackageVersion Include="coverlet.msbuild" Version="6.0.4" />
    <PackageVersion Include="JunitXml.TestLogger" Version="5.0.0" />
    <PackageVersion Include="Microsoft.AspNetCore.Mvc.Testing" Version="9.0.1" />
    <PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.13.0" />
    <PackageVersion Include="Microsoft.OpenApi.Readers" Version="1.6.23" />
    <PackageVersion Include="Moq" Version="4.20.72" />
    <PackageVersion Include="Moq.AutoMock" Version="3.5.0" />
    <PackageVersion Include="xunit.runner.visualstudio" Version="3.0.2" />
    <PackageVersion Include="xunit.v3" Version="1.1.0" />
    ...
  </ItemGroup>
</Project>


It was supposed to be a simple question. But turn out to be a rabbit hole.
I'm not really blocked. We don't plan to migrate any soon so I can continue to use NCrunch as usual.

I don't know if it's becoming off topic to you. Let me know.


Quote:
there is simply no need for having both runners active!
use ncrunch or use that testing platform.

regarding xunit.v3, have a look at https://xunit.net/docs/g...ng-started/v3/migration
If you're still expecting issues please open a new thred.

regards


What if my team want to use the new plateform in the CI, but at the same time I want to still be able to open Visual Studio and have NCrunch working out of the box?
Back to top
 |  Edit by user
latop2604
#9 Posted : Tuesday, February 11, 2025 7:29:06 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/10/2025(UTC)
Posts: 7
Location: France

Thanks: 1 times
And by the way I tried with the xunit v3 template with no edit (https://www.nuget.org/packages/xunit.v3.templates) and Configuring NCrunch give me the same error.

Step to repro:

Code:

dotnet new install xunit.v3.templates
mkdir testxunitv3
cd testxunitv3
dotnet new xunit3
dotnet new sln 
dotnet sln add .\testxunitv3.csproj 
dotnet run


Then open the sln with visual studio and configure NCrunch. ❌

Doing the same but with xunit template (not v3) work out of the box with NCrunch ✅

Code:

mkdir testxunit
cd testxunit
dotnet new xunit
dotnet new sln 
dotnet sln add .\testxunit.csproj 
dotnet test
Back to top
 |  Edit by user
Remco
#10 Posted : Wednesday, February 12, 2025 4:52:29 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,213

Thanks: 974 times
Was thanked: 1302 time(s) in 1207 post(s)
Thanks for sharing these extra details.

I've reproduced the problem exactly as you've described it. It seems that 5 days ago there was an update to Xunit that added in an extra build step including this check, which by default will always fail under NCrunch. So it seems that we're broken at the moment.

I'm working on getting a build out containing a fix for this. Meanwhile, there are a couple of ways you can work around the problem:

1. Dropping back to the previous version of Xunit v3 should work just fine

or

2. Under the NCrunch 'Custom build properties' project-level configuration setting, try adding a build property that sets 'UseAppHost' = 'true'. This will allow the new check in Xunit to pass.

By default, NCrunch forces UseAppHost=false because this property adds significant complexity to the build system. The app host build steps are responsible for setting up an entire hosting environment for the project being built (including generated .exe stubs and everything). This also introduces extra package dependencies late in the cycle that can cause synchronisation problems with grid nodes when using distributed processing. It's also not normally needed when working with NCrunch, because NCrunch itself sets up the environment when it runs the tests. This check would have been added to Xunit because when Xunit is being run outside of NCrunch, it uses the apphost itself to build the test environment. However, at build time, the check is still applied to everything, so NCrunch is still hit by it.

The fix I'm working on will simply suppress the check so that things can continue as normal. I'll let you know when it's available.

Quote:

What if my team want to use the new plateform in the CI, but at the same time I want to still be able to open Visual Studio and have NCrunch working out of the box?


Nothing in NCrunch will prevent you from doing this. However, note that you will need to design your tests in a way that will work with both runners. Because it's more feature-rich, NCrunch is more constrained than other runners, so it's possible to design tests that might run without problems using a different runner but may fail under NCrunch (see the 'considerations and constraints' in the NCrunch documentation). Likewise, if you use any of the features in NCrunch's runtime framework, these won't be available when using other runners, which could cause tests to fail there.

Most of the time it won't be a problem, it's just worth being aware of.
Back to top
 |  Edit by user
latop2604
#11 Posted : Wednesday, February 12, 2025 4:43:27 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/10/2025(UTC)
Posts: 7
Location: France

Thanks: 1 times
I tested the "Custom build properties" fix on the xunit.v3 template. And now it work as expected. 👍

But on my real app I still got an other build error.

Quote:
An error occurred while analysing this project after it was built: Unable to resolve an assembly required for the execution of Xunit3 on this machine. The assembly was expected at the following path: D:\Cache\NCrunch\Workspace\16276\7\tests\xunit.v3.core\bin\_content\runners\netcore\xunit.v3.runner.utility.netcore.dll
Please ensure the package containing this assembly is installed on your machine. If you introduce an assembly reference directly to this binary in your test project, NCrunch will resolve it directly using your specified path.


Maybe it's unrelated. I tried to look for differences from the template. But nothing obvious.

So I think we can close this topic.

Thank you again for your time. I hope I have been useful to you too.
NCrunch is really an awesome tool 🤘
Back to top
Remco
#12 Posted : Wednesday, February 12, 2025 11:12:55 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,213

Thanks: 974 times
Was thanked: 1302 time(s) in 1207 post(s)
latop2604;17923 wrote:

Maybe it's unrelated. I tried to look for differences from the template. But nothing obvious.


This is caused by a different issue. There's a fix coming for it in the next release of NCrunch. Alternatively, you can find an early build with the fix here.

Xunit v3 is still very new so things are still maturing a bit in this area.

Quote:

Thank you again for your time. I hope I have been useful to you too.
NCrunch is really an awesome tool 🤘


Thanks! I'm glad you like it :)
Back to top
Remco
#13 Posted : Thursday, February 13, 2025 11:51:43 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,213

Thanks: 974 times
Was thanked: 1302 time(s) in 1207 post(s)
Back to top
1 user thanked Remco for this useful post.
latop2604 on 2/13/2025(UTC)
latop2604
#14 Posted : Thursday, February 13, 2025 7:35:00 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/10/2025(UTC)
Posts: 7
Location: France

Thanks: 1 times
Unfortunately my license is too old for 5.12 😭
Back to top
Remco
#15 Posted : Thursday, February 13, 2025 11:32:54 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,213

Thanks: 974 times
Was thanked: 1302 time(s) in 1207 post(s)
latop2604;17929 wrote:
Unfortunately my license is too old for 5.12 😭


I've added a short evaluation license to your account. It should allow you to test whether this solves your problem, which I hope will help with your decision on whether to upgrade your license :)
Back to top
latop2604
#16 Posted : Friday, February 14, 2025 3:40:52 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/10/2025(UTC)
Posts: 7
Location: France

Thanks: 1 times
So, I tried the new version and ... everything work well now. xunitv3 template but also my real test project.

Well done. Thx. Now I need to talk with my company for license update.

By the way I have a mix of xunit v3 and not v3 in the sln, and everything still work. 👍
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.095 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download