Build/Test Issues

xunit.v3.mtp-v2 update to v4.0.0 makes all tests to fail

Started by gieniowski on 203 views

Hey,

after updating xunit package to version 4 as shown below:
<!--<PackageVersion Include="xunit.v3.mtp-v2" Version="3.2.2" />-->
<PackageVersion Include="xunit.v3.mtp-v2" Version="4.0.0" />

all of my tests are failing with this message:
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
---> System.NotSupportedException: This constructor has been deprecated. Please use one of the static factory functions (e.g., WithTestCaseIDs).
at Xunit.FrontControllerRunSettings..ctor(ITestFrameworkExecutionOptions options, IReadOnlyCollection`1 serializedTestCases) in /_/src/xunit.v3.runner.common/Frameworks/FrontControllerRunSettings.cs:line 19
at System.Reflection.MethodBaseInvoker.InterpretedInvoke_Constructor(Object obj, IntPtr* args)
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
--- End of inner exception stack trace ---
at System.Reflection.MethodBaseInvoker.InvokeDirectByRefWithFewArgs(Object obj, Span`1 copyOfArgs, BindingFlags invokeAttr)
at System.Reflection.MethodBaseInvoker.InvokeWithFewArgs(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.RuntimeConstructorInfo.Invoke(BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at System.Reflection.ConstructorInfo.Invoke(Object[] parameters)
at nCrunch.Module.XUnit3.ReflectionWrapper.Xunit3Reflector.InvokeSoleConstructor(Assembly xunit3Assembly, String typeName, Object[] args)
at nCrunch.Module.XUnit3.ReflectionWrapper.FrontControllerRunSettings..ctor(TestFrameworkOptions options, IReadOnlyCollection`1 serializedTestCases)
at nCrunch.Module.XUnit3.XUnit3FrameworkRuntimeEnvironment.RunTests(TestTaskOutput output, TestExecutionMapSet testMapSet, TestExecutionParameters parameters, IIpcMessageProcessor ipcMessageProcessor)
at nCrunch.TestExecution.TestRunnerThread.doRunTests()

I am using VS 2026 on windows 11 with all of the latest updates.

Thanks!
Afternoon,

Confirming this on my side, with some extra detail that might save you some digging.

Environment
- NCrunch 5.20.0.2 (nCrunch.Module.XUnit3.dll, file version 5.20.0.2), VS 2026 Enterprise, Windows 11
- Test project targets net10.0
- xunit.v3 4.0.0, xunit.runner.visualstudio 4.0.0, Microsoft.NET.Test.Sdk 18.9.0
- Same NotSupportedException as the original post, on every test in the assembly

Which assembly is involved
--------------------------
The type NCrunch reflects into lives in xunit.v3.runner.utility.netcore.dll. That file is placed in the test project's output directory by xunit's own MSBuild targets - buildTransitive/xunit.v3.core.mtp-v2.targets in the xunit.v3.core.mtp-v2 package contains a target named _XunitCopyRunnerDependencies whose comment reads "Copy dependencies used by dynamic linked runners like NCrunch". So this copy is deliberate and aimed at you; it just now delivers a 4.0.0.0 assembly.

Confirmed in my output folder:
xunit.v3.runner.utility.netcore.dll 4.0.0.0
xunit.v3.runner.common.dll 4.0.0.0
xunit.v3.mtp-v2.dll 4.0.0.0

The exact API change
--------------------
Comparing the shipped XML docs for xunit.v3.runner.common between versions:

3.2.2 exposes only:
Xunit.FrontControllerRunSettings.#ctor(ITestFrameworkExecutionOptions, IReadOnlyCollection<string>)

4.0.0 keeps that ctor (it now throws NotSupportedException at runtime) and adds three static factories:
FrontControllerRunSettings.WithSerializedTestCases(ITestFrameworkExecutionOptions, IReadOnlyCollection<string>)
FrontControllerRunSettings.WithTestCaseIDs(ITestFrameworkExecutionOptions, IReadOnlyCollection<string>)
FrontControllerRunSettings.WithSerializedTestCasesAndTestCaseIDs(ITestFrameworkExecutionOptions, IReadOnlyCollection<string>, IReadOnlyCollection<string>)

The old single-collection argument is now ambiguous between serialized test cases and test case IDs, which is presumably why the ctor had to go rather than being kept as an overload. That likely makes this more than a one-line swap in XUnit3FrameworkRuntimeEnvironment.RunTests - the adapter has to decide which of the two collections it is supplying.

The 4.0.0 release notes list other breaking changes for runner authors that may bite in the same pass:

OutOfProcessTestProcessLauncherBase.StartTestProcess gained a required shutdownProcessWaitSeconds parameter, and ProjectAssemblyRunner.Run gained resultWriters and testContextInitializedCallback. https://xunit.net/releases/v3/4.0.0

Scope - this is NCrunch-only
----------------------------
This is the result of our reproduction:
- Restore and build succeed on 4.0.0, no source changes needed
- dotnet test passes the full suite
- The VSTest adapter (xunit.runner.visualstudio 4.0.0) is fine
- Only NCrunch's execution path fails, and it fails 100% of tests

One more thing that may matter for the fix: xunit.v3 3.2.2 depends on xunit.v3.mtp-v1, while 4.0.0 depends on xunit.v3.mtp-v2 (Microsoft.Testing.Platform 2.3.3 vs 1.9.1). xUnit dropped MTP v1 support entirely in 4.0, so the adapter may need to account for the platform change as well as the ctor.

Workaround
----------
Pinning back to xunit.v3 3.2.2 + xunit.runner.visualstudio 3.1.5 restores normal NCrunch operation.
Verified that this puts xunit.v3.runner.utility.netcore.dll back to 3.2.2.0 in the output folder.
Clear bin/obj first - the mtp-v1/mtp-v2 switch regenerates the auto-generated entry point and self-registered extensions, so an incremental build is not enough.

Question for Remco Dev Team: 5.22 shipped 2026-07-22 and xunit.v3 4.0.0 released 2026-08-14, so I assume no current release handles this - can you confirm 5.22 is also affected, and whether a fix is likely in a patch or the next minor?
Thanks for letting me know about this. Based on your analysis, it looks like a breaking change and will most likely require an NCrunch update to fix. I'll let you know as soon as I have a solution available.
A fix for this has been included in the v5.23 build that I've just pushed out now. Please give it a try and let me know if it solves the problem for you.
Can confirm that new build works for me. Thank you Remco!

Post a reply

Log in to reply.