I'm trying to run the sample integration test project from the ASP.NET Core documentation, but I am receiving the following error:
System.InvalidOperationException: Solution root could not be located using application root C:\Users\Name\AppData\Local\NCrunch\27496\3\RazorPagesProject.Tests\bin\Debug\netcoreapp2.1\.
at Microsoft.AspNetCore.TestHost.WebHostBuilderExtensions.UseSolutionRelativeContentRoot(IWebHostBuilder builder, String solutionRelativePath, String applicationBasePath, String solutionName)
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.SetContentRoot(IWebHostBuilder builder)
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer()
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers)
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient(WebApplicationFactoryClientOptions options)
at RazorPagesProject.Tests.IntegrationTests.BasicTests.CanGetAGithubUser() in C:\Users\Name\source\repos\IntegrationTestSample\RazorPagesProject.Tests\IntegrationTests\BasicTests.cs:line 74
at Xunit.Sdk.TestInvoker`1.<>c__DisplayClass48_1.<<InvokeTestMethodAsync>b__1>d.MoveNext() in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\Runners\TestInvoker.cs:line 260
--- End of stack trace from previous location where exception was thrown ---
at Xunit.Sdk.ExecutionTimer.AggregateAsync(Func`1 asyncAction) in C:\Dev\xunit\xunit\src\xunit.execution\Sdk\Frameworks\ExecutionTimer.cs:line 48
at Xunit.Sdk.ExceptionAggregator.RunAsync(Func`1 code) in C:\Dev\xunit\xunit\src\xunit.core\Sdk\ExceptionAggregator.cs:line 90
The tests run correctly using the Visual Studio and ReSharper test runners.
The documentation says to disable shadow copy in the xunit.runner.json for the xUnit runner, however I'm not able to find a way to make this work using NCrunch.
Changing the Copy referenced assemblies to workspace setting doesn't appear to make any difference (same exception, still looking in the AppData folder).
We'll do some deeper looking into this after .NET Core 2.1 is released. At the moment I'm having trouble getting the sample solution to build in my environment. The current situation with pre-release dependency conflicts seems to create all kinds of challenges.
Based on the message you're receiving here, I'm not optimistic that we can find a way to make this work without significant effort. The code here appears to be trying to find a solution file and using this to discover the web code, which is a major problem for NCrunch as in our workspaces the solution as a concept just doesn't exist. It's quite possible that MS have finally found a design that is so baked into the solution structure that we have no easy way to cater for it. It's hard for me to be certain about any of this without taking more time to analyse the issue, I'm merely stating this to help manage your expectations. If you have alternative design options, I do recommend exploring them.
Remco wrote:Thanks :) We'll take a look and will let you know when we have more information.
It looks like you could investigate further around this area.
If we can get the correct TestAssemblies and the WebApplicationFactoryContentRoot configured correctly (no easy feat :)) it may be possible to load the correct content
To update everyone on this issue, we've taken a deep look at it, and it's not good.
This package works by hard-baking the location of the web project into the test project's binary. So if the web project moves relative to the test project, everything breaks. Given the current design of NCrunch, that's a dealbreaker because we re-use test project binaries across different test runs with environments that are dynamically constructed out of whichever projects happen to be on the latest version of the code at the time. In short, given it's structure, there is no way we can make this tool work with NCrunch without either:
1. Significantly redesigning the NCrunch engine to introduce new concurrency constraints, introducing loss of performance, extra complexity, lots of work and all the potential issues that go with it (not really feasible)
2. Introducing a code change to the toolset to force it to use the web project from a specific location at run time
We can, of course, introduce IL-level hacks to force the toolset to work under the existing structure, but it feels cleaner to get a solution implemented in the toolset itself. I'll raise this issue with MS internally to see if they can help here.
At this stage it's safe to say that it will likely be at least a few weeks before we have a solution to this problem.
To update everyone on this issue, we've taken a deep look at it, and it's not good.
This package works by hard-baking the location of the web project into the test project's binary. So if the web project moves relative to the test project, everything breaks. Given the current design of NCrunch, that's a dealbreaker because we re-use test project binaries across different test runs with environments that are dynamically constructed out of whichever projects happen to be on the latest version of the code at the time. In short, given it's structure, there is no way we can make this tool work with NCrunch without either:
1. Significantly redesigning the NCrunch engine to introduce new concurrency constraints, introducing loss of performance, extra complexity, lots of work and all the potential issues that go with it (not really feasible)
2. Introducing a code change to the toolset to force it to use the web project from a specific location at run time
We can, of course, introduce IL-level hacks to force the toolset to work under the existing structure, but it feels cleaner to get a solution implemented in the toolset itself. I'll raise this issue with MS internally to see if they can help here.
At this stage it's safe to say that it will likely be at least a few weeks before we have a solution to this problem.
Has there been any movement with regard to this issue?
austsw wrote:
Has there been any movement with regard to this issue?
Yes! There has been movement, but no usable results yet. It's looking like this will require code changes on both sides (NCrunch and the package). I've submitted a PR to MS and we're discussing it with them directly.
The pull request for this change has been accepted from MS's side. My understanding is that it should be in the v2.2 release of the library. I'm not sure when v2.2 is expected. Hopefully when it arrives, we'll have NCrunch v3.18 released so that we're ready for it. If not, I'll just push an early build of v3.18 with a fix for this specific issue.
ASP.Net Core 2.2 Roadmap
https://github.com/aspnet/Announcements/issues/307
(Copy/Paste)
We are currently planning to have 3 previews before RTM:
August – Preview 1
September - Preview 2
October - Preview 3
Before End-of-year – RTM
NCrunch v3.18 has just been released including the NCrunch-side solution to this problem. Unfortunately, the solution won't be effective until ASP.NET pull request 8073 goes out the door in one of MS's releases. Hopefully they'll have a preview of .NET Core 2.2 out soon and the change will be included in this.
Unfortunately, I know of no workaround for this problem. I can honestly say that I did try very hard to find one.
Just bumped into the same issue for a new test project and verified it works when using 2.2.0-preview2-35157 (released 12th of September)
(I am using NCrunch 3.20)
So, with net core 2.1 you can use:
//Use Microsoft.AspNetCore.TestHost (2.1.1) in test project. Test web app target framework netcoreapp2.1.
private HttpClient _client;
public MiscWebTest()
{ //constructor (non static for simplicity)
var builder = new WebHostBuilder()
.UseStartup<Startup>();
var testServer = new TestServer(builder);
_client = testServer.CreateClient();
}
[Test]
public void CanInvokeValues()
{
var response = _client.GetAsync("/api/values").Result;
response.EnsureSuccessStatusCode();
}
with net core 2.2 preview you can use:
//Web project: Microsoft.AspNetCore.App (2.2.0-preview2-35157), target framework netcoreapp2.2
//Test project: Microsoft.AspNetCore.Mvc.Testing (2.2.0-preview2-35157)
private HttpClient _client;
public MiscWebTest()
{ //constructor (non static for simplicity)
var webAppFactory = new WebApplicationFactory<Startup>();
_client = webAppFactory.CreateClient();
[Test]
public void CanInvokeValues()
{
var client = _testServer.CreateClient();
var response = client.GetAsync("/api/values").Result;
response.EnsureSuccessStatusCode();
}
*Edit: I bumped into another issue with above WebapplicationFactory for 2.2 when using Startup (deriving from the "production startup" type) defined in my test project. so i reverted back to the way I did it in 2.1
System.NullReferenceException : Object reference not set to an instance of an object.
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateWebHostBuilder()
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer()
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers)
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(Uri baseAddress, DelegatingHandler[] handlers)
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient(WebApplicationFactoryClientOptions options)
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient()
Hi,
Is this issue still occurring?
I have a project set up with SDK schema <Project Sdk="Microsoft.NET.Sdk"> and target <TargetFramework>net462</TargetFramework>.
In the code I use aspnetcore 2.2 for the web api application (which runs ok).
So I am using aspnetcore but I am targetting net462 for now.
My tests are written in NUnit and use the approach of a class derived from WebApplicationFactory<Startup>
The tests run OK in Visual Studio 2019 with the test runner but in NCrunch I get errors like because it cannot find the ".Service.deps.json" file which is in the actual bin folder but not in the NCrunch folder.
This error shows up for the test class:
System.InvalidOperationException : Can't find'C:\Users\[]\AppData\Local\NCrunch\30904\2\[].Service.Tests\bin\Debug\net462\[].Service.deps.json'.
This file is required for functional tests to run properly. There should be a copy of the file on your source project bin folder.
If that is not the case, make sure that the property PreserveCompilationContext is set to true on your project file. E.g '<PreserveCompilationContext>true</PreserveCompilationContext>'.
For functional tests to work they need to either run from the build output folder or the [].Service.deps.json file from your application's output directory must be copied to the folder where the tests are running on.
A common cause for this error is having shadow copying enabled when the tests run.
TearDown : System.NullReferenceException : Object reference not set to an instance of an object.
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureDepsFile()
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.EnsureServer()
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateDefaultClient(DelegatingHandler[] handlers)
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient(WebApplicationFactoryClientOptions options)
at Microsoft.AspNetCore.Mvc.Testing.WebApplicationFactory`1.CreateClient()
at [].Service.Tests.[]Tests.Setup() in C:\[].Service.Tests\[]Tests.cs:line 24
--TearDown
at [].Service.Tests.[]Tests.TearDown() in C:\[].Service.Tests\[]Tests.cs:line 52
and this error for the individual test:
OneTimeSetUp: System.InvalidOperationException : Can't find'C:\Users\[]\AppData\Local\NCrunch\30904\2\[].Service.Tests\bin\Debug\net462\[].Service.deps.json'.
This file is required for functional tests to run properly. There should be a copy of the file on your source project bin folder.
If that is not the case, make sure that the property PreserveCompilationContext is set to true on your project file. E.g '<PreserveCompilationContext>true</PreserveCompilationContext>'.
For functional tests to work they need to either run from the build output folder or the [].Service.deps.json file from your application's output directory must be copied to the folder where the tests are running on.
A common cause for this error is having shadow copying enabled when the tests run.
What shoud I do to have NCrunch tests running correctly on my machine and on other devs machines?