Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
3 Pages<123
Tests utilising Microsoft.AspNet.TestHost fail in NCrunch
Options
Previous Topic Next Topic
Remco
#42 Posted : Wednesday, September 27, 2017 11:01:59 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Are you seeing this from a project that has a project.assets.json file? (i.e. .NET Core or CPS project). The error itself is being kicked up because NCrunch is expecting a project assets file, but doesn't seem to have knowledge of one. I'm aware of a race condition that can cause some problems where the IDE tries to update the project.assets.json file at the same time NCrunch is loading the project, but this should be a rare occurrence and usually appears as a locking error instead.
Back to top
alastairs
#43 Posted : Thursday, September 28, 2017 9:26:56 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
Yes, it's a .NET Core project - maybe targeting .NET Standard. It does appear that it's fairly transient...
Back to top
WWW
Remco
#44 Posted : Thursday, September 28, 2017 9:34:38 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Do you notice much of a pattern around how/when it appears? My particular interest is whether this happens mostly when NCrunch is first initialised. The problem itself seems to be a load issue. When NCrunch interrogates MSBuild as it loads the project, MSBuild isn't returning a path for the project.assets.json file. Subsequently there's an explosion downstream. Off the top of my head, the only reason I could think this might happen is if there was external interference at the time NCrunch is loading the project, for example, Visual Studio actions.
Back to top
alastairs
#45 Posted : Saturday, September 30, 2017 4:34:36 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
The only discernible pattern I've identified is that when it happens once, it then happens consistently across re-initialisations of NCrunch, even when there's nothing further happening inside or outside VS. However, "Reload and rebuild" on individual projects seems to work more reliably.
Back to top
WWW
Remco
#46 Posted : Sunday, October 1, 2017 12:03:34 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Is there any chance you could submit a bug report immediately after you've had this happen? Maybe the log will reveal something.
Back to top
Remco
#47 Posted : Thursday, October 5, 2017 2:42:59 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Thanks for sending through the logs. It's a bit unfortunate that the size of your solution prevented the loading of the projects from sharing information in the logs (which would have been the most interesting part), but I have identified something wrong going in NCrunch's project synchronisation system that does have the potential for downstream instability and poor performance.

I'd like to try and introduce a codefix to this and get you a new build. I'll let you know as soon as I have this available.
Back to top
1 user thanked Remco for this useful post.
alastairs on 10/5/2017(UTC)
alastairs
#48 Posted : Thursday, October 5, 2017 8:32:55 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
Thanks Remco. I've seen it in a smaller solution too (15 projects) - I thought I'd submitted logs for that as well? I'm currently focussed on work in that solution so I'll send another bug report through unless that's still too large?
Back to top
WWW
alastairs
#49 Posted : Thursday, October 5, 2017 3:18:24 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
Some additional information: one trigger is running `dotnet ef migrations add` on one of the projects in the solution. This generates code which is added to the project, so I suspect editing the code files/project directories outside of VS will trigger this bug.

** EDIT **

It seems running `dotnet test` will also trigger this bug, so maybe it's one of the `obj` files rather than source files.
Back to top
WWW
 |  Edit by user
Remco
#50 Posted : Monday, October 9, 2017 5:41:04 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Back to top
1 user thanked Remco for this useful post.
alastairs on 10/9/2017(UTC)
alastairs
#51 Posted : Monday, October 9, 2017 2:05:35 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
Thanks Remco. I've tried the new build, and it looks as though running `dotnet test` on this solution's startup project will still trigger the bug. I'll send a fresh bug report in case the logs show up anything useful.
Back to top
WWW
Remco
#52 Posted : Tuesday, October 10, 2017 1:01:12 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
alastairs;11342 wrote:
Thanks Remco. I've tried the new build, and it looks as though running `dotnet test` on this solution's startup project will still trigger the bug. I'll send a fresh bug report in case the logs show up anything useful.


Thanks for the extra bug report. Just to clarify, what are the exact steps you take to reproduce this? Are you running 'dotnet test' on your bin\debug assembly while NCrunch is running in the IDE?
Back to top
alastairs
#53 Posted : Tuesday, October 10, 2017 2:00:49 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
Remco;11346 wrote:
Thanks for the extra bug report. Just to clarify, what are the exact steps you take to reproduce this? Are you running 'dotnet test' on your bin\debug assembly while NCrunch is running in the IDE?

  1. Open solution
  2. Reload and Rebuild all projects via NCrunch (or some subset, it's not clear and seems to be most of them at least)
  3. cd src\MyProject
  4. dotnet test ..\..\test\MyProject.Tests


I wonder if this is related to the TestHost package somehow? I've just tried these repro steps on a different solution which does not utilise TestHost, and the bug is not present.
Back to top
WWW
Remco
#54 Posted : Wednesday, October 11, 2017 3:56:56 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
I've done a deeper dive into the bug reports you've sent through, and I can see something quite unusual in the logs.

On the log you sent through yesterday, there are projects with their obj\project.assets.json file stored on F: drive while their build output path and source files seem to be stored on D: drive.

Is this intentional? If so, how has this project been configured? The placement of project.assets.json on a different drive breaks NCrunch's project atomicity model and results in downstream exceptions.

I've noticed that 'dotnet test' seems to cause the project.assets.json file to be completely recreated and packages to be restored if necessary, in the same way as though the solution has just been opened in Visual Studio. As a result, this triggers NCrunch to reload the project, because a change in project.assets.json can result in a significant change to project structure.
Back to top
1 user thanked Remco for this useful post.
alastairs on 10/11/2017(UTC)
alastairs
#55 Posted : Wednesday, October 11, 2017 8:14:30 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
Oh balls. Yes, that is intentional. I set up a junction point between the folders on the D: and F: drives, for a variety of reasons that basically boils down convenience driven by laziness 😂 The "different solution" I mentioned in my previous post was opened from the F: drive (where the code actually resides). What I was hoping would be a sort of symlink actually wasn't quite as transparent as that. Sorry for the wild goose chase, Remco.
Back to top
WWW
Remco
#56 Posted : Wednesday, October 11, 2017 8:42:28 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
alastairs;11357 wrote:
Oh balls. Yes, that is intentional. I set up a junction point between the folders on the D: and F: drives, for a variety of reasons that basically boils down convenience driven by laziness 😂 The "different solution" I mentioned in my previous post was opened from the F: drive (where the code actually resides). What I was hoping would be a sort of symlink actually wasn't quite as transparent as that. Sorry for the wild goose chase, Remco.


Could you clarify how you've managed to do this? Is there a custom build property you've set? I'd like to improve the error handling in this scenario.
Back to top
alastairs
#57 Posted : Wednesday, October 11, 2017 8:49:11 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/13/2011(UTC)
Posts: 54
Location: Cambridge, UK

Thanks: 22 times
Was thanked: 4 time(s) in 4 post(s)
Remco;11358 wrote:
Could you clarify how you've managed to do this? Is there a custom build property you've set? I'd like to improve the error handling in this scenario.


Sure, I used mklink - so it's not a junction point at all actually - so the folder on F: is mapped to the folder on D:. (Apparently you can do this natively in PowerShell on Win10 with New-Item: https://stackoverflow.com/a/34905638/5296).

I had the solution opened from the folder on D: and a PowerShell window opened on F:, and ran `dotnet test` from that PowerShell window.
Back to top
WWW
Remco
#58 Posted : Wednesday, October 11, 2017 8:54:54 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,123

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
alastairs;11359 wrote:

I had the solution opened from the folder on D: and a PowerShell window opened on F:, and ran `dotnet test` from that PowerShell window.


This explains a great deal. I couldn't understand why this wasn't failing every time. With the files being on different drives, you should have had an error that consistently showed itself. It looks like MSBuild's caching of file paths was actually the problem here. When you ran 'dotnet test', it would redirect all the file paths to the other drive, where NCrunch would then read them via MSBuild, and explode in confusion when nothing connected properly. This is actually quite a remarkable set of interactions. I've revised the error message under NCrunch itself so that it'll be a bit more clear than just an NRE, though I don't expect that anyone else will experience this in quite the same way that you have.
Back to top
Users browsing this topic
Guest
3 Pages<123
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.109 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download