Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Receiving test errors after updating to 3.29
Options
Previous Topic Next Topic
DanAtkinson
#1 Posted : Friday, August 9, 2019 4:49:09 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 14
Location: United Kingdom

Thanks: 2 times
Hi,

I'm running Visual Studio 2019 (16.2.1), my tests are written against NUnit (3.6), and I'm using NCrunch 3.30. I updated NCrunch after seeing Remco's message about 3.29 but it didn't resolve my problem.

Many (but not all) of my tests across multiple solutions are now returning the following error:

Quote:
NCrunch was unable to retrieve a meaningful result from this test due to an unexpected error - was the execution process terminated?


This occurs with even the simplest unit test that has no other dependencies, with an example below.

Code:
Imports NUnit.Framework

<TestFixture>
Public Class SchedulerTests

    <Test>
    Public Sub NoTest()
        'Arrange
        Dim lbTestVariable As Boolean

        'Act
        lbTestVariable = True

        'Assert
        Assert.IsTrue(lbTestVariable)
    End Sub

End Class


I reverted my installation to 3.28 and all of my tests are now green again, so something does appear to have been broken in 3.29+. I'm happy to submit a bug report detailing this if required.
Back to top
 |  Edit by user
Remco
#2 Posted : Saturday, August 10, 2019 12:04:55 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for sharing this issue. The error message describes a situation where the test execution process suddenly disappears without warning. This makes it hard to establish the source of the problem as the process likely took any error information with it.

I hope you'll excuse the questions below, as they help to deductively narrow down where things are going wrong:

- You mentioned you started having this problem after upgrading your NCrunch. Does downgrading back to the older version you were previously using resolve the problem?
- Are you running a virus scanner? If so, does disabling it make any difference?
- Can you check your Windows Event Log for any error messages?
- Can you confirm if this problem appears consistently (i.e. you can never ever get a test result), or is it intermittent? If it's intermittent, have you identified any pattern?
- Can you try making an MSTest based test project? Do tests in this project fail to run also?
- Can you confirm if this is affecting .NET Framework or .NET Core? Or both?
- Could you submit a bug report after you've had this happen to you? I don't think the log file will contain any error information, but it might still have a useful gem or two.
Back to top
DanAtkinson
#3 Posted : Monday, August 12, 2019 10:21:45 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 14
Location: United Kingdom

Thanks: 2 times
Hi Remco,

Thanks for the response. I'll try to answer your questions to the best of my abilities.

Remco;13749 wrote:
- You mentioned you started having this problem after upgrading your NCrunch. Does downgrading back to the older version you were previously using resolve the problem?

Yes, I had this issue with 3.29 and 3.30 but when I downgraded to 3.28, the issue disappeared. I have kept Visual Studio 2017 (which I use invariably) on 3.30 for the time being.

Remco;13749 wrote:
- Are you running a virus scanner? If so, does disabling it make any difference?

I am running Trend Micro OfficeScan. I am unable to disable it as I'm running as a LAPS (Local Administrator Password Solution) account.

Remco;13749 wrote:
- Can you check your Windows Event Log for any error messages?

There are two errors logged and one crash.

  1. Application: nCrunch.TestHost40.x86.exe
    Framework Version: v4.0.30319
    Description: The process was terminated due to an internal error in the .NET Runtime at IP 71163AED (710C0000) with exit code 80131506.
  2. Faulting application name: nCrunch.TestHost40.x86.exe, version: 3.30.0.1, time stamp: 0x5d4b2482
    Faulting module name: clr.dll, version: 4.7.3416.0, time stamp: 0x5cabfd2c
    Exception code: 0xc0000005
    Fault offset: 0x000a3aed
    Faulting process id: 0x2758
    Faulting application start time: 0x01d550f23b5d0eff
    Faulting application path: c:\program files (x86)\microsoft visual studio\2017\professional\common7\ide\extensions\remco software\ncrunch for visual studio 2017\nCrunch.TestHost40.x86.exe
    Faulting module path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
    Report Id: 66a4e9b7-eb26-4aaf-8f4d-cdf10f7df13d
    Faulting package full name:
    Faulting package-relative application ID:
  3. Fault bucket , type 0
    Event Name: APPCRASH
    Response: Not available
    Cab Id: 0

    Problem signature:
    P1: nCrunch.TestHost40.x86.exe
    P2: 3.30.0.1
    P3: 5d4b2482
    P4: clr.dll
    P5: 4.7.3416.0
    P6: 5cabfd2c
    P7: c0000005
    P8: 000a3aed
    P9:
    P10:

    Attached files:
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER51C0.tmp.mdmp
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER527D.tmp.WERInternalMetadata.xml
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER528E.tmp.xml
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER528C.tmp.csv
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER52AC.tmp.txt
    \\?\C:\Users\master\AppData\Local\Temp\WER7DF5.tmp.appcompat.txt
    \\?\C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_nCrunch.TestHost_5593cf846367591ee468c5de66341966d55babc5_fc69aab3_cab_5e8e7fe5\memory.hdmp

    These files may be available here:
    \\?\C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_nCrunch.TestHost_5593cf846367591ee468c5de66341966d55babc5_fc69aab3_cab_5e8e7fe5

    Analysis symbol:
    Rechecking for solution: 0
    Report Id: 66a4e9b7-eb26-4aaf-8f4d-cdf10f7df13d
    Report Status: 100
    Hashed bucket:
    Cab Guid: 0
    NB: I can provide these files privately if required.


Remco;13749 wrote:
- Can you confirm if this problem appears consistently (i.e. you can never ever get a test result), or is it intermittent? If it's intermittent, have you identified any pattern?

It appears consistently. Not all tests fail however, and there is no discernible difference between a test that fails and one that passes in terms of the work that is undertaken.

Remco;13749 wrote:
- Can you try making an MSTest based test project? Do tests in this project fail to run also?

Yes and no. I created a C# unit test project by accident and it worked. I then deleted the C# project and created a VB.NET project. The test immediately failed.

Remco;13749 wrote:
- Can you confirm if this is affecting .NET Framework or .NET Core? Or both?

.NET Framework. I'm not currently using .NET Core. My particular solutions are using .NET 4.0.

Remco;13749 wrote:
- Could you submit a bug report after you've had this happen to you? I don't think the log file will contain any error information, but it might still have a useful gem or two.

I have submitted the bug report now.
Back to top
Remco
#4 Posted : Monday, August 12, 2019 11:24:26 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for these extra details.

Could you try running one of the tests that consistently produces this problem with the VS debugger attached? (i.e. use the debug option in NCrunch). Make sure you have your exception settings to break on ALL exceptions.

At the moment I'm strongly suspecting the virus scanner. Almost all the changes we've made in the v3.29 release were sitting at a higher level in the structure (i.e. around the IDE and NCrunch engine). We've touched very little down in the test execution system, certainly nothing that would cause such catastrophic instability. Hopefully the debug session can provide more detailed error information.
Back to top
DanAtkinson
#5 Posted : Monday, August 12, 2019 12:58:56 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 14
Location: United Kingdom

Thanks: 2 times
I was unable to debug the very simple test I created. After it left the End Sub, Visual Studio froze. After about 10 minutes, I get the following dialog message:

Quote:
A fatal error has occurred and debugging needs to be terminated. For more details, please see the Microsoft Help and Support web site. HRESULT=0x80131c08. ErrorCode=0x0.
Back to top
Remco
#6 Posted : Tuesday, August 13, 2019 12:29:33 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Could you try deactivating the following two config settings at solution level (or for all projects in the solution) and confirm for me if this makes any difference in the error?

Instrument output assembly
Track file dependencies
Back to top
1 user thanked Remco for this useful post.
DanAtkinson on 8/13/2019(UTC)
DanAtkinson
#7 Posted : Tuesday, August 13, 2019 9:23:06 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 14
Location: United Kingdom

Thanks: 2 times
Remco;13759 wrote:
Could you try deactivating the following two config settings at solution level (or for all projects in the solution) and confirm for me if this makes any difference in the error?

Instrument output assembly
Track file dependencies


Disabling 'Instrument output assembly' made no difference. However, when I disabled 'Track file dependencies', the tests immediately started passing. I re-enabled the first flag again and it worked fine. I was then able to switch on and off 'track file dependencies' and see the tests fail or pass.

I'm not sure what the next steps are for this. This appears to be a solution (thank you!) but I'm not sure if it's intended to be a permanent one, or what the impact of it will be.

Any advice here would be appreciated.
Back to top
Remco
#8 Posted : Tuesday, August 13, 2019 11:50:34 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
DanAtkinson;13761 wrote:
[quote=Remco;13759]
I'm not sure what the next steps are for this. This appears to be a solution (thank you!) but I'm not sure if it's intended to be a permanent one, or what the impact of it will be.


Excellent. I can now guess with reasonable accuracy what is happening here.

This particular feature uses low level API hooks to track file system access calls made by your code during the test run. This data is useful for impact detection, as it means NCrunch can trigger your test when it makes use of any resource files that might change during your coding session.

Because the API hooks themselves work using memory hacks, I would expect the potential for their reliability to affected by the state of the environment they are running on (for example, O/S or .NET versions, virus scanners, other anomalies). It isn't really possible for us to establish exactly what is causing the hooks to destabilise the process without a way to reproduce the problem, and right now I see no way for us to do that. Regardless, the effective solution is simply to disable this setting on your machine. You probably won't notice it gone. On newer platforms (i.e .NET Core) we don't have this feature anyway, as we couldn't find a way to get the hooks to work.
Back to top
1 user thanked Remco for this useful post.
DanAtkinson on 8/14/2019(UTC)
DanAtkinson
#9 Posted : Wednesday, August 14, 2019 9:02:14 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 14
Location: United Kingdom

Thanks: 2 times
Okay, thank you. In that case, I consider this particular issue resolved and closed.

I saw in the documentation for "track file dependencies" that it didn't work in .NET Core and assumed that, as a result, the impact can't be that significant.

Thank you for your help in this matter.
Back to top
DanielM
#10 Posted : Wednesday, October 23, 2019 7:54:00 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/13/2017(UTC)
Posts: 9
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
Hello,

I just recently updated from v3.22.1 to v3.31.1 and discovered I have a very similar issue as described here. The difference is turning the settings off didn't fix the issue. I tried to turn off both of the settings noted above but I still get the error. Then I individually turned each one off. The tests still failed. I revered back to v3.22.1 and my tests pass again. I also tried v3.28.6 that DanAtkinson suggested worked for him and I observed tests still failed but it also consistently froze with a few hundred tests left in the queue before finishing the run. Do you have any other suggestions?

Faulting application name: nCrunch.TestHost45.x86.exe, version: 3.31.0.3, time stamp: 0x5d81c737
Faulting module name: clr.dll, version: 4.8.4010.0, time stamp: 0x5d3a2220
Exception code: 0xc00000fd
Fault offset: 0x00069ae4
Faulting process id: 0x563c
Faulting application start time: 0x01d589cf3f564375
Faulting application path: c:\program files (x86)\microsoft visual studio\2017\professional\common7\ide\extensions\remco software\ncrunch for visual studio 2017\nCrunch.TestHost45.x86.exe
Faulting module path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
Report Id: e527bc68-1e38-44b3-94c3-e60aca222d24
Faulting package full name:
Faulting package-relative application ID:


Fault bucket 1266577770779753011, type 1
Event Name: APPCRASH
Response: Not available
Cab Id: 0

Problem signature:
P1: nCrunch.TestHost45.x86.exe
P2: 3.31.0.3
P3: 5d81c737
P4: clr.dll
P5: 4.8.4010.0
P6: 5d3a2220
P7: c00000fd
P8: 00069ae4
P9:
P10:

Attached files:
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER8036.tmp.mdmp
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER818F.tmp.WERInternalMetadata.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER81AF.tmp.xml
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER81AF.tmp.csv
\\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER81CF.tmp.txt

These files may be available here:
\\?\C:\ProgramData\Microsoft\Windows\WER\ReportArchive\AppCrash_nCrunch.TestHost_326bfc572bc347c31f3423729b922cd726c3ac_3c62d1ab_213c8382

Analysis symbol:
Rechecking for solution: 0
Report Id: e527bc68-1e38-44b3-94c3-e60aca222d24
Report Status: 268435456
Hashed bucket: dcf5bcb206de48e96193c9c59bbbba33
Cab Guid: 0
Back to top
DanielM
#11 Posted : Wednesday, October 23, 2019 8:15:19 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/13/2017(UTC)
Posts: 9
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
Just to add a little more information, in case it helps. I do have a couple of tests that I'm working to resolve an issue for where the message displayed in v3.22.1 is "System.InvalidOperationException : The calling thread cannot access this object because a different thread owns it." I see the issue but have yet to isolate it from the unit tests. In the newer versions, this message is not displayed and many many more tests fail with "NCrunch was unable to retrieve a meaningful result from this test due to an unexpected error - was the execution process terminated?"
Back to top
Remco
#12 Posted : Thursday, October 24, 2019 12:31:39 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for posting.

The error message described is a catch-all message that NCrunch provides when a test task runner process suddenly disappears without providing any error information. As you can probably imagine, there are a near limitless number of things that can cause this problem ranging from heap corruption in the process right through to issues deep within the .NET platform and the O/S itself. Because the problem is generally tied to the memory state of the process, it may behave differently under different versions of NCrunch, even if we haven't introduced any code that might have changed the runtime behaviour.

Could you confirm the following for me?

- What happens if you turn off the 'Instrument output assembly' setting for ALL the projects in your solution?
- Are you running a virus scanner? If so, does disabling it make any difference?
- Can you check your Windows Event Log for any error messages?
- Can you confirm if this problem appears consistently (i.e. you can never ever get a test result), or is it intermittent? If it's intermittent, have you identified any pattern?
- Can you try making an MSTest based test project? Do tests in this project fail to run also?
- Can you confirm if this is also occurring if you switch your test projects to a different version of .NET? (Maybe a later one?)
Back to top
DanielM
#13 Posted : Wednesday, November 20, 2019 10:24:16 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/13/2017(UTC)
Posts: 9
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
Thanks for the quick response. We were able to resolve the issue and found some UI related code that needed to use the Dispatcher. I'm still using the older version but I plan on upgrading soon and giving it another shot.
Thanks again.
Back to top
1 user thanked DanielM for this useful post.
Remco on 11/21/2019(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.122 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download