Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Receiving test errors after updating to 3.29
DanAtkinson
#1 Posted : Friday, August 9, 2019 4:49:09 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 10
Location: United Kingdom

Thanks: 2 times
Hi,

I'm running Visual Studio 2019 (16.2.1), my tests are written against NUnit (3.6), and I'm using NCrunch 3.30. I updated NCrunch after seeing Remco's message about 3.29 but it didn't resolve my problem.

Many (but not all) of my tests across multiple solutions are now returning the following error:

Quote:
NCrunch was unable to retrieve a meaningful result from this test due to an unexpected error - was the execution process terminated?


This occurs with even the simplest unit test that has no other dependencies, with an example below.

Code:
Imports NUnit.Framework

<TestFixture>
Public Class SchedulerTests

    <Test>
    Public Sub NoTest()
        'Arrange
        Dim lbTestVariable As Boolean

        'Act
        lbTestVariable = True

        'Assert
        Assert.IsTrue(lbTestVariable)
    End Sub

End Class


I reverted my installation to 3.28 and all of my tests are now green again, so something does appear to have been broken in 3.29+. I'm happy to submit a bug report detailing this if required.
Remco
#2 Posted : Saturday, August 10, 2019 12:04:55 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,722

Thanks: 744 times
Was thanked: 950 time(s) in 905 post(s)
Thanks for sharing this issue. The error message describes a situation where the test execution process suddenly disappears without warning. This makes it hard to establish the source of the problem as the process likely took any error information with it.

I hope you'll excuse the questions below, as they help to deductively narrow down where things are going wrong:

- You mentioned you started having this problem after upgrading your NCrunch. Does downgrading back to the older version you were previously using resolve the problem?
- Are you running a virus scanner? If so, does disabling it make any difference?
- Can you check your Windows Event Log for any error messages?
- Can you confirm if this problem appears consistently (i.e. you can never ever get a test result), or is it intermittent? If it's intermittent, have you identified any pattern?
- Can you try making an MSTest based test project? Do tests in this project fail to run also?
- Can you confirm if this is affecting .NET Framework or .NET Core? Or both?
- Could you submit a bug report after you've had this happen to you? I don't think the log file will contain any error information, but it might still have a useful gem or two.
DanAtkinson
#3 Posted : Monday, August 12, 2019 10:21:45 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 10
Location: United Kingdom

Thanks: 2 times
Hi Remco,

Thanks for the response. I'll try to answer your questions to the best of my abilities.

Remco;13749 wrote:
- You mentioned you started having this problem after upgrading your NCrunch. Does downgrading back to the older version you were previously using resolve the problem?

Yes, I had this issue with 3.29 and 3.30 but when I downgraded to 3.28, the issue disappeared. I have kept Visual Studio 2017 (which I use invariably) on 3.30 for the time being.

Remco;13749 wrote:
- Are you running a virus scanner? If so, does disabling it make any difference?

I am running Trend Micro OfficeScan. I am unable to disable it as I'm running as a LAPS (Local Administrator Password Solution) account.

Remco;13749 wrote:
- Can you check your Windows Event Log for any error messages?

There are two errors logged and one crash.

  1. Application: nCrunch.TestHost40.x86.exe
    Framework Version: v4.0.30319
    Description: The process was terminated due to an internal error in the .NET Runtime at IP 71163AED (710C0000) with exit code 80131506.
  2. Faulting application name: nCrunch.TestHost40.x86.exe, version: 3.30.0.1, time stamp: 0x5d4b2482
    Faulting module name: clr.dll, version: 4.7.3416.0, time stamp: 0x5cabfd2c
    Exception code: 0xc0000005
    Fault offset: 0x000a3aed
    Faulting process id: 0x2758
    Faulting application start time: 0x01d550f23b5d0eff
    Faulting application path: c:\program files (x86)\microsoft visual studio\2017\professional\common7\ide\extensions\remco software\ncrunch for visual studio 2017\nCrunch.TestHost40.x86.exe
    Faulting module path: C:\Windows\Microsoft.NET\Framework\v4.0.30319\clr.dll
    Report Id: 66a4e9b7-eb26-4aaf-8f4d-cdf10f7df13d
    Faulting package full name:
    Faulting package-relative application ID:
  3. Fault bucket , type 0
    Event Name: APPCRASH
    Response: Not available
    Cab Id: 0

    Problem signature:
    P1: nCrunch.TestHost40.x86.exe
    P2: 3.30.0.1
    P3: 5d4b2482
    P4: clr.dll
    P5: 4.7.3416.0
    P6: 5cabfd2c
    P7: c0000005
    P8: 000a3aed
    P9:
    P10:

    Attached files:
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER51C0.tmp.mdmp
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER527D.tmp.WERInternalMetadata.xml
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER528E.tmp.xml
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER528C.tmp.csv
    \\?\C:\ProgramData\Microsoft\Windows\WER\Temp\WER52AC.tmp.txt
    \\?\C:\Users\master\AppData\Local\Temp\WER7DF5.tmp.appcompat.txt
    \\?\C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_nCrunch.TestHost_5593cf846367591ee468c5de66341966d55babc5_fc69aab3_cab_5e8e7fe5\memory.hdmp

    These files may be available here:
    \\?\C:\ProgramData\Microsoft\Windows\WER\ReportQueue\AppCrash_nCrunch.TestHost_5593cf846367591ee468c5de66341966d55babc5_fc69aab3_cab_5e8e7fe5

    Analysis symbol:
    Rechecking for solution: 0
    Report Id: 66a4e9b7-eb26-4aaf-8f4d-cdf10f7df13d
    Report Status: 100
    Hashed bucket:
    Cab Guid: 0
    NB: I can provide these files privately if required.


Remco;13749 wrote:
- Can you confirm if this problem appears consistently (i.e. you can never ever get a test result), or is it intermittent? If it's intermittent, have you identified any pattern?

It appears consistently. Not all tests fail however, and there is no discernible difference between a test that fails and one that passes in terms of the work that is undertaken.

Remco;13749 wrote:
- Can you try making an MSTest based test project? Do tests in this project fail to run also?

Yes and no. I created a C# unit test project by accident and it worked. I then deleted the C# project and created a VB.NET project. The test immediately failed.

Remco;13749 wrote:
- Can you confirm if this is affecting .NET Framework or .NET Core? Or both?

.NET Framework. I'm not currently using .NET Core. My particular solutions are using .NET 4.0.

Remco;13749 wrote:
- Could you submit a bug report after you've had this happen to you? I don't think the log file will contain any error information, but it might still have a useful gem or two.

I have submitted the bug report now.
Remco
#4 Posted : Monday, August 12, 2019 11:24:26 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,722

Thanks: 744 times
Was thanked: 950 time(s) in 905 post(s)
Thanks for these extra details.

Could you try running one of the tests that consistently produces this problem with the VS debugger attached? (i.e. use the debug option in NCrunch). Make sure you have your exception settings to break on ALL exceptions.

At the moment I'm strongly suspecting the virus scanner. Almost all the changes we've made in the v3.29 release were sitting at a higher level in the structure (i.e. around the IDE and NCrunch engine). We've touched very little down in the test execution system, certainly nothing that would cause such catastrophic instability. Hopefully the debug session can provide more detailed error information.
DanAtkinson
#5 Posted : Monday, August 12, 2019 12:58:56 PM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 10
Location: United Kingdom

Thanks: 2 times
I was unable to debug the very simple test I created. After it left the End Sub, Visual Studio froze. After about 10 minutes, I get the following dialog message:

Quote:
A fatal error has occurred and debugging needs to be terminated. For more details, please see the Microsoft Help and Support web site. HRESULT=0x80131c08. ErrorCode=0x0.

Remco
#6 Posted : Tuesday, August 13, 2019 12:29:33 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,722

Thanks: 744 times
Was thanked: 950 time(s) in 905 post(s)
Could you try deactivating the following two config settings at solution level (or for all projects in the solution) and confirm for me if this makes any difference in the error?

Instrument output assembly
Track file dependencies
1 user thanked Remco for this useful post.
DanAtkinson on 8/13/2019(UTC)
DanAtkinson
#7 Posted : Tuesday, August 13, 2019 9:23:06 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 10
Location: United Kingdom

Thanks: 2 times
Remco;13759 wrote:
Could you try deactivating the following two config settings at solution level (or for all projects in the solution) and confirm for me if this makes any difference in the error?

Instrument output assembly
Track file dependencies


Disabling 'Instrument output assembly' made no difference. However, when I disabled 'Track file dependencies', the tests immediately started passing. I re-enabled the first flag again and it worked fine. I was then able to switch on and off 'track file dependencies' and see the tests fail or pass.

I'm not sure what the next steps are for this. This appears to be a solution (thank you!) but I'm not sure if it's intended to be a permanent one, or what the impact of it will be.

Any advice here would be appreciated.
Remco
#8 Posted : Tuesday, August 13, 2019 11:50:34 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 5,722

Thanks: 744 times
Was thanked: 950 time(s) in 905 post(s)
DanAtkinson;13761 wrote:
[quote=Remco;13759]
I'm not sure what the next steps are for this. This appears to be a solution (thank you!) but I'm not sure if it's intended to be a permanent one, or what the impact of it will be.


Excellent. I can now guess with reasonable accuracy what is happening here.

This particular feature uses low level API hooks to track file system access calls made by your code during the test run. This data is useful for impact detection, as it means NCrunch can trigger your test when it makes use of any resource files that might change during your coding session.

Because the API hooks themselves work using memory hacks, I would expect the potential for their reliability to affected by the state of the environment they are running on (for example, O/S or .NET versions, virus scanners, other anomalies). It isn't really possible for us to establish exactly what is causing the hooks to destabilise the process without a way to reproduce the problem, and right now I see no way for us to do that. Regardless, the effective solution is simply to disable this setting on your machine. You probably won't notice it gone. On newer platforms (i.e .NET Core) we don't have this feature anyway, as we couldn't find a way to get the hooks to work.
1 user thanked Remco for this useful post.
DanAtkinson on 8/14/2019(UTC)
DanAtkinson
#9 Posted : Wednesday, August 14, 2019 9:02:14 AM(UTC)
Rank: Member

Groups: Registered
Joined: 2/8/2017(UTC)
Posts: 10
Location: United Kingdom

Thanks: 2 times
Okay, thank you. In that case, I consider this particular issue resolved and closed.

I saw in the documentation for "track file dependencies" that it didn't work in .NET Core and assumed that, as a result, the impact can't be that significant.

Thank you for your help in this matter.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.071 seconds.