Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
NCrunch Stuck/Hangs With 'Running (First time run)' For Most Tests In Project
Options
Previous Topic Next Topic
nko12
#1 Posted : Tuesday, April 14, 2020 12:35:58 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/13/2020(UTC)
Posts: 6
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
Setup:
- Windows 10 v1909
- VS 2017 + VS 2019 (version makes no difference)
- NCrunch 4.3.0.13 (reproduced in all versions I've tried, 3.29 (something like that) and up)
- MSTest v14
- TestFramework + TestFramework.Extensions v14

I have a test assembly that is using NSubstitute and FluentAssertions for test control. I am testing a number of Asp.Net related libraries. Currently, I have a little under 30 tests. Most of them are declared as

Code:
public async Task TestName()


Of these tests, most (but not all) of them are stuck with a status of "Running (First time run)". Code wise, there is nothing between the stuck and working tests that should lead to this issue. The problem reproduces regardless of NCrunch version and Visual Studio version.
- Reloading the assembly doesn't help.
- Killing NCrunch processes doesn't help.
- Uninstalling and reinstalling NCrunch doesn't help.
- Deleting the binary output directory that NCrunch automatically generates doesn't help.
- Debugging the stuck tests starts up the debugger, but it immediately closes before actually executing any lines of code.
- Using the build-in VS Unit Test runner executes and debugs the tests without any issues.

This is only reproducing in this particular project; in another (significantly larger) project, NCrunch is working perfectly.
Back to top
Remco
#2 Posted : Tuesday, April 14, 2020 1:22:06 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this issue.

It might be possible to retrieve additional information by setting your 'Log Verbosity' to 'Detailed' and turning on the 'Log to Output Window' setting, then restarting the engine and examining the output in the NCrunch VS debug text window.

Or you can submit a bug report (via the NCrunch menu), which will contain the log. If there's an error in there, I should be able to fish it out.

Is this something you're able to reproduce in a project/solution you can share with me? You can submit small code samples in ZIP form through the NCrunch contact form.
Back to top
nko12
#3 Posted : Tuesday, April 14, 2020 2:02:08 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/13/2020(UTC)
Posts: 6
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
Unfortunately no, there are too many internal dependencies the code relies on; you wouldn't be able to compile it.

Is there a way to upload the log output? There's way too much of it to post directly as a reply.
Back to top
Remco
#4 Posted : Tuesday, April 14, 2020 4:37:57 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
nko12;14605 wrote:

Is there a way to upload the log output? There's way too much of it to post directly as a reply.


The bug reporter automatically uploads the log output. I'd recommend sending one through.

It's often hard to reproduce problems like this using the log output alone ... Usually a sample project is required. But it never hurts to try.
Back to top
nko12
#5 Posted : Thursday, April 16, 2020 12:45:56 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/13/2020(UTC)
Posts: 6
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
Okay, I uploaded a bug report with the title "NCrunch Stuck/Hangs With 'Running (First time run)' For Most Tests In Project".

For additional context, I deleted all NCrunch related files in the project (so that it shows the setup dialog for the project), disabled parallelized test running and made test running manual, and then ran exactly one test. It still gets stuck in this way. I think something is happening that is preventing the outermost NCrunch threads from ever returning. Attempting to debug these tests results in the Debugging process starting and then immediately closing without doing anything more than loading some dlls (only debug output is the dlls that are loaded). Additionally, nothing happens if I try Right Click -> Advanced -> Reset selected test status.
Back to top
Remco
#6 Posted : Thursday, April 16, 2020 4:36:51 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for submitting the bug report. It looks like this is being caused by an assembly resolution error:

System.IO.FileNotFoundException: Could not load file or assembly 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' or one of its dependencies. The system cannot find the file specified.
File name: 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'
at System.ModuleHandle.ResolveType(RuntimeModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type)
at System.ModuleHandle.ResolveTypeHandleInternal(RuntimeModule module, Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(CustomAttributeRecord caRecord, MetadataImport scope, Assembly& lastAptcaOkAssembly, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, Object[] attributes, IList derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctor, Boolean& ctorHasParameters, Boolean& isVarArg)
at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeModule decoratedModule, Int32 decoratedMetadataToken, Int32 pcaCount, RuntimeType attributeFilterType, Boolean mustBeInheritable, IList derivedAttributes, Boolean isDecoratedTargetSecurityTransparent)
at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeType type, RuntimeType caType, Boolean inherit)
at nCrunch.TestExecution.Reflection.Clr.ClrType.getAttributesWithoutInheritance()
at nCrunch.TestExecution.Reflection.ReflectedType..(ReflectedType )
at System.Linq.Enumerable.<SelectManyIterator>d__17`2.MoveNext()
at System.Linq.Enumerable.WhereEnumerableIterator`1.MoveNext()
at nCrunch.TestExecution.Reflection.ReflectedType.GetValuesFromNamedAttributes[T](String[] attributeTypeNames, Func`2 getValueFunc)
at nCrunch.TestExecution.TimeoutValueReflector.(String )
at nCrunch.TestExecution.TimeoutValueReflector.GetTestTimeout(ClassMethodSignature classMethod)
at nCrunch.TestExecution.ExecutingTest.get_Timeout()
at nCrunch.TestExecution.TimeoutEnforcement.BaseTimeoutEnforcer.BeginEnforcement(ExecutingTest currentlyExecutingTest)
at nCrunch.TestExecution.TimeoutEnforcement.AbortingTimeoutEnforcer.BeginEnforcement(ExecutingTest currentlyExecutingTest)
at nCrunch.TestExecution.TimeoutEnforcement.AbortingWithTerminationFallbackTimeoutEnforcer.BeginEnforcement(ExecutingTest currentlyExecutingTest)
at nCrunch.Module.MSTest.Integration.MSTestListener.FixtureStarted(MSTestFixture fixture)
at nCrunch.Module.MSTest.Integration.MSTestRunner.runTestsWithinAssembly(IList`1 testsToRun, TestTaskOutput testOutput)
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackUnreliableActivity(String name, Action activity)
at nCrunch.Module.MSTest.Integration.MSTestRunner.RunTests(IList`1 testsToRun, TestTaskOutput testOutput)
at nCrunch.TestExecution.TimeoutEnforcement.AbortingTimeoutEnforcer.ExecuteWithPossibleEnforcement(Action codeToExecute)
at nCrunch.TestExecution.TimeoutEnforcement.TerminatingTimeoutEnforcer.ExecuteWithPossibleEnforcement(Action codeToExecute)
at nCrunch.TestExecution.TestExecutionMonitor.PerformMonitoredTestExecution(Action testExecutionAction)
at nCrunch.TestExecution.TestRunnerThread.()


How are you currently referencing the Microsoft.VisualStudio.QualityTools.UnitTestFramework assembly? Is this assembly installed in your GAC? Could you try converting any reference to point to a fixed assembly stored in your solution directory (i.e. copy the file in).
Back to top
nko12
#7 Posted : Monday, April 20, 2020 9:57:27 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/13/2020(UTC)
Posts: 6
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
It doesn't look like it's directly referenced in the project file. Actually, I don't think that assembly should be being referenced at all; it sounds like it's old. It's not in the GAC either after a CTRL+F through the output of gacutil -l.

My Project File does contain:
Code:
<Import Project="$(PkgMSTest_TestAdapter)\build\net45\MSTest.TestAdapter.props" Condition="'$(EnvironmentConfig)'!='' And Exists('$(PkgMSTest_TestAdapter)\build\net45\MSTest.TestAdapter.props')" />
<Import Project="$(PkgMSTest_TestAdapter)\build\net45\MSTest.TestAdapter.targets" Condition="'$(EnvironmentConfig)'!=''" />



but neither of these imports are trying to include that assembly.

Is it possible NCrunch is trying to pull it in itself?
Back to top
Remco
#8 Posted : Monday, April 20, 2020 11:51:37 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
nko12;14632 wrote:

Is it possible NCrunch is trying to pull it in itself?


This is definitely a possibility. It's generally assumed that your code would be relying on this assembly so that it can declare the tests using the MSTest attributes.

Somehow, your project must be referencing an MSTest framework assembly. Perhaps it's a different one. Could you use your code navigation on one of your MSTest attributes to find out which assembly is being referenced? This might yield some clues.
Back to top
nko12
#9 Posted : Tuesday, April 21, 2020 12:48:26 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/13/2020(UTC)
Posts: 6
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
Right clicking "Go to Definition" of one of the [TestMethodAttribute] tags on one of the tests that are stuck shows the attribute is in "Microsoft.VisualStudio.TestTools.UnitTesting", which is in the assembly "Microsoft.VisualStudio.TestPlatform.TestFramework.dll". This assembly is explicitly referenced in the project file, which is expected. There are also references to the assemblies "Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions.dll" which is in the same package and "Microsoft.VisualStudio.TestPlatform.MSTest.TestAdapter.dll", which is in a different package. But as both packages are used in the project file, all three of these assembly references are expected to be included.

In the "Object Browser", I cannot locate the assembly "Microsoft.VisualStudio.QualityTools.UnitTestFramework", even when set to "All Components".
Back to top
Remco
#10 Posted : Tuesday, April 21, 2020 1:26:50 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for sharing these extra details.

I've reviewed the code involved and cross checked this with the stack trace you've provided. It looks to me like your compiled test project assembly has a binary reference to 'Microsoft.VisualStudio.QualityTools.UnitTestFramework, Version=10.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a' inside its CIL structure. I'm not sure how this is possible. Are you using any tooling or nuget package references that might somehow introduce this? Are you able to examine the compiled assembly in ILDASM or ilspy to inspect its structure?

I could not find any code in NCrunch that declares a reference to the missing assembly or would introduce it to your codebase.
Back to top
nko12
#11 Posted : Thursday, May 14, 2020 9:31:13 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 4/13/2020(UTC)
Posts: 6
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
Sorry for a late reply, putting out other fires with this project, now back to testing.

You're right; my assembly doesn't have this reference, but one of the other assemblies (assembly C) does. Assembly C contains code to help test usage of its library like code in assembly B within my own production assembly.

I can't easily reference the Microsoft.VisualStudio.QualityTools.UnitTestFramework, so I just removed the test code that used Assembly C; and then removed Assembly C entirely. The test code is not super required right now. After removing Assembly C, NCrunch is working perfectly again.

If possible, it would be great if this assembly loading issue could be more easily be identified by the NCrunch UI, though I understand if this is not possible. Anyway, thank you for your assistance, I greatly appreciate it!
Back to top
1 user thanked nko12 for this useful post.
Remco on 5/14/2020(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.093 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download