Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Tests permanently show as "running (first time run)"
Options
Previous Topic Next Topic
NeilMacMullen
#1 Posted : Wednesday, April 6, 2022 12:36:24 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 12/15/2016(UTC)
Posts: 55
Location: United Kingdom

Thanks: 29 times
Was thanked: 12 time(s) in 11 post(s)
I have some tests which are behaving oddly...
Screenshot of Ncrunch tests window

Status shows...
Screenshot of status window

These tests are "awkward" in that they rely on loading a (managed) C++ dll but they used to run fine in Ncrunch. It's not clear to me how long they've been broken (since the status icon just indicates that all tests are passing) but I'll try regressing the codebase to see if something I've done has triggered this.

In the meantime, are there any tips for figuring out what Ncrunch might be doing here?
Back to top
 |  Edit by user
Remco
#2 Posted : Wednesday, April 6, 2022 12:58:31 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this.

Can I safely assume that the tests stay this way forever (i.e. they never time out, even if left for 15+ minutes)?

Can you check the processing queue to see if the tasks involved have kicked up any errors?

One possibility I can think of is if the unmanaged code is causing the test process to hang in a position where the timeout enforcement is unable to terminate the run. If this is the case, it might be possible to identify the faulty code by attaching a debugger to the hung process, or running the test with a debugger attached and breaking into the process after a few minutes to check the stack.
Back to top
1 user thanked Remco for this useful post.
NeilMacMullen on 4/6/2022(UTC)
NeilMacMullen
#3 Posted : Wednesday, April 6, 2022 1:53:52 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 12/15/2016(UTC)
Posts: 55
Location: United Kingdom

Thanks: 29 times
Was thanked: 12 time(s) in 11 post(s)
Thanks Remco.

>Can I safely assume that the tests stay this way forever (i.e. they never time out, even if left for 15+ minutes)?

Correct

>Can you check the processing queue to see if the tasks involved have kicked up any errors?

All jobs in the Processing Queue window show "complete" including the "Build assembly" for TestProject.

>One possibility I can think of is if the unmanaged code is causing the test process to hang in a position

Sorry I should have noted that all of these tests run (and pass) if I run them under the resharper or VS test-runners so I don't think the process is crashing.

One other thing I've just noticed... if I right click on one of these tests and choose "run selected test in new task runner process" I see:

- Ncrunch status icon changes from 'N' to '1' with permanent spinner.
- the ProcessingQueue adds a new row which shows
>Run 1 test | Complete | TestProject
with a sub-row of
>[hollow circle icon] | Running | test-name

Seems odd that the parent Status is "complete" while the child is "running" ?
Back to top
 |  Edit by user
Remco
#4 Posted : Wednesday, April 6, 2022 11:19:18 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for these extra details. I wonder if this is a UI refresh issue. Do the tests clear from the Tests Window if you turn one of the filter options on and off?

Are you able to run the tests with a debugger attached?

Note that although the tests may work fine with a different runner, this doesn't completely exclude the hung thread/process issue. There are differences in the NCrunch environment that can cause tests to behave differently. Sometimes there may also be race conditions involved.
Back to top
 |  Edit by user
NeilMacMullen
#5 Posted : Thursday, April 7, 2022 12:50:00 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 12/15/2016(UTC)
Posts: 55
Location: United Kingdom

Thanks: 29 times
Was thanked: 12 time(s) in 11 post(s)
Changing the filter doesn't make a difference but I think you are right that the UI is out of step with the actual state. None of the listed processes that have a "running" sub-row are actually present. I did notice that there are some errors present though which probably are a clue....

As a side-issue, I'm looking at the Processing Queue list which has a list of tasks then an "output" window pane below the task list. Clicking on a row in the task list only sometimes updates the text in the output pane ... I was expecting it to show the output with a PID corresponding to the Process Id in the task row. E.g when I click on a row with Process Id of 35540, the output text still shows "[PID 21032.." and when I click on a row for a task where the test did actually run correctly, the output text pane still shows a previous error. Anyway, that said, I can now see these errors....

[PID:21032 13:31:25.4577 ?-4] System.BadImageFormatException: Could not load file or assembly 'TestProject, Culture=neutral, PublicKeyToken=null'. An attempt was made to load a program with an incorrect format.
File name: 'TestProject, Culture=neutral, PublicKeyToken=null'
at System.Reflection.RuntimeAssembly.InternalLoad(ObjectHandleOnStack assemblyName, ObjectHandleOnStack requestingAssembly, StackCrawlMarkHandle stackMark, Boolean throwOnFileNotFound, ObjectHandleOnStack assemblyLoadContext, ObjectHandleOnStack retAssembly)
at System.Reflection.RuntimeAssembly.InternalLoad(AssemblyName assemblyName, RuntimeAssembly requestingAssembly, StackCrawlMark& stackMark, Boolean throwOnFileNotFound, AssemblyLoadContext assemblyLoadContext)
at System.Runtime.Loader.AssemblyLoadContext.LoadFromAssemblyName(AssemblyName assemblyName)
at nCrunch.Common.ExecutionPlatform.LoadAssembly(FilePath assemblyFilePath)
at nCrunch.Module.MSTest.Integration.MSTestFrameworkRuntimeEnvironment.InitialiseFrameworkRuntimeEnvironment(TestEnvironmentConfig testEnvironmentConfig, DynamicProxy[] dynamicProxies)
at nCrunch.TestExecution.RemoteTaskRunner.<>c__DisplayClass20_0.<getInitialisedTestFrameworkEnvironment>b__0()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.TestExecution.RemoteTaskRunner.getInitialisedTestFrameworkEnvironment(TestFrameworkRuntimeEnvironmentReference executorReference, TestExecutionParameters parameters)
at nCrunch.TestExecution.RemoteTaskRunner.RunTests(TestFrameworkRuntimeEnvironmentReference executorReference, TestExecutionParameters parameters)

[PID:35540 13:31:30.1297 ?-4] System.Exception: Framework already initialised
at nCrunch.Module.MSTest.Integration.MSTestFrameworkRuntimeEnvironment.InitialiseFrameworkRuntimeEnvironment(TestEnvironmentConfig testEnvironmentConfig, DynamicProxy[] dynamicProxies)
at nCrunch.TestExecution.RemoteTaskRunner.<>c__DisplayClass20_0.<getInitialisedTestFrameworkEnvironment>b__0()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.TestExecution.RemoteTaskRunner.getInitialisedTestFrameworkEnvironment(TestFrameworkRuntimeEnvironmentReference executorReference, TestExecutionParameters parameters)
at nCrunch.TestExecution.RemoteTaskRunner.RunTests(TestFrameworkRuntimeEnvironmentReference executorReference, TestExecutionParameters parameters)
[PID:33684 13:31:30.1297 LocalTestExecutionTask-40] ERROR (Internal): System.Exception: Framework already initialised
at nCrunch.Module.MSTest.Integration.MSTestFrameworkRuntimeEnvironment.InitialiseFrameworkRuntimeEnvironment(TestEnvironmentConfig testEnvironmentConfig, DynamicProxy[] dynamicProxies)
at nCrunch.TestExecution.RemoteTaskRunner.<>c__DisplayClass20_0.<getInitialisedTestFrameworkEnvironment>b__0()
at nCrunch.Common.PerformanceTracking.PerfTracker.TrackActivity(String name, Action activity)
at nCrunch.TestExecution.RemoteTaskRunner.getInitialisedTestFrameworkEnvironment(TestFrameworkRuntimeEnvironmentReference executorReference, TestExecutionParameters parameters)
at nCrunch.TestExecution.RemoteTaskRunner.RunTests(TestFrameworkRuntimeEnvironmentReference executorReference, TestExecutionParameters parameters)

I suspect that for some reason the LoadDll for the managed C++ dll is failing - will check the "copy additional files" step....
Back to top
NeilMacMullen
#6 Posted : Thursday, April 7, 2022 4:15:55 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 12/15/2016(UTC)
Posts: 55
Location: United Kingdom

Thanks: 29 times
Was thanked: 12 time(s) in 11 post(s)
Looks like switching UseCpuArchitecture to x86 (from x64) clearing caches and and reloading ncrunch resolves the issue. :-)
Back to top
Remco
#7 Posted : Thursday, April 7, 2022 11:03:20 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
NeilMacMullen;16066 wrote:
Looks like switching UseCpuArchitecture to x86 (from x64) clearing caches and and reloading ncrunch resolves the issue. :-)


Excellent :)

We have a fix pending for the processing queue text pane sync issue. It doesn't update correctly when the task has no output. I might see what I can do about improving that error, ideally it should show in the Tests Window.
Back to top
1 user thanked Remco for this useful post.
NeilMacMullen on 4/8/2022(UTC)
NeilMacMullen
#8 Posted : Friday, April 8, 2022 9:40:22 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 12/15/2016(UTC)
Posts: 55
Location: United Kingdom

Thanks: 29 times
Was thanked: 12 time(s) in 11 post(s)
Thanks. Yes I think the most serious "issue" here is really just that the main Ncrunch icon color made it appear that all the tests were still running and passing; hence it was a couple of months before I noticed the problem (I usually filter to only failed tests in the test window). Hopefully that's a relatively easy fix... Anyway, keep up the good work, Ncrunch is indispensable :-)
Back to top
Remco
#9 Posted : Friday, April 8, 2022 11:40:11 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
NeilMacMullen;16068 wrote:
Thanks. Yes I think the most serious "issue" here is really just that the main Ncrunch icon color made it appear that all the tests were still running and passing; hence it was a couple of months before I noticed the problem (I usually filter to only failed tests in the test window). Hopefully that's a relatively easy fix... Anyway, keep up the good work, Ncrunch is indispensable :-)


I've reproduced the issue on my side and implemented a fix. It will be out with v4.12 :)
Back to top
1 user thanked Remco for this useful post.
NeilMacMullen on 4/11/2022(UTC)
NeilMacMullen
#10 Posted : Monday, April 11, 2022 1:18:12 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 12/15/2016(UTC)
Posts: 55
Location: United Kingdom

Thanks: 29 times
Was thanked: 12 time(s) in 11 post(s)
Super - look forward to it :-)
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.075 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download