NUnit3 / Teamcity TestExecutionException: Cannot start monitoring test...

Hi,

I am working on a project with about 1500 Nunit tests and use Ncrunch on both my Dev machine and my Teamcity server. Since updating NUnit from 2.6.3 to version 3.12.0, I am having problems with my unit tests on the TeamCity Server.

The tests are still run and all are passing, however, I get lots of build errors such as this one:

nCrunch.TestExecution.TestExecutionException: Cannot start monitoring test 'Zeiterfassung.UnitTests.Model.HourlyRates.HourlyRatesViewModelTest.' - test Zeiterfassung.UnitTests.Model.HourlyRates.HourlyRateSummaryTests. is still being monitored
at nCrunch.TestExecution.TestExecutionMonitor.StartMonitoringTest(ExecutingTest test)
at nCrunch.TestExecution.ExecutingTest.Start()
at nCrunch.Module.NUnit3.Integration.NUnit3Listener.suiteStarted(NUnit3Message message)
at nCrunch.Module.NUnit3.Integration.NUnit3Listener.RaiseCallbackEvent(String eventArgument)

As a result, my Build is marked as failed in Teamcity.

I have no problems running the same tests in Visual Studio 2019 with NCrunch. The problems only occur in TeamCity.

Please let me know how to fix this.

Thanks,

Adrian

Just in case it matters, this is the command from my TeamCity NCrunch build step:

"%system.teamcity.build.checkoutDir%\MySolutionName.sln" /c c:\globalconfig.crunch.v3.xml /o "%system.teamcity.build.checkoutDir%\NCrunchResults" -NCrunchCacheStoragePath C:\NCrunchTestRunnerCache

Thanks,

Adrian

Hi Adrian,

Thanks for sharing this issue.

This error appears when the sequence of events provided by NUnit is inconsistent. NUnit did not send a completion message for the previous fixture before it sent a message saying that the next test had started its execution (invalid sequence).

Are you using any NUnit plugins or modifications of any kind?

Do you have parallel execution turned on for NUnit? Note that there are several ways to do this (check them all).

Can you get a capture of the test run containing the error with LogVerbosity set to Detailed? You can submit this via the NCrunch contact form.

Hi Remco,

thanks for the quick reply. The issue was that I had marked my test fixture base with [Parallelizable(ParallelScope.All)] . I thought Once I removed this, the problem on Teamcity went away. Naturally, my tests are not parallelized anymore when I use a different test runner, but usually Ncrunch is stable enough so that I don't have to do that.

Thanks again for your help!

Best Regards,

Adrian

P.S.: The new "Optimized" instrumentation mode is great! Well worth getting the update just for this feature :-)

Thanks for confirming that this was the problem!

I'm glad the new instrumentation mode is working well for you. I look forward to making this the default mode of operation in future :)

Hi Remco,

unfortunately the problem has reappeared, though I am not using the Parallelizable attribute. The error message is almost the same:

nCrunch.TestExecution.TestExecutionException: Cannot start monitoring test 'Zeiterfassung.Tests.Controllers.TimeEntrySearchControllerTest.' - test Zeiterfassung.Tests.Business.ProjectsServiceTest. is still being monitored

Unfortunately this caused my CI build to fail, even though my test are all passing. I changed my build step so that it uses -LogVerbosity Detailed, but once I did that the problem did not reappear in the next test run.

This is quite annoying since it causes my CI builds to intermittently fail even when all test are in fact passing. Is there any way to suppress this error so that the build is not marked as failed in my CI server?

Thanks,

Adrian

Hi Adrian,

Unfortunately, suppressing this issue won't help here, as when tests results are reported out of order, NCrunch has no way to make sense of the results. The error invalidates the entire test run. You need to find out why NUnit is running your tests in parallel.

Make sure you are using the latest version of NUnit. Search your entire codebase for ParallelizableAttribute and ParallelScope. See here for a detailed guide on the configuration options available for this under NUnit, and make sure you don't have this configured. To my present knowledge, NUnit parallelisation is the only thing that can cause the error you've encountered.

If you're able to isolate the issue in a sample solution you can share with me, I'll be happy to trace the engine through it to understand what is going wrong.