Command line wont run tests in parallel

We have had success getting tests to run in parallel via the VS tool, but on the command line it will only ever run 1 test at a time.

This is despite using the same configuration file.

Do you know what causes the difference in behaviour ?

Hi, thanks for sharing this issue.

How are you identifying that it will only run one test at a time? Is this done by analysing the log file?

Is the tool being run on the same machine being used for development, or are you using it on a build server?

Hi

The tool is being used on the dev machine as a POC, we will be looking to use on the build server though.

We can see only one is being run at a time as we only see one executing (these are selenium tests), when using via visual studio we see 4 execute at once.

Kind regards

Thanks. Is there any chance you could capture a log from the execution run? If I can take a look through it, I might be able to work out why the tool is behaving in this way.

Try running:

ncrunch.exe -LogVerbosity Detailed > logfile.txt

You can then compress the logfile.txt and submit it through the contact form and I'll see what I can do.

Sorry for the delay, please see below:






NCrunch Console Tool v2.16.0.13
Copyright © 2010-2015 Remco Software Ltd
Usage of this tool is permitted only under the terms described in License.rtf



The evaluation period for this product will end in 30 days

[15:59:42.9577-Core-6] Loading NCrunch modules
[15:59:42.9627-Core-6] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\nCrunch.Module.Gallio.crunchmodule
[15:59:42.9742-Core-6] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\nCrunch.Module.MSpec.crunchmodule
[15:59:42.9782-Core-6] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\nCrunch.Module.MSTest.crunchmodule
[15:59:42.9822-Core-6] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\nCrunch.Module.NUnit.crunchmodule
[15:59:42.9862-Core-6] Loading module: C:\Program Files (x86)\Remco Software\NCrunch Console Tool\nCrunch.Module.XUnit2.crunchmodule
[15:59:43.0552-Core-6] Loading project: C:\git\A1Tests\TestCompany.A1.AqT.SmokeTests.Production\TestCompany.A1.AqT.smoketests.production.csproj
[15:59:46.0113-Core-6] Framework description manager has loaded data for 5 test frameworks
[15:59:46.0838-Core-6] Loading project: C:\git\A1Tests\TestCompany.A1.Domain\TestCompany.A1.Domain\TestCompany.A1.Domain.csproj
[15:59:46.4774-Core-6] Loading project: C:\git\A1Tests\TestCompany.A1.AqT.Steps\TestCompany.A1.AqT.Steps.csproj
[15:59:46.5454-Core-6] ERROR (Load): Error occurred during load of component at C:\git\A1Tests\TestCompany.A1.AqT.Steps\TestCompany.A1.AqT.Steps.csproj: System.Exception: An exception was thrown in the remote environment: Sys
tem.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation. ---> nCrunch.Common.UserException: Errors occurred while trying to load the project file:
This project references NuGet package(s) that are missing on this computer. Enable NuGet Package Restore to download them. For more information, see http://go.microsoft.com/fwlink/?LinkID=322105. The missing file i
s C:\git\A1Tests\TestCompany.A1.AqT.SmokeTests.Production\\.nuget\NuGet.targets.
at nCrunch.Compiler.RemoteBuildRunner.☻(FilePath ☻, ♠ ♥, String ♣, String, DirectoryPath ♠, String ♫)
at nCrunch.Compiler.RemoteBuildRunner.AnalyseComponentBuild(FilePath projectFilePath, BuildXml buildXml, String useBuildConfiguration, String useBuildPlatform, DirectoryPath solutionDir, String solutionName, List
`1 importExpressionsToEvaluate, ProcessorArchitecture buildProcessProcessorArchitecture, DirectoryPath workspaceBasePath)
--- End of inner exception stack trace ---
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at nCrunch.TaskRunner.Ipc.IpcMessageProcessor.☻(CallMethodMessage ☻)
at nCrunch.TaskRunner.Ipc.IpcMessageProcessor.ProcessMessageReturningResult(Byte[] data)
at nCrunch.TaskRunner.Ipc.RemoteInstance.☻[T](Byte[] ☻)
at nCrunch.TaskRunner.Ipc.RemoteInstance.Invoke(IMessage msg)
at System.Runtime.Remoting.Proxies.RealProxy.PrivateInvoke(MessageData& msgData, Int32 type)
at nCrunch.Compiler.IRemoteBuildRunner.AnalyseComponentBuild(FilePath projectFilePath, BuildXml buildXml, String useBuildConfiguration, String useBuildPlatform, DirectoryPath solutionDir, String solutionName, Lis
t`1 importExpressionsToEvaluate, ProcessorArchitecture buildProcessProcessorArchitecture, DirectoryPath workspaceBasePath)
at nCrunch.Core.BuildManagement.BuildProcessLauncher.♥.☻(IRemoteBuildRunner ☻)
at nCrunch.Core.BuildManagement.BuildProcessLauncher.☻(Action`1 ☻, ProcessorArchitecture ♥, GridClientId ♣, IBuildableProject, IList`1 ♠)
at nCrunch.Core.BuildManagement.BuildProcessLauncher.AnalyseComponentBuildInExternalProcess(FilePath projectFilePath, BuildXml buildXml, String useBuildConfiguration, String useBuildPlatform, DirectoryPath soluti
onDir, String solutionName, List`1 importExpressionsToEvaluate, ProcessorArchitecture processorArchitecture, VisualStudioVersion vsVersion, IList`1 customEnvironmentVariables)
at nCrunch.Client.ComponentLoader.SnapshotComponentFactory.☻(ProcessorArchitecture ☻)
at nCrunch.Client.ComponentLoader.SnapshotComponentFactory.CreateSnapshotComponentFromXml(FilePath projectFilePath, BuildXml projectXml, FilePath solutionFilePath, String[] additionalFilesToIncludeAtSolutionLevel
, Boolean isLoadedFromFile, ISnapshotConfig snapshotConfig, VisualStudioVersion vsVersion)

[15:59:46.5999-Core-6] All projects have been loaded
[15:59:47.1975-LocalBuildTask-5] Building new workspace C:\Users\hrobson\AppData\Local\NCrunch\9872\1 for component TestCompany.A1.Domain in context of grid client (local)
[15:59:48.4868-LocalBuildTask-5] Now building TestCompany.A1.Domain
[15:59:51.3689-LocalBuildTask-5] Build was successful for C:\Users\hrobson\AppData\Local\NCrunch\9872\1\TestCompany.A1.Domain\TestCompany.A1.Domain\bin\Debug\TestCompany.A1.Domain.dll
[15:59:51.4134-LocalBuildTask-5] Building new workspace C:\Users\hrobson\AppData\Local\NCrunch\9872\2 for component TestCompany.A1.AqT.smoketests.production in context of grid client (local)
[15:59:51.6004-LocalBuildTask-5] Now building TestCompany.A1.AqT.smoketests.production
[15:59:52.7457-LocalBuildTask-5] Build was successful for C:\Users\hrobson\AppData\Local\NCrunch\9872\2\TestCompany.A1.AqT.SmokeTests.Production\bin\Debug\TestCompany.A1.AqT.SmokeTests.Production.dll
[15:59:52.9692-LocalAnalysisTask-6] Calling into task runner to analyse target assembly: C:\Users\hrobson\AppData\Local\NCrunch\9872\2\TestCompany.A1.AqT.SmokeTests.Production\bin\Debug\TestCompany.A1.AqT.SmokeTests.Productio
n.dll
[15:59:53.2307-Core-5] Queuing 73 tests for passive execution
[15:59:53.2908-LocalTestExecutionTask-8] Calling into task runner to run 1 tests:
TestCompany.A1.AqT.SmokeTests.Production._09_Exports.INVOICEEXPORTMarkedForExportFeature._262535INVOICEEXPORTMarkedForExport
[16:01:00.0241-Core-13] Successfully completed execution of 1 tests on local machine
[16:01:00.0771-LocalTestExecutionTask-8] Calling into task runner to run 8 tests:
TestCompany.A1.AqT.SmokeTests.Production._03_Delivery_GRN.CheckingForPendingDeliveryFeature._139861DELIVERYRECEIPTINGCheckingForThePendingDelivery("\"C0154 Queen Eleanor (Test) > Stock\"","\"DrinksSupplier UK\"","\"2\""
,"\"Black Sheep Bitter Cask, 9GAL\"",null)
TestCompany.A1.AqT.SmokeTests.Production._03_Delivery_GRN.EditingDeliveryNoteWithFullNumericNumberOrDecimalNumberFeature._151544151566DELIVERYRECEIPTINGEditingADeliveryNoteWithAFullNumericNumberDecimalNumber("\"C01
54 Queen Eleanor (Test) > Stock\"","\"DrinksSupplier UK\"","\"2\"","\"Black Sheep Bitter Cask, 9GAL\"","\"0.5\"","\"Not on Delivery\"","\"Testing quantity field\"",null)
TestCompany.A1.AqT.SmokeTests.Production._03_Delivery_GRN.EditingDeliveryNoteWithFullNumericNumberOrDecimalNumberFeature._151544151566DELIVERYRECEIPTINGEditingADeliveryNoteWithAFullNumericNumberDecimalNumber("\"C01
54 Queen Eleanor (Test) > Stock\"","\"DrinksSupplier UK\"","\"2\"","\"Black Sheep Bitter Cask, 9GAL\"","\"1\"","\"Incorrect Delivery Quantity\"","\"Testing quantity field\"",null)
TestCompany.A1.AqT.SmokeTests.Production._05_SupplierReturns.SUPPLIERRETURNCreateACreditRequestFeature._46096SUPPLIERRETURNCreateACreditRequest
TestCompany.A1.AqT.SmokeTests.Production._06_Transfers.CreateSiteTransferByProductGFNRCostCenterFeature._38155CreateSiteTransferByProduct_GFNRCostCenter("\"Product\"","\"GNFR\"","\"LIGHT EQUIPMENT\"","\"* All\"","\
"5\"","\"A2 POSTER QUIZ NIGHT MONDAY (SGL)\"",null)
TestCompany.A1.AqT.SmokeTests.Production._08_StockManagement.StockCount.CheckStockCountPageFeature._118400STOCKCOUNTCheckStockCountPage("\"C0600 Adam and Eve (Test) > Stock\"",null)
TestCompany.A1.AqT.SmokeTests.Production._08_StockManagement.UnitAccountingFigure.RunAccountFigureReportFeature._253778STOCKMANAGEMENTRunAccountFigureReport("\"Tolouse Midland > Coventry\"",null)
TestCompany.A1.AqT.SmokeTests.Production._11_Others.FORCEDDROPS_CentralOrderAdjustmentFeature._261098FORCEDDROPS_CentralOrderAdjustment

Thanks for sharing this.

It looks like this solution hasn't had its NuGet references restored prior to being run by NCrunch. Note that the NCrunch console tool won't do this for you right now, although this would make sense as a feature request. If you're running this on a build server, try running MSBuild over the solution prior to invoking the console tool.

Unfortunately there wasn't enough of a log here for me to understand why you haven't been seeing any parallel execution. This forum isn't really a good place to drop a log, as logs can be very big. It would be better to zip it up and send it through the contact form.