Project:ProjectName.Tests
Result:Failed
Server used:GridNodeName
<Show test output text>
Clicking "test output text" displays the output from a setup (2 linesconsole output).
Clicking on "TheActualTestName" reveals an assertion failure.
Why is it reporting 2 test failures? Is it so the user can troubleshoot the log output from the common setup method in WhenSomethingFixture base class or is it a bug?
*Edit:
Furthermore, in the test result view in TeamCity (not via AllResult.Html), it only reports the "ProjectName.Tests.SubTests.MoreTests.WhenSomethingFixture.*" strings for failed tests, and when expanding they all seem to only contain
------- Stdout: -------
text
Which provides neither what test is failing nor any valuable info of what is going wrong.
Is the integration with e.g. TeamCity planned to be improved?
The parent fixture failure is being reported because of the child failure. Most test frameworks do this implicitly - where they don't, NCrunch will do it for them. A fixture is always considered to be failing if any of its child tests are failing. If a fixture itself is the source of the failure, all child tests also fail with it.
The error you've described in the TeamCity output isn't by design. I'll need to have a look at this to see whether this can be fixed.
I can confirm this is also happening for us. We just began using NCrunch in conjunction with TeamCity. The ability to run our tests in parallel is using the distributed processing of NCrunch is really, really helpful. Unfortunately, the failures are not yet, but I think I might see the problem? Looks like a closing quote is missing in the ##teamcity service message. You can see it in the build log. I hope this helps!
From the build log:
[15:52:51][Step 9/9] Behaviors
[15:52:51][Behaviors] SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory
[15:52:51]
[SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] [Test Output]
text
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] Given An open inventory with a room exists
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] Name:.ASPXAUTH | Domain: .xxxxxxxxxxxx | Expiry: | Path: /
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] -> done: TakingAnInventorySteps.GivenAnOpenInventoryWithARoomExists() (14.8s)
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] And The user has selected that Room from an existing Open Inventory
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] -> done: TakingAnInventorySteps.GivenTheUserHasSelectedARoomFromAnExistingOpenInventory() (3.7s)
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] When The user enters an Asset ID that was not previously created
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] -> done: TakingAnInventorySteps.WhenTheUserEntersAnAssetIdThatWasNotPreviouslyCreated() (0.5s)
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] Then The system should allow the user to enter data in the Asset Quick Add fields
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] -> error: Timed out after 10 seconds
blah, blah, blah...
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] ##teamcity[testFailed name='SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory' message='OpenQA.Selenium.WebDriverTimeoutException : Timed out after 10 seconds' details='OpenQA.Selenium.WebDriverTimeoutException : Timed out after 10 seconds|n at OpenQA.Selenium.Support.UI.DefaultWait`1.ThrowTimeoutException(String exceptionMessage, Exception lastException)|n at OpenQA.Selenium.Support.UI.DefaultWait`1.Until[TResult]
[15:52:53]
[SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] Value should end with "'"
Valid property list format is (name( )*=( )*'escaped_value'( )*)* where escape symbol is "|"
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory] (Func`2 condition)|n at SST.AssetTracking.Tests.Utilities.Selenium.Pages.Inventory.InventoryModule.QuickAddAsset(String newAssetId) in C:\TcBuilds\29ebf5476a185798\AssetTracking\Application\AssetTracking.Web\TestUtilities\Selenium\Pages\Inventory\InventoryModule.cs:line 208|n at SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventorySteps.ThenTheSystemShouldAllowTheUserToEnterDataInTheAssetQuickAddFields() in C:\TcBuilds\29ebf5476a185798\AssetTracking\Application\AssetTracking.Web\Behaviors\Inventory\TakeInventory\TakingAnInventorySteps.cs:line 187|n at lambda_method(Closure , IContextManager )|n at TechTalk.SpecFlow.Bindings.BindingInvoker.InvokeBinding(IBinding binding, IContextManager contextManager, Object[] arguments, ITestTracer testTracer, TimeSpan& duration)|n at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStepMatch(BindingMatch match, Object[] arguments)|n at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.ExecuteStep(StepInstance stepInstance)|n at TechTalk.SpecFlow.Infrastructure.TestExecutionEngine.OnAfterLastStep()|n at TechTalk.SpecFlow.TestRunner.CollectScenarioErrors()|n at SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.ScenarioCleanup() in C:\TcBuilds\29ebf5476a185798\AssetTracking\Application\AssetTracking.Web\Behaviors\Inventory\TakeInventory\TakingAnInventory.feature.cs:line 0|n at SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.AddingNewAssetsDuringInventory() in C:\TcBuilds\29ebf5476a185798\AssetTracking\Application\AssetTracking.Web\Behaviors\Inventory\TakeInventory\TakingAnInventory.feature:line 12']
[15:52:51][Behaviors] SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.*
[15:52:51]
[SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.*] [Test Output]
text
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.*] One or more child tests had errors
[15:52:51][SST.AssetTracking.Behaviors.Inventory.TakeInventory.TakingAnInventoryFeature.*]
[15:52:51][Step 9/9] [15:52:51.9713-Core-31] Sending processing instructions to node SELTESTBOX03 for 1 tasks
[15:53:14][Step 9/9] [15:53:14.6418-Core-36] Grid node SELTESTBOX02 reports task completed: [LocalTestExecutionTask: [SnapshotComponent: Behaviors, 1, 16553355], 1 test(s), NUnit, ProcessingSucceeded, SELTESTBOX02]
It looks like TeamCity is struggling a bit with the closing brace ']' inside the exception message.
I've done some more experimentation on this, and it seems that the presence of this brace inside an output message will cause TeamCity to create a bit of a mess. It seems these braces can't be escaped, so I'll implement a workaround in the full 2.11 release that will replace them with rounded brackets where NCrunch encounters them. This should probably an effective (if slightly untidy) solution to the problem you've encountered.