We have a build which runs some specflow (v3.1.74 MSTest) and selenuim based UI tests, and runs them on a few grid nodes in teamcity. We had a problem recently where the build would report all tests as passing but then output this in the log:

[17:53:40]E: [Step 10/13] NCrunch result: TestFailure
[17:53:36]i: [Step 10/13] ##teamcity[buildProblem description='NCrunch result: TestFailure' timestamp='2020-03-12T17:53:36.849+0000']

No other exceptions were reported.

Having experienced some of this stuff before I know that usually this is caused by the _Fixture_ test that NCrunch creates for the setup for specflow failing, but I added logging around things happening in this and couldn't see anything untoward. Suspecting that maybe NCrunch would give a clue with more logging I turned on all logging. And the tests passed. 3 times in a row. Annoying. Turned logging back down and they started failing repeatedly again.

Previously we were on v3.24 I think so we decided to upgrade to the latest version before we asked for help, knowing that the first suggestion would be that. And that seemed to fix things, we had several days with no issues. Today, same problem. I have not tried turning verbose logging on yet, but a standard log file can be downloaded from here

On a previous thread for another issue you suggested a preview build of 4.4. When we updated we didn't use this version (I didn't realise that it had not yet made it into the main release), and we are considering testing with this version even though the symptoms for this issue are different.

Any suggestions on what we can do to identify this issue, or how we might work around it?

Thanks! yeah that shows which test is failing, though no exception details or anything, just a single line of output, but that at least gives us somewhere to start...

Are you able to reproduce this failure in any form using the NCrunch VS Client?

The reason I ask this is because I'm trying to establish if this is a reporting difference in the console tool itself. To my knowledge, it's not possible for a test to fail with no error information. If you can produce the error in the VS NCrunch client, it could indicate a potential problem in the adapter rather than in the manner of reporting.

Can you try setting your Log Verbosity to Detailed inside your build system, then tracing through the log to examine the execution of the failing test? It may be that the log yields information that isn't available in the TeamCity report.

Could you also confirm whether the test is marked as failing inside the NCrunch console tool's HTML report?