no stack traces when test fails

When a test fails, this is all I see:

External image

(http://screencast.com/t/3YlKTb53RqF)

In the Nunit gui, I'd also get the stack trace, so I'd have some shot at telling what went wrong. Is that a setting, or do I just not know where to look?

Thanks,
Tom

Hi Tom,

In this case, the test failed because its parent fixture failed on setup. If you choose the parent fixture in the tree view, it should show you the stack trace.

NCrunch has a strong distinction between fixtures and tests, as mechanically they behave very differently. To show the stack trace of a fixture failure on its child tests could create confusion, as NCrunch does actually allow different tests under a fixture to have been executed against different versions of a codebase.

Cheers,

Remco

Remco wrote:Hi Tom,

In this case, the test failed because its parent fixture failed on setup. If you choose the parent fixture in the tree view, it should show you the stack trace.

NCrunch has a strong distinction between fixtures and tests, as mechanically they behave very differently. To show the stack trace of a fixture failure on its child tests could create confusion, as NCrunch does actually allow different tests under a fixture to have been executed against different versions of a codebase.

Cheers,

Remco

Gotcha. Unfortunately, I don't see the parent fixture in TreeView. My tests mostly inherit from a base class that has the TestFixture attribute, so that they can also all inherit the datarollback functionality from the TeamAgile library. Unfortunately the setup methods on that base class do not seem to appear.

In this situation, the parent fixture will be the derived class you've created that is shown as the owner of the test inside the tree view. When you are inheriting from another fixture, you are effectively creating a new fixture with a different name that inherits many of the properties of the base class. There is much complexity within NCrunch and other test runners to deal with situations like this, as inheritance is a very common way to reuse code between tests. The [TestFixture] attribute itself in most situations has little consequence, as from NUnit 2.5 it is possible for tests to be exposed without it.

As far as NCrunch is concerned, the parent fixture will always be the fixture that is shown above the test in the test tree view. If the test throws an exception in a TestFixtureSetUp, you should always see the stack trace on this node ... if not, then it's definitely a bug! :)


Cheers,

Remco

Remco wrote:
As far as NCrunch is concerned, the parent fixture will always be the fixture that is shown above the test in the test tree view. If the test throws an exception in a TestFixtureSetUp, you should always see the stack trace on this node ... if not, then it's definitely a bug! :)

I guess I'm reporting a bug then - this is the test fixture that contains the failing test:

External image
(http://screencast.com/t/c6i5Rh18Na)

Then, the parent of *that* node is just the assembly/project itself, which has a red question mark next to it, but no messages whatsoever.

Gotcha. Yes - this is a bug. NCrunch doesn't deal with issues in SetUpFixtures too well because they can't associate with any individual test. The most intuitive way to handle them would be to write their results back into the project itself (as you've described). This is something I'll need to look at :)

You may be able to get more information about why the test failed if you set your logging to a more verbose level (i.e. Detailed), then find the task that was used to execute your tests in the processing queue. It will be a bit noisy, but the answer should be in there.

Remco wrote:
You may be able to get more information about why the test failed if you set your logging to a more verbose level (i.e. Detailed), then find the task that was used to execute your tests in the processing queue. It will be a bit noisy, but the answer should be in there.

Getting back to this finally... I really want ncrunch to work since it seems so useful.

When I run these tests in the debugger, using the ncrunch built-in feature, they still error out and I don't can't get a breakpoint to fire. Even when I put the breakpoint on the first line of the method with the [TestFixtureSetUp] attribute, that line never catches. ncrunch just errors out, and tells me nothing except "Parent SetUp failed in TestSetup"

Is there any better way to debug? Also, when you say "set your logging" are you referring to logging in my code, or in ncrunch itself? I poked around each of the menus, as well as the button toolbar, as well as googled "ncrunch logging" and didn't see any sign of a setting for logging.

Thanks!

Hi, thanks for coming back to this! My suspicion is that there is something in your test environment that is causing the test runner to bomb out. The easiest way to identify the cause of this problem is to turn on the logging in NCrunch. You should find a setting to control this under your NCrunch configuration, in 'All Solutions'. There is an option called 'Log Verbosity' that should be set to detailed. When this has been set, you should see more information (and possibly an exception) appear if you click on one of the testing tasks inside the processing queue window (after you hit the Reset button that is).

If you can share the details of the exception, I may be able to help you find out why the test runner is failing.

Remco wrote:Hi, thanks for coming back to this! My suspicion is that there is something in your test environment that is causing the test runner to bomb out. The easiest way to identify the cause of this problem is to turn on the logging in NCrunch. You should find a setting to control this under your NCrunch configuration, in 'All Solutions'. There is an option called 'Log Verbosity' that should be set to detailed. When this has been set, you should see more information (and possibly an exception) appear if you click on one of the testing tasks inside the processing queue window (after you hit the Reset button that is).

If you can share the details of the exception, I may be able to help you find out why the test runner is failing.

I enabled the log verbosity at detailed, and hit the reset button, but I see no visible differences. The entire Ncrunch output window is empty ( http://screencast.com/t/wcCw1vSC ) and the errors on my individual tests still just say "Parent SetUp failed in TestSetup" and "TestFixtureSetUp failed in TestSetup" and on the top-level test, it has no output at all.

Would you be able to submit a bug report after the error happens? I'll then be able to take a look at the log file this sends through.