Hi, thanks for posting. Answers are below:
scjones;14849 wrote:
1) Why is there a test reported for each "fixture" class? How is that information captured - is it when the first test (or any test) is run within that fixture, or is it manually discovered and executed first before any other test is run?
This is due to the abstractions that exist in the reporting structure of NUnit (and other frameworks too).
When we integrate with a framework, we essentially interrogate its code for information on the test assembly, so that we can build a list of tests. The data the framework provides to us conforms with a basic hierarchical structure that contains no specific detail on the attributes being used to specify the tests, just the basic form and the various names involved. Under NUnit, all tests must reside under a fixture, and the fixture itself is an important object that can contain the same sorts of data as any other test within the suite (for example, a pass/fail result, code coverage data and trace output).
As with tests, we build fixtures in response to the structural data provided by NUnit during discovery. We then populate these fixtures with results when the tests are run. The NUnit results output contains separate elements for these 'fixture tests' including pass/fail results, exception messages, etc.
scjones;14849 wrote:
2) If there are no [Setup] or [OneTimeSetup] methods within the class, with is the "fixture" class still reported in the count or test failures?
This is due to the reporting limitations of Team City. At the time we implemented our integration with TC, we didn't have a way to report the fixtures separately from the tests (TC didn't seem to have a concept capturing them that we could use). Because the fixture results are just as important as the test results and we couldn't afford for them to be excluded from the run, we simply report them as virtual tests instead. TC therefore adds them to its internal count which is then reported outside of our control.
As earlier described, we don't have a safe way to assume whether a fixture is likely to have a OneTimeSetup without sidestepping the abstractions in place within NUnit (which would be terrible for forwards compatibility). Thus we cannot safely exclude fixtures from the result set without the risk of swallowing relevant results.
scjones;14849 wrote:
3) Can we please have a way to disable/filter the _Fixture_ tests displayed in Team City, as for us I don't believe we will ever have fixture-level test failures and this is really causing us an issue.
This is probably a reasonable thing to request, and you are welcome to make a request for it on
uservoice if you like.
However, you should be aware that there may be unintended consequences to doing this. Removing the fixtures from the TC result set can cause you to lose relevant error information if one of your fixtures fails inside a OneTimeSetup. Currently under NUnit the child tests will fail sympathetically if their parent fixture fails, but they do not contain the source exception details.