In hopes of being able to use NCrunch with xUnit v2 I downloaded NCrunch 1.11.0.10 today to try out the xUnit 2 support.
Two big problems:
1. I am unable to debug tests. Right-click debugging from the NCrunch Tests window appears to do nothing and right clicking on a code coverage line and choosing break into or debug also appears to do nothing.
2. While I do see the coverage X marker showing me which line fails, clicking on it does nothing and right clicking on it and choosing "show exception details" does nothing. If I look at the NCrunch Tests window I see the following code block and clicking on the link to source/line works correctly. I can only assume that this is a problem with NCrunch correctly interpreting xUnit output?
Xunit.Sdk.EqualException: Assert.Equal() Failure
Expected: 1
Actual: 2
at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) in c:\TeamCity\buildAgent\work\74856245f07a90f0\src\xunit.assert\Asserts\EqualityAsserts.cs:line 35
at Xunit.Assert.Equal[T](T expected, T actual) in c:\TeamCity\buildAgent\work\74856245f07a90f0\src\xunit.assert\Asserts\EqualityAsserts.cs:line 19
at Test.ClassTests.test_method() in D:\Source\Test\Test\Class.cs:line 13
On the plus side, xUnit theories now show up, which is really what I have been waiting for in nCrunch/xUnit integration. Assuming the above two bugs are fixed (or workarounds provided) then I think I'll be very happy with nCrunch 1.11.
Here is the class I have been testing with for documentation of the above bugs:
public class ClassTests
{
[Fact]
public void test_method()
{
Assert.Equal(1, 1);
Assert.Equal(2, 2);
Assert.Equal(1, 2);
}
[Fact]
public void when_()
{
Assert.Equal(1, 1);
}
[Theory]
[InlineData(true)]
[InlineData(false)]
public void theory(Boolean value)
{
Assert.True(value);
}
Thanks for sharing these issues. NCrunch 2.11 is an early build containing a new feature that allows the engine to run outside the IDE, communicating with devenv.exe using IPC. The problems you've described (UI options not working) suggest that the IPC communication isn't working correctly. I'm wondering if you would be able to help me with a bit more information so that I can resolve these problems promptly. Do you mind if I ask a few specific questions about this behaviour?
1. Would you be able to submit a bug report directly after you've had a UI option fail to work correctly?
2. Do you experience this only in a specific solution, or others also? I.e. if you build a sample test solution with a single test, does the problem occur?
3. Do you still experience this problem if you set the 'Engine hosting strategy' global configuration setting to 'HostInIDE'?
4. Which NCrunch UI windows do you usually keep open while you work? Does closing these windows, shutting down the IDE, then re-opening the IDE and using NCrunch with the windows closed make any difference for you?
5. After the UI problems start appearing, try disabling the engine using the disable option in the main menu. Does the engine shut down promptly, or does it seem to hang forever and never disable?
1. Done.
2. Initially experienced it in a project I am working on. Switched over to a test solution/project (class library I keep handy for reproducing bugs for report) to reproduce the problem. Deleted all but one test in test project/solution and the problem persists.
3. Changing the Engine hosting strategy to "HostiInIDE" does not resolve the issue. My other answers are with this setting set to HostInIDE. I restarted VS after changing this setting and before retesting.
4. I usually have the NCrunch Tests window open. I followed the instructions (closed it, restarted IDE, tested right-click UI) and the problem persisted.
5. Disabling NCrunch does not appear to hang. Code coverage markers go away immediately and I am able to re-enable the engine and they pop back up a moment or two later.
The update appears to have fixed "break into a debugger" but it does not appear to have done anything for "show coverage for covering tests" or "show exception details".
Note: I can hover my mouse over the red X and see partial exception details, but left clicking does not do anything despite what the tooltip shows.