Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

NCrunch 1.11 and XUnit 2
Micah71381
#1 Posted : Saturday, December 27, 2014 12:40:13 AM(UTC)
Rank: Member

Groups: Registered
Joined: 10/18/2013(UTC)
Posts: 27
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
In hopes of being able to use NCrunch with xUnit v2 I downloaded NCrunch 1.11.0.10 today to try out the xUnit 2 support.

Two big problems:

1. I am unable to debug tests. Right-click debugging from the NCrunch Tests window appears to do nothing and right clicking on a code coverage line and choosing break into or debug also appears to do nothing.

2. While I do see the coverage X marker showing me which line fails, clicking on it does nothing and right clicking on it and choosing "show exception details" does nothing. If I look at the NCrunch Tests window I see the following code block and clicking on the link to source/line works correctly. I can only assume that this is a problem with NCrunch correctly interpreting xUnit output?
Code:
Xunit.Sdk.EqualException: Assert.Equal() Failure
Expected: 1
Actual:   2
   at Xunit.Assert.Equal[T](T expected, T actual, IEqualityComparer`1 comparer) in c:\TeamCity\buildAgent\work\74856245f07a90f0\src\xunit.assert\Asserts\EqualityAsserts.cs:line 35
   at Xunit.Assert.Equal[T](T expected, T actual) in c:\TeamCity\buildAgent\work\74856245f07a90f0\src\xunit.assert\Asserts\EqualityAsserts.cs:line 19
   at Test.ClassTests.test_method() in D:\Source\Test\Test\Class.cs:line 13


On the plus side, xUnit theories now show up, which is really what I have been waiting for in nCrunch/xUnit integration. Assuming the above two bugs are fixed (or workarounds provided) then I think I'll be very happy with nCrunch 1.11.

Here is the class I have been testing with for documentation of the above bugs:
Code:
public class ClassTests
{
	[Fact]
	public void test_method()
	{
		Assert.Equal(1, 1);
		Assert.Equal(2, 2);
		Assert.Equal(1, 2);
	}

	[Fact]
	public void when_()
	{
		Assert.Equal(1, 1);
	}

	[Theory]
	[InlineData(true)]
	[InlineData(false)]
	public void theory(Boolean value)
	{
		Assert.True(value);
	}
Micah71381
#2 Posted : Saturday, December 27, 2014 12:48:45 AM(UTC)
Rank: Member

Groups: Registered
Joined: 10/18/2013(UTC)
Posts: 27
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
Show coverage for starting tests only also does not appear to work.
Remco
#3 Posted : Saturday, December 27, 2014 1:06:53 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi,

Thanks for sharing these issues. NCrunch 2.11 is an early build containing a new feature that allows the engine to run outside the IDE, communicating with devenv.exe using IPC. The problems you've described (UI options not working) suggest that the IPC communication isn't working correctly. I'm wondering if you would be able to help me with a bit more information so that I can resolve these problems promptly. Do you mind if I ask a few specific questions about this behaviour?

1. Would you be able to submit a bug report directly after you've had a UI option fail to work correctly?
2. Do you experience this only in a specific solution, or others also? I.e. if you build a sample test solution with a single test, does the problem occur?
3. Do you still experience this problem if you set the 'Engine hosting strategy' global configuration setting to 'HostInIDE'?
4. Which NCrunch UI windows do you usually keep open while you work? Does closing these windows, shutting down the IDE, then re-opening the IDE and using NCrunch with the windows closed make any difference for you?
5. After the UI problems start appearing, try disabling the engine using the disable option in the main menu. Does the engine shut down promptly, or does it seem to hang forever and never disable?

Thanks!

Remco
Micah71381
#4 Posted : Saturday, December 27, 2014 1:25:36 AM(UTC)
Rank: Member

Groups: Registered
Joined: 10/18/2013(UTC)
Posts: 27
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
1. Done.
2. Initially experienced it in a project I am working on. Switched over to a test solution/project (class library I keep handy for reproducing bugs for report) to reproduce the problem. Deleted all but one test in test project/solution and the problem persists.
3. Changing the Engine hosting strategy to "HostiInIDE" does not resolve the issue. My other answers are with this setting set to HostInIDE. I restarted VS after changing this setting and before retesting.
4. I usually have the NCrunch Tests window open. I followed the instructions (closed it, restarted IDE, tested right-click UI) and the problem persisted.
5. Disabling NCrunch does not appear to hang. Code coverage markers go away immediately and I am able to re-enable the engine and they pop back up a moment or two later.
Micah71381
#5 Posted : Saturday, December 27, 2014 1:26:29 AM(UTC)
Rank: Member

Groups: Registered
Joined: 10/18/2013(UTC)
Posts: 27
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
Forgot to mention, I am running:

Microsoft Visual Studio Professional 2013
Version 12.0.31101.00 Update 4
Microsoft .NET Framework
Version 4.5.51650
Remco
#6 Posted : Saturday, December 27, 2014 3:58:55 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Micah71381
#7 Posted : Saturday, December 27, 2014 5:37:48 AM(UTC)
Rank: Member

Groups: Registered
Joined: 10/18/2013(UTC)
Posts: 27
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
The update appears to have fixed "break into a debugger" but it does not appear to have done anything for "show coverage for covering tests" or "show exception details".

Note: I can hover my mouse over the red X and see partial exception details, but left clicking does not do anything despite what the tooltip shows.
Remco
#8 Posted : Saturday, December 27, 2014 6:02:23 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Thanks, it looks like this is a similar problem. I'll see what I can do about getting an update out shortly.
Remco
#9 Posted : Saturday, December 27, 2014 7:02:42 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Micah71381
#10 Posted : Saturday, December 27, 2014 7:36:45 AM(UTC)
Rank: Member

Groups: Registered
Joined: 10/18/2013(UTC)
Posts: 27
Location: United States of America

Was thanked: 2 time(s) in 2 post(s)
All fixed, thanks!
1 user thanked Micah71381 for this useful post.
Remco on 12/27/2014(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.061 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download