Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

2 Pages12>
Can't go to selected Test from NCrunch Tests Tool Window in VS 2015 CTP5
Der-Albert.com
#1 Posted : Thursday, January 22, 2015 9:32:59 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/17/2011(UTC)
Posts: 209

Thanks: 11 times
Was thanked: 55 time(s) in 51 post(s)
Hi,

can't go to selected Test from NCrunch Tests Tool Window in VS 2015 CTP5.

Works in VS 2013 Update 4

Best Regards
Remco
#2 Posted : Thursday, January 22, 2015 9:35:34 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this issue.

Does this problem seem to apply to just specific tests, or all of them?

Do you get it if you try it on a new blank solution?

Is there any chance you could submit a bug report after you've had it happen?


Thanks!

Remco
Der-Albert.com
#3 Posted : Friday, January 23, 2015 9:56:46 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/17/2011(UTC)
Posts: 209

Thanks: 11 times
Was thanked: 55 time(s) in 51 post(s)
All of the tests, all projects, all computers, even a simple One Test Only Project. I've submitted a bug report from that.
Remco
#4 Posted : Friday, January 23, 2015 11:37:11 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks for the bug report.

I'm sorry but I'm having trouble reproducing this problem on my end. I'll need to try and narrow it down with some more questions...

- Do you experience it on any other versions of Visual Studio? (if you have them installed)
- Do you see the little '>' arrows on the starting line of tests in your codebase? Specifically the tests that you are having trouble navigating to
- Which test framework are you using? Do you notice the same problem with all test frameworks?
Der-Albert.com
#5 Posted : Friday, January 23, 2015 11:52:23 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/17/2011(UTC)
Posts: 209

Thanks: 11 times
Was thanked: 55 time(s) in 51 post(s)
In Visual Studio 2013 Update 4 it works, on both computers.

No Arrows, only a little green or red circle.

I use MSpecs, current Version from Nuget.

Sample Project
https://onedrive.live.co...91B36AC797CD65%21136006
SampleSpecs.7z

Remco
#6 Posted : Friday, January 23, 2015 9:18:13 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Thanks, I've now managed to reproduce this problem.

It seems to be caused by subtle differences in the IL generation between Roslyn and the unmanaged C++ compiler. It looks like Roslyn generates IL slightly differently inside constructors. NCrunch manually walks the IL in MSpec context constructors in order to match up generated compiler methods with their fields ... the differences in Roslyn are causing this code to fail, and NCrunch therefore can't identify the entry points for the tests.

So this issue is specific to MSpec and VS2015.

I'll see if I can get this fixed.
Der-Albert.com
#7 Posted : Saturday, January 24, 2015 7:20:32 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/17/2011(UTC)
Posts: 209

Thanks: 11 times
Was thanked: 55 time(s) in 51 post(s)
You can do that ;)
Remco
#8 Posted : Monday, January 26, 2015 2:11:26 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Der-Albert.com
#9 Posted : Monday, January 26, 2015 9:38:26 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 5/17/2011(UTC)
Posts: 209

Thanks: 11 times
Was thanked: 55 time(s) in 51 post(s)
Quick Test, works for me .... Thank You!
1 user thanked Der-Albert.com for this useful post.
Remco on 1/26/2015(UTC)
GlobalConcepts
#10 Posted : Tuesday, April 12, 2016 8:40:26 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/7/2013(UTC)
Posts: 52
Location: Germany

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
Since Version 2.20 this problem occurs again.
We have some setups that are different from each other.
All of them are using combinations of Visual Studio 2015 Enterprise with either Update 1 or Update 2 and either Windows 8.1 or Windows 10.

With Version 2.19 that problem did not happen.
The tests are getting executed on an ncrunch server but the problem is also happen when I'm not connected to it.

What does not work:
- double-click on a green or red code coverage circle
- righ-click an select "go to selected" test
- double-click on any test through "NCrunch Tests" Window
Remco
#11 Posted : Tuesday, April 12, 2016 9:19:29 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi,

Can share share some details about which tests are causing this problem for you? Are you using MSpec? If you can give me some sample code of a test that you can't navigate to, I may be able to reproduce and fix this promptly.
GlobalConcepts
#12 Posted : Tuesday, April 12, 2016 9:42:36 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/7/2013(UTC)
Posts: 52
Location: Germany

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
We're using xunit tests. The test as it selve gets recognized and executed without problems.
For example we have some NullGuard tests but we have extended the AutoFixture and NSubstitute to make testing smarter and faster implemented for us. So you cannot execute that code on your environment.

I've exhumed some other project that has just some simpled xUnit tests marked with [Fact] attribute. Here "Go to selected test" works fine.
Means: [Fact] => is working, [Theory, NSubstituteCrmAutoData] => not working.
Did you have something change determining tests?
Remco
#13 Posted : Tuesday, April 12, 2016 9:50:42 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
I can't think of anything changed in 2.20 that would affect this, but I really need a code sample to analyse it first.

Are you sure you can't put anything together that demonstrates the problem in a way you can share with me?
GlobalConcepts
#14 Posted : Tuesday, April 12, 2016 9:52:46 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/7/2013(UTC)
Posts: 52
Location: Germany

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
Remco;8565 wrote:
I can't think of anything changed in 2.20 that would affect this, but I really need a code sample to analyse it first.

Are you sure you can't put anything together that demonstrates the problem in a way you can share with me?


[Theory, NSubstituteCrmAutoData]
public void Run_ExecutesUpdateEntityOnce(
[Frozen] IInterface interface,
InterfaceImplementation sut)
{
// Arrange


// Act
sut.Run();

// Assert
Assert.Equal(1, interface.ReceivedCalls().Count());
}

Maybe this is helping you. But you don't have NSubstituteCrmAutoData. That is part of our framework.
Remco
#15 Posted : Tuesday, April 12, 2016 9:54:52 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
How are the NSubstituteCrmAutoData and Frozen attributes implemented? Is this a custom framework of some kind?
GlobalConcepts
#16 Posted : Tuesday, April 12, 2016 10:01:23 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/7/2013(UTC)
Posts: 52
Location: Germany

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
Frozen is part of AutoFixture Xunit that provides you implementations of provided types or interface to not have implement it froh var interface = Substitute.For<IInterface>(); for example.
NSubstituteCrmAutoDataAttribute is a modified kind AutoDataAttribute that is also part of AutoFixture.AutoNSubstitute or AutoFixture.Xunit This comes through our custom framework. For further details I have to ask the responsible developer first if I may provide this to you.
Remco
#17 Posted : Tuesday, April 12, 2016 10:09:24 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
NCrunch is completely reliant on the Xunit framework to supply it with the full name of the physical method that represents the test.

As you're likely aware, Xunit has ways in which you can override or extend its behaviour using custom attributes that feed into user defined code. Depending on what this code does, it can have a significant impact on how the tests are represented to a runner like NCrunch. Because it's impossible to know how this user code might surface test information, it's impossible to know whether NCrunch will be able to work with it correctly, or provide accurate error information when tests are not structured as expected.

The functionality you're having problems with (navigation to the first line of a test) is dependent on NCrunch being able to recognise the method that represents this test at the time it's discovered using Xunit. This method needs to be exposed to NCrunch from Xunit's ITestCase.TestMethod.Method property. If the property isn't supplied or the data attached to it is incoherent (for example, the underlying instance of IMethodInfo does not match a true method in the assembly), NCrunch will then consider the test as having its entry point undeclared, and you won't be able to navigate to it in the UI.

If you have access to the framework code involved here, I suggest comparing its behaviour to that of a standard Xunit test discovery run. The method data provided via ITestCaseDiscoveryMessage must be the same as it would be for a normal Xunit discovery run.
GlobalConcepts
#18 Posted : Tuesday, April 12, 2016 11:17:36 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/7/2013(UTC)
Posts: 52
Location: Germany

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
Here we go:

using Microsoft.Xrm.Sdk;
using NSubstitute;
using Ploeh.AutoFixture;
using Ploeh.AutoFixture.AutoNSubstitute;
using Ploeh.AutoFixture.Idioms;
using Ploeh.AutoFixture.Xunit;
using Xunit;
using Xunit.Extensions;

public class NSubstituteCrmAutoData : AutoDataAttribute
{
/// <summary>
/// Initializes a new instance of the <see cref="Test"/> class.
/// </summary>
public NSubstituteCrmAutoData()
: base(
new Fixture().Customize(
new TestConventions(new CrmTestConventions(), new AutoNSubstituteCustomization())))
{ }
}

public class TestConventions : CompositeCustomization
{
/// <summary>
/// Initializes a new instance of the <see cref="TestConventions"/> class.
/// </summary>
/// <param name="crmTestConventions"><see cref="CrmTestConventions"/> or a
/// derived implementation setting up conventions for the fixture.</param>
/// <param name="autoMockCustomization">One of the AutoFixture automocking customizations.</param>
public TestConventions(
CrmTestConventions crmTestConventions,
ICustomization autoMockCustomization)
: base(
new OmitAutoPropertiesCustomization(),
crmTestConventions,
autoMockCustomization)
{ }

private class OmitAutoPropertiesCustomization : ICustomization
{
public void Customize(IFixture fixture)
{
fixture.OmitAutoProperties = true;
}
}
}

public class CrmTestConventions : ICustomization
{
/// <summary>
/// Customizes the specified fixture to return working <see cref="Entity"/> and <see cref="EntityReference"/>objects.
/// </summary>
/// <param name="fixture">The fixture to customize.</param>
public virtual void Customize(IFixture fixture)
{
fixture.Customize<Entity>(f => f.WithAutoProperties().Without(e => e.ExtensionData));
fixture.Customize<EntityReference>(f => f.WithAutoProperties().Without(e => e.ExtensionData));

this.CustomizeSpecific(fixture);
}

/// <summary>
/// Override to register more specific default implementations for testing individual projects.
/// </summary>
/// <param name="fixture">The fixture to customize.</param>
protected virtual void CustomizeSpecific(IFixture fixture)
{
}
}
GlobalConcepts
#19 Posted : Tuesday, April 12, 2016 11:20:00 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 2/7/2013(UTC)
Posts: 52
Location: Germany

Thanks: 5 times
Was thanked: 6 time(s) in 6 post(s)
BTW: ITestCaseDiscoveryMessage isn't used anywhere
Remco
#20 Posted : Tuesday, April 12, 2016 12:30:25 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
The following is a test project I'm trying to build to reproduce the issue you've described.

Quote:
using Microsoft.Xrm.Sdk;
using Ploeh.AutoFixture;
using Ploeh.AutoFixture.AutoNSubstitute;
using Ploeh.AutoFixture.Xunit;
using Xunit;

namespace ClassLibrary3
{
public class NSubstituteCrmAutoData : AutoDataAttribute
{
/// <summary>
/// Initializes a new instance of the <see cref="Test" /> class.
/// </summary>
public NSubstituteCrmAutoData()
: base(
new Fixture().Customize(
new TestConventions(new CrmTestConventions(), new AutoNSubstituteCustomization())))
{
}
}

public class TestConventions : CompositeCustomization
{
/// <summary>
/// Initializes a new instance of the <see cref="TestConventions" /> class.
/// </summary>
/// <param name="crmTestConventions">
/// <see cref="CrmTestConventions" /> or a
/// derived implementation setting up conventions for the fixture.
/// </param>
/// <param name="autoMockCustomization">One of the AutoFixture automocking customizations.</param>
public TestConventions(
CrmTestConventions crmTestConventions,
ICustomization autoMockCustomization)
: base(
new OmitAutoPropertiesCustomization(),
crmTestConventions,
autoMockCustomization)
{
}

private class OmitAutoPropertiesCustomization : ICustomization
{
public void Customize(IFixture fixture)
{
fixture.OmitAutoProperties = true;
}
}
}

public class CrmTestConventions : ICustomization
{
/// <summary>
/// Customizes the specified fixture to return working <see cref="Entity" /> and <see cref="EntityReference" />objects.
/// </summary>
/// <param name="fixture">The fixture to customize.</param>
public virtual void Customize(IFixture fixture)
{
fixture.Customize<Entity>(f => f.WithAutoProperties().Without(e => e.ExtensionData));
fixture.Customize<EntityReference>(f => f.WithAutoProperties().Without(e => e.ExtensionData));

CustomizeSpecific(fixture);
}

/// <summary>
/// Override to register more specific default implementations for testing individual projects.
/// </summary>
/// <param name="fixture">The fixture to customize.</param>
protected virtual void CustomizeSpecific(IFixture fixture)
{
}
}

public interface IInterface
{
}

public class InterfaceImplemented : IInterface
{
}

public class FixtureX
{
[Theory, NSubstituteCrmAutoData]
public void Run_ExecutesUpdateEntityOnce([Frozen] IInterface intr, InterfaceImplemented sut)
{
// Arrange

// Act

// Assert
}
}
}


The test currently gives the following error -
"System.InvalidOperationException: No data found for ClassLibrary3.FixtureX.Run_ExecutesUpdateEntityOnce"

I admit that my knowledge of the substitution frameworks here is very limited, so I'm not entirely sure how the theory is expected to function in this instance. However, so far NCrunch does seem to be able to navigate to the test.

Could you please extend this to provide a working sample that can reproduce the problem?


Users browsing this topic
Guest
2 Pages12>
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.122 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download