Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

With optimized instrumentation generic MSpec behavior fails with "This test was not executed..."
ljohnston
#1 Posted : Thursday, March 26, 2020 6:27:08 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/17/2014(UTC)
Posts: 31

Thanks: 17 times
Was thanked: 5 time(s) in 5 post(s)
With Instrumentation Mode set to Optimized I'm seeing MSpec behaviors with generic types fail when instrumented with a custom type (though not with .NET types such as Object or String). After changing the instrumentation mode to Legacy everything works as expected.

Requirements:
Latest Machine.Specifications and Machine.Specification.Should NuGet packages.
NCrunch instrumentation mode set to Optimized.

Code:

// Everything works as expected for this test (one test passes and one test fails)
public class when_concrete_type_is_string : TestBehavior<String> {

    Establish context = () => {
        Console.Out.WriteLine("Context ran.");
    };
     
    Behaves_like<TestBehavior<String>> a_test_object = () => { };
}

// Both tests fail with this message: 
// This test was not executed during a planned execution run.
// Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures.
public class when_concrete_type_is_test_object : TestBehavior<TestObject> {

    Establish context = () => {
        Console.Out.WriteLine("Context ran.");
    };

    Behaves_like<TestBehavior<TestObject>> a_test_object = () => { };
}

[Behaviors]
public class TestBehavior<T> {

    It should_run_passing_test = () => {
        true.ShouldBeTrue();
    };

    It should_run_failing_test = () => {
        false.ShouldBeTrue();
    };

}

public class TestObject { 
}



Let me know if you need me to gather any additional information for you.
Remco
#2 Posted : Thursday, March 26, 2020 11:00:30 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi,

Thanks for sharing this problem. I've managed to reproduce it exactly as described.

We'll take a look and will update you when a fix is available.
1 user thanked Remco for this useful post.
ljohnston on 3/30/2020(UTC)
michaelkroes
#3 Posted : Tuesday, April 7, 2020 9:52:14 AM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 280
Location: Netherlands

Thanks: 124 times
Was thanked: 63 time(s) in 60 post(s)
1 user thanked michaelkroes for this useful post.
ljohnston on 4/8/2020(UTC)
ljohnston
#4 Posted : Wednesday, April 8, 2020 4:34:36 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/17/2014(UTC)
Posts: 31

Thanks: 17 times
Was thanked: 5 time(s) in 5 post(s)
Hi,

Thanks for working on this.

This fixed behaviors with one generic argument, but behaviors with two generic arguments are still broken (I haven't tried more than two). This can be reproduced using the code in the original post by changing TestBehavior<T> to TestBehavior<T, U> and passing the same type a second time everywhere it's used (e.g. TestBehavior<TestObject, TestObject>). The error message is the same.
Remco
#6 Posted : Wednesday, April 8, 2020 11:57:19 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
ljohnston;14591 wrote:

This fixed behaviors with one generic argument, but behaviors with two generic arguments are still broken (I haven't tried more than two). This can be reproduced using the code in the original post by changing TestBehavior<T> to TestBehavior<T, U> and passing the same type a second time everywhere it's used (e.g. TestBehavior<TestObject, TestObject>). The error message is the same.


Thanks for picking this up. We'll see if we can get it fixed.
Remco
#5 Posted : Sunday, April 12, 2020 10:49:28 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
ljohnston;14591 wrote:

This fixed behaviors with one generic argument, but behaviors with two generic arguments are still broken (I haven't tried more than two). This can be reproduced using the code in the original post by changing TestBehavior<T> to TestBehavior<T, U> and passing the same type a second time everywhere it's used (e.g. TestBehavior<TestObject, TestObject>). The error message is the same.


I'm having some trouble reproducing this as you've described. It seems to work correctly for me. Is there any chance you can share a code sample with me?
ljohnston
#7 Posted : Monday, April 20, 2020 5:51:25 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/17/2014(UTC)
Posts: 31

Thanks: 17 times
Was thanked: 5 time(s) in 5 post(s)
Sorry about the delay, I didn't get a notification for your latest reply (I think I must have viewed the previous one without logging in/using the link from the email).

Here's an updated sample (which appears to fail in the same way as the previous one did):
Code:

// Everything works as expected for this test (one test passes and one test fails)
public class when_concrete_type_is_string : TestBehavior<String, String> {

    Establish context = () => {
        Console.Out.WriteLine("Context ran.");
    };
 
    Behaves_like<TestBehavior<String, String>> a_test_object = () => { };
}

// Both tests fail with this message: 
// This test was not executed during a planned execution run.
// Ensure your test project is stable and does not contain issues in initialisation/teardown fixtures.
public class when_concrete_type_is_test_object : TestBehavior<TestObject, TestObject> {

    Establish context = () => {
        Console.Out.WriteLine("Context ran.");
    };

    Behaves_like<TestBehavior<TestObject, TestObject>> a_test_object = () => { };
}

[Behaviors]
public class TestBehavior<T, U> {

    It should_run_passing_test = () => {
        true.ShouldBeTrue();
    };

    It should_run_failing_test = () => {
        false.ShouldBeTrue();
    };

}

public class TestObject { 
}
Remco
#8 Posted : Monday, April 20, 2020 11:56:10 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Thanks for the sample. That did it for me (red across the board).

I'll let you know as soon as we have a fix available.
Remco
#9 Posted : Wednesday, April 22, 2020 2:34:52 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
1 user thanked Remco for this useful post.
ljohnston on 4/22/2020(UTC)
ljohnston
#10 Posted : Wednesday, April 22, 2020 10:49:23 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 1/17/2014(UTC)
Posts: 31

Thanks: 17 times
Was thanked: 5 time(s) in 5 post(s)
All our behaviors are behaving as expected now. Thanks!
1 user thanked ljohnston for this useful post.
Remco on 4/23/2020(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.078 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download