Hey,
I really like the idea of NCrunch! Therefore I am testing it right now and I wanna try to convince my coleagues next week they should use it to and after that my boss so he will buy it when the trial is over! But while testing I have some issue with some test. I think I do everything OK and that is a kind of bug or maybe I am doing something wrong.
Using:
-xUnit V2 2.0.0 (the final version and not a pre release or something)
-Visual Studio 2013 Premium
-NCrunch for Visual Studio 2013 - 2.13.0.5
-TFS (Visual Studio Online, dunno version)
Code:
Code:
namespace PvS.Common.UnitTests
{
using System.Collections.Generic;
using System.Linq;
using FluentAssertions;
using Xunit;
public class NCrunchDemo
{
public enum Value
{
First,
Second,
Other
}
public enum Parameters
{
Inputs1,
Inputs2,
Inputs1And2
}
public static IEnumerable<object[]> BaseData()
{
return new[]
{
// Data: input1 input2
new object[] { new List<Value>() { Value.First }, Value.First },
new object[] { new List<Value>() { Value.Second }, Value.Second },
};
}
public static IEnumerable<object[]> Data(Parameters parameters)
{
var skipCount = parameters == Parameters.Inputs2 ? 1 : 0;
var parameterCount = parameters == Parameters.Inputs1And2 ? 2 : 1;
return BaseData()
.Select(f => f.Skip(skipCount)
.Take(parameterCount)
.ToArray())
.ToList();
}
// FailAll
[Theory]
[MemberData("Data", Parameters.Inputs1)]
public void DoFailAll_Input1(IEnumerable<Value> input1)
{
input1.Single().Should().Be(Value.Other);
}
[Theory]
[MemberData("Data", Parameters.Inputs2)]
public void DoFailAll_Input2(Value input2)
{
input2.Should().Be(Value.Other);
}
[Theory]
[MemberData("Data", Parameters.Inputs1And2)]
public void DoFailAll_Input1And2(IEnumerable<Value> input1, Value input2)
{
input2.Should().Be(Value.Other);
}
// FailFirst
[Theory]
[MemberData("Data", Parameters.Inputs1)]
public void DoFailFirst_Input1(IEnumerable<Value> input1)
{
input1.Single().Should().Be(Value.Second);
}
[Theory]
[MemberData("Data", Parameters.Inputs2)]
public void DoFailFirst_Input2(Value input2)
{
input2.Should().Be(Value.Second);
}
[Theory]
[MemberData("Data", Parameters.Inputs1And2)]
public void DoFailFirst_Input1And2(IEnumerable<Value> input1, Value input2)
{
input1.Single().Should().Be(Value.Second);
}
// FailSecond
[Theory]
[MemberData("Data", Parameters.Inputs1)]
public void DoFailSecond_Input1(IEnumerable<Value> input1)
{
input1.Single().Should().Be(Value.First);
}
[Theory]
[MemberData("Data", Parameters.Inputs2)]
public void DoFailSecond_Input2(Value input2)
{
input2.Should().Be(Value.First);
}
[Theory]
[MemberData("Data", Parameters.Inputs1And2)]
public void DoFailSecond_Input1And2(IEnumerable<Value> input1, Value input2)
{
input1.Single().Should().Be(Value.First);
}
}
}
So we have Below all results of the unit test.
I have 3 types of test:
-Fail All
-Fail First
-Fail Second
And I call it with 3 signatures:
-1st parameter
-2nd parameter
-Both parameters
With 2 theories each.
So in the most perfect world I see 3*3*2 tests in an overview ow which 6+3+3 fail and 3+3 do not.
Results TFS:
TFS results look perfect. A single row for each theory (=3*3*2) with expected results.
[img=(- BROKEN LINK -)](- BROKEN LINK -)[/img]
Results Visual Studio:
Visual Studio results are not optimal, not all test have a seperate entry. When one or more parameters are of the non basic types the results of the theories are clusterred. But hey, the conclusion looks correct. PvS.Common.UnitTests.NCrunchDemo.DoFailFirst_Input1 fails as a whole. But when clicking on it I see, that only 1 of the theories fails.
[img=(- BROKEN LINK -)](- BROKEN LINK -)[/img]
Results NCrunch:
Results are not OK? Like Visual Studio there is only one entry for PvS.Common.UnitTests.NCrunchDemo.DoFailFirst_Input1. The difference is according to NCrunch is does pass.
[img=(- BROKEN LINK -)](- BROKEN LINK -)[/img]
Because TFS shows everything correct I assume this is a bug in NCrunch? But because I am quite new to NCrunch I cannot rule out I am doeing omething wrong..
Greetz,
Pieter