I have some problems, when using Arbitraries from the FsCheck package. I am not sure if this is a problem in NCrunch or FsCheck itself, but it might be a concurrency issue, as it does not happen in R# runner when the property tests are run sequentially.
The project file:
<Project Sdk="Microsoft.NET.Sdk">
<PropertyGroup>
<TargetFramework>netcoreapp1.1</TargetFramework>
<IsPackable>false</IsPackable>
</PropertyGroup>
<ItemGroup>
<PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0-preview-20170425-07" />
<PackageReference Include="xunit" Version="2.2.0" />
<PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
<PackageReference Include="FsCheck.Xunit" Version="*" />
<PackageReference Include="UnitsNet" Version="*" />
</ItemGroup>
</Project>
The cs file:
using FsCheck;
using FsCheck.Xunit;
using UnitsNet;
namespace FsCheckNcrunchExample
{
public class UnitTest1
{
class Arbitraries
{
public static Arbitrary<Length> Lengths()
=> Arb.From<double>()
.Generator
.Where(i => i > 0 && i < 100)
.Select(Length.FromMeters)
.ToArbitrary();
}
static UnitTest1()
{
Arb.Register<Arbitraries>();
}
[Property]
public void Test1(Length a, Length b)
{
}
[Property]
public void Test2(Length a, Length b)
{
}
}
}
Other things to note:
- When using double types as the parameters to the tests, it works. It is only if I try to use the registered arbitraries.
- The failures are quite consistent whenever NCrunch is running the tests automatically.
- I noticed that this is failing with structs and I have not tested complex classes yet. The UnitsNet library just provides a useful set of predifined structs, so I used that in this example.
EDIT: I forgot to attach the stacktrace:
Exception during test
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
at FsCheck.Runner.checkMethod(Config config, MethodInfo m, FSharpOption`1 target)
at Xunit.Sdk.ExecutionTimer.Aggregate(Action action)
at <StartupCode$FsCheck-Xunit>.$PropertyAttribute.testExec@187(PropertyTestCase this, IMessageBus messageBus, Object[] constructorArguments, CancellationTokenSource cancellationTokenSource, XunitTest test, RunSummary summary, TestOutputHelper outputHelper, Unit unitVar0)
System.Exception: The type UnitsNet.Length is not handled automatically by FsCheck. Consider using another type or writing and registering a generator for it.
at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1379.Invoke(String message)
at FsCheck.ReflectArbitrary.reflectGenObj@129.Invoke(Type t)
at FsCheck.Common.f@1[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n, Unit _arg1)
at FsCheck.Common.memoizeWith[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n)
at FsCheck.Arb.Derive@1138.get_Generator()
at FsCheck.Arbitrary`1.FsCheck-IArbitrary-get_GeneratorObj()
at FsCheck.ReflectArbitrary.gs@64.GenerateNext(IEnumerable`1& next)
at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
at Microsoft.FSharp.Collections.SeqModule.ToArray[T](IEnumerable`1 source)
at FsCheck.ReflectArbitrary.productGen@63[a](FSharpFunc`2 getGenerator, IEnumerable`1 ts, FSharpFunc`2 create)
at FsCheck.ReflectArbitrary.reflectObj(FSharpFunc`2 getGenerator, Type t)
at FsCheck.ReflectArbitrary.reflectGenObj@129.Invoke(Type t)
at FsCheck.Common.f@1[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n, Unit _arg1)
at FsCheck.Common.memoizeWith[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n)
at FsCheck.Arb.Derive@1138.get_Generator()
at FsCheck.Testable.forAll[a,b](Arbitrary`1 arb, FSharpFunc`2 body)
at FsCheck.Runner.check[a](Config config, a p)