Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Problems using FsCheck arbitraries
aignas
#1 Posted : Monday, September 25, 2017 11:22:45 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 7/24/2017(UTC)
Posts: 5
Location: Lithuania

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
Hello,

I have some problems, when using Arbitraries from the FsCheck package. I am not sure if this is a problem in NCrunch or FsCheck itself, but it might be a concurrency issue, as it does not happen in R# runner when the property tests are run sequentially.

The project file:
Code:
<Project Sdk="Microsoft.NET.Sdk">

  <PropertyGroup>
    <TargetFramework>netcoreapp1.1</TargetFramework>

    <IsPackable>false</IsPackable>
  </PropertyGroup>

  <ItemGroup>
    <PackageReference Include="Microsoft.NET.Test.Sdk" Version="15.3.0-preview-20170425-07" />
    <PackageReference Include="xunit" Version="2.2.0" />
    <PackageReference Include="xunit.runner.visualstudio" Version="2.2.0" />
    <PackageReference Include="FsCheck.Xunit" Version="*" />
    <PackageReference Include="UnitsNet" Version="*" />
  </ItemGroup>

</Project>


The cs file:
Code:
using FsCheck;
using FsCheck.Xunit;
using UnitsNet;

namespace FsCheckNcrunchExample
{
    public class UnitTest1
    {
        class Arbitraries
        {
            public static Arbitrary<Length> Lengths()
                => Arb.From<double>()
                    .Generator
                    .Where(i => i > 0 && i < 100)
                    .Select(Length.FromMeters)
                    .ToArbitrary();
        }

        static UnitTest1()
        {
            Arb.Register<Arbitraries>();
        }

        [Property]
        public void Test1(Length a, Length b)
        {
        }

        [Property]
        public void Test2(Length a, Length b)
        {
        }
    }
}


Other things to note:

  • When using double types as the parameters to the tests, it works. It is only if I try to use the registered arbitraries.
  • The failures are quite consistent whenever NCrunch is running the tests automatically.
  • I noticed that this is failing with structs and I have not tested complex classes yet. The UnitsNet library just provides a useful set of predifined structs, so I used that in this example.


EDIT: I forgot to attach the stacktrace:
Code:
Exception during test
System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor)
   at System.Reflection.RuntimeMethodInfo.UnsafeInvokeInternal(Object obj, Object[] parameters, Object[] arguments)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at FsCheck.Runner.checkMethod(Config config, MethodInfo m, FSharpOption`1 target)
   at Xunit.Sdk.ExecutionTimer.Aggregate(Action action)
   at <StartupCode$FsCheck-Xunit>.$PropertyAttribute.testExec@187(PropertyTestCase this, IMessageBus messageBus, Object[] constructorArguments, CancellationTokenSource cancellationTokenSource, XunitTest test, RunSummary summary, TestOutputHelper outputHelper, Unit unitVar0)
System.Exception: The type UnitsNet.Length is not handled automatically by FsCheck. Consider using another type or writing and registering a generator for it.
   at Microsoft.FSharp.Core.PrintfModule.PrintFormatToStringThenFail@1379.Invoke(String message)
   at FsCheck.ReflectArbitrary.reflectGenObj@129.Invoke(Type t)
   at FsCheck.Common.f@1[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n, Unit _arg1)
   at FsCheck.Common.memoizeWith[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n)
   at FsCheck.Arb.Derive@1138.get_Generator()
   at FsCheck.Arbitrary`1.FsCheck-IArbitrary-get_GeneratorObj()
   at FsCheck.ReflectArbitrary.gs@64.GenerateNext(IEnumerable`1& next)
   at Microsoft.FSharp.Core.CompilerServices.GeneratedSequenceBase`1.MoveNextImpl()
   at System.Collections.Generic.List`1..ctor(IEnumerable`1 collection)
   at Microsoft.FSharp.Collections.SeqModule.ToArray[T](IEnumerable`1 source)
   at FsCheck.ReflectArbitrary.productGen@63[a](FSharpFunc`2 getGenerator, IEnumerable`1 ts, FSharpFunc`2 create)
   at FsCheck.ReflectArbitrary.reflectObj(FSharpFunc`2 getGenerator, Type t)
   at FsCheck.ReflectArbitrary.reflectGenObj@129.Invoke(Type t)
   at FsCheck.Common.f@1[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n, Unit _arg1)
   at FsCheck.Common.memoizeWith[a,b](IDictionary`2 memo, FSharpFunc`2 f, a n)
   at FsCheck.Arb.Derive@1138.get_Generator()
   at FsCheck.Testable.forAll[a,b](Arbitrary`1 arb, FSharpFunc`2 body)
   at FsCheck.Runner.check[a](Config config, a p)
aignas
#3 Posted : Monday, September 25, 2017 2:05:09 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 7/24/2017(UTC)
Posts: 5
Location: Lithuania

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
I have raised an issue on the FsCheck github page as well: https://github.com/fscheck/FsCheck/issues/398
Remco
#4 Posted : Tuesday, September 26, 2017 3:18:33 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,122

Thanks: 957 times
Was thanked: 1286 time(s) in 1193 post(s)
Hi,

Thanks for sharing this problem.

This doesn't appear to be an issue caused by NCrunch. I observe the exact same behaviour when I execute this code using the xunit visualstudio runner.

There is considerable complexity here inside FsCheck, and the types involved are completely unknown to me. Sorry, I can be of no help in troubleshooting this issue.
aignas
#5 Posted : Tuesday, September 26, 2017 4:48:57 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 7/24/2017(UTC)
Posts: 5
Location: Lithuania

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
Hello Remco,

The test code that was posted above has an multithreading issue because of the fact that I was using a static constructor to register the arbitraries. Once the constructor is made non-static, everything works as intended. I guess FsCheck has a steep learning curve for a non F# programmer and the lack of examples on how to exactly use the Arbitraries in C# made it easier to make such a mistake. So this is not an issue caused by NCrunch.

Thanks for your time.
1 user thanked aignas for this useful post.
Remco on 9/26/2017(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.032 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download