Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Find out which DistributeByCapabilitiesAttribute capability a test is executing under.
Magnus Lidbom
#1 Posted : Tuesday, June 23, 2020 10:59:46 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/30/2012(UTC)
Posts: 14

Thanks: 2 times
Was thanked: 3 time(s) in 3 post(s)
Hi,

I'm trying to set up testing for the framework I'm developing so that all the tests will automatically run once for each supported storage provider. SQL Server, MySql, PostgreSQL etc.
DistributeByCapabilitiesAttribute seemed perfect for this until I tried it and realized that I can find no way to figure out which capability is currently demanded so that I can instantiate the correct storage provider for the tests.
I neither can, nor think it a good idea, to dedicate a whole machine to each storage provider.

Is there some way of accessing the currently required capabilities? If not, is there some other way to make all the tests in a test assembly execute multiple times with different configurations/environment settings?

Any help would be greatly appreciated. I would be perfectly fine even with digging the data out of some ncrunch type using reflection as long as I'm able to get past this hurdle without having to do major surgery to many hundreds of tests to implement some hackish solution of my own. DistributeByCapabilities seems exactly what I need except for this hurdle.
michaelkroes
#2 Posted : Tuesday, June 23, 2020 6:44:15 PM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 213
Location: Netherlands

Thanks: 96 times
Was thanked: 46 time(s) in 44 post(s)
Hi thanks for posting!

We've got a solution for this that hasn't been released yet. We do have a test release available if you want to give this a spin:

NCrunch_Console_4.4.0.10.msi
NCrunch_Console_4.4.0.10.zip
NCrunch_GridNodeServer_4.4.0.10.msi
NCrunch_GridNodeServer_4.4.0.10.zip
NCrunch_LicenseServer_4.4.0.10.zip
NCrunch_VS2008_4.4.0.10.msi
NCrunch_VS2010_4.4.0.10.msi
NCrunch_VS2010_4.4.0.10.zip
NCrunch_VS2012_4.4.0.10.msi
NCrunch_VS2012_4.4.0.10.zip
NCrunch_VS2013_4.4.0.10.msi
NCrunch_VS2013_4.4.0.10.zip
NCrunch_VS2015_4.4.0.10.msi
NCrunch_VS2015_4.4.0.10.msi.7z
NCrunch_VS2015_4.4.0.10.zip
NCrunch_VS2017_4.4.0.10.msi
NCrunch_VS2017_4.4.0.10.msi.7z
NCrunch_VS2017_4.4.0.10.zip
NCrunch_VS2019_4.4.0.10.msi
NCrunch_VS2019_4.4.0.10.msi.7z
NCrunch_VS2019_4.4.0.10.zip


The new version has introduced a new attribute called DuplicateByDimensions. The attribute can be applied similarly to DistributeByCapabilities, So on Assembly, Class and Methods. NCrunch will duplicate the tests that have attributes specified. It will run those tests in separate processes. The current dimension is accessible via NCrunchEnvironment.GetDuplicatedDimension().

I've included a small example below:

Code:

using NCrunch.Framework;
using NUnit.Framework;

namespace DuplicateByDimensions
{
    [DuplicateByDimensions("Dimension1", "Dimension2")]
    public class TestClassWithDimensions : TestClassDimensionsBase
    {
        [Test]
        public void TestWithNoDimension()
        {
            NCrunchEnvironment.GetDuplicatedDimension();
        }

        [Test]
        [DuplicateByDimensions("Dimension3", "Dimension4")]
        public void TestWithOwnDimensions()
        {
            NCrunchEnvironment.GetDuplicatedDimension();
        }

    }
}


I hope this works for you! Let me know if there's anything I can do.
1 user thanked michaelkroes for this useful post.
Magnus Lidbom on 6/23/2020(UTC)
Magnus Lidbom
#3 Posted : Tuesday, June 23, 2020 6:52:01 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/30/2012(UTC)
Posts: 14

Thanks: 2 times
Was thanked: 3 time(s) in 3 post(s)
I most certainly do want to give it a spin. I've started trying to achieve the same thing using Shared Projects (strange project type that shares source rather than create an assembly that is shared) and it is a sad mess trying to do it that way. I'm so glad I won't need to go that route. Will report back as soon I've had time to install the test release on my computers and get rid of all the horrifying code :)
1 user thanked Magnus Lidbom for this useful post.
michaelkroes on 6/24/2020(UTC)
Magnus Lidbom
#4 Posted : Tuesday, June 23, 2020 8:25:26 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/30/2012(UTC)
Posts: 14

Thanks: 2 times
Was thanked: 3 time(s) in 3 post(s)
Got the test version installed on my whole grid and DuplicateByDimensions seems to do exactly what I need.
Thank you so very much for the quick and helpful response!

All the best :)
1 user thanked Magnus Lidbom for this useful post.
michaelkroes on 6/24/2020(UTC)
michaelkroes
#5 Posted : Wednesday, June 24, 2020 3:45:02 AM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 213
Location: Netherlands

Thanks: 96 times
Was thanked: 46 time(s) in 44 post(s)
Thanks for the quick feedback!
Magnus Lidbom
#6 Posted : Wednesday, June 24, 2020 7:15:25 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/30/2012(UTC)
Posts: 14

Thanks: 2 times
Was thanked: 3 time(s) in 3 post(s)
You're welcome.

I do have a suggestion though, that would really help me in time. Frequently libraries/frameworks have multiple pluggable parts. In my case they might be Serializer, NetworkTransport, and StorageProvider. To really be sure that things are working all possible combinations should be tested. With the current attribute in testing I would have to do something like:
Code:

[DistributeByCapability("Newtonsoft-NetMQ-SqlServer", "Newtonsoft-HTTP-SqlServer"......etc etc for all combinations ]


It would be awesome if the attribute could take care of that for me. Like this test code:

Code:

[assembly:DistributeByCapabilities(capabilities: new[]{"SqlServer", "MySql", "PostgreSql"},
                                      orthogonalCapabilities1:new[]{"Newtonsoft", "Json.Net"},
                                      orthogonalCapabilities2:new[]{"NetMQ", "HTTP"})]


//Skeleton attribute:

[AttributeUsage(AttributeTargets.Assembly)] public class DistributeByCapabilities : Attribute
{
    public DistributeByCapabilities(string[] capabilities, string[] orthogonalCapabilities1 = null, string[] orthogonalCapabilities2 = null /*Keeps going for a few more capabilities*/) {}
}


Maybe multiple applications of the DistributeByCapability attribute or some other method could be used to specify the different dimensions so there is no arbitrary limit to the number of dimensions.

What do you think?
Magnus Lidbom
#7 Posted : Wednesday, June 24, 2020 7:41:04 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/30/2012(UTC)
Posts: 14

Thanks: 2 times
Was thanked: 3 time(s) in 3 post(s)
(Edit: This is a broken design, see next comment.)
Maybe something like this would be better:

Code:

[assembly:CombinatorialDistributeByCapabilities("SqlServer", "MySql", "PostgreSql")]
[assembly:CombinatorialDistributeByCapabilities("Newtonsoft", "Microsoft")]
[assembly:CombinatorialDistributeByCapabilities("NetMQ", "HTTP")]

[AttributeUsage(AttributeTargets.Assembly,AllowMultiple = true)] public class CombinatorialDistributeByCapabilitiesAttribute : Attribute
{
    public CombinatorialDistributeByCapabilitiesAttribute(params string[] capabilities) {}
}


This would enable scenarios such as having one attribute on the assembly and another on specific tests or test fixtures so that some combinations are not global, but the global capabilities are applied together with the attributes on the local class. Possibly along with a flag such as "IgnoreGlobalCapabilities" for tests or fixtures that want to specify things on their own.

Hmm, what happens currently if you apply DistributeByCapabilities on multiple levels?
Magnus Lidbom
#8 Posted : Wednesday, June 24, 2020 8:06:50 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/30/2012(UTC)
Posts: 14

Thanks: 2 times
Was thanked: 3 time(s) in 3 post(s)
Hmm, no. The last version is no good. There is no way to know which capability is intended by which attribute.

Maybe something like:

Code:

[assembly:CombinatorialDistributeByCapabilities(capabilityName: "StorageProvider", capabilities: new []{"SqlServer", "MySql", "PostgreSql"})]
[assembly:CombinatorialDistributeByCapabilities(capabilityName: "Serializer", capabilities: new []{ "Newtonsoft", "Microsoft"})]
[assembly:CombinatorialDistributeByCapabilities(capabilityName: "NetworkTransport", capabilities: new []{  "NetMQ", "HTTP"})]

[AttributeUsage(AttributeTargets.Assembly,AllowMultiple = true)] public class CombinatorialDistributeByCapabilitiesAttribute : Attribute
{
    public CombinatorialDistributeByCapabilitiesAttribute(string capabilityName, params string[] capabilities) {}
}
Magnus Lidbom
#9 Posted : Wednesday, June 24, 2020 8:28:44 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/30/2012(UTC)
Posts: 14

Thanks: 2 times
Was thanked: 3 time(s) in 3 post(s)
Heh, I was just about to start writing out another possible option. I'll control myself and not do that though. I figure you get the idea :)
If you want me to, I'll be happy to comment on any other approach to this you might want feedback on.
michaelkroes
#10 Posted : Wednesday, June 24, 2020 6:56:38 PM(UTC)
Rank: NCrunch Developer

Groups: Registered
Joined: 9/22/2017(UTC)
Posts: 213
Location: Netherlands

Thanks: 96 times
Was thanked: 46 time(s) in 44 post(s)
Thanks for the detailed posts! We really appreciate it.

It would be possible to implement this, but it would introduce some complexity. We would like to gauge the user interest for this first. If you want you can add this to https://ncrunch.uservoic...45203-feature-requests.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.055 seconds.