Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
MSTest runsettings file appears to not load
Options
Previous Topic Next Topic
CreepyGnome
#1 Posted : Thursday, August 24, 2017 5:15:23 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/21/2012(UTC)
Posts: 22
Location: Arizona

Thanks: 2 times
Was thanked: 5 time(s) in 5 post(s)
I have an MS Test project for .NET Core 1.1. It has a simple runsettings file like this:

Code:
<RunSettings>
  <TestRunParameters>  
    <Parameter name="foo" value="bar" />  
  </TestRunParameters>  
</RunSettings>


These parameters are supposed to be injected at runtime into all test classes TestContext property


Code:
public TestContext TestContext { get; set; }



Then you can use that property like this:

Code:
[TestMethod]
public void TestMethod1()
{
    Assert.AreEqual("bar", TestContext.Properties["foo"]);
}



This works fine with the Visual Studio 2017 test runner and in the VSTS cloud build and release processes. As they allow you to specify the runsettings file that is passed to the dotnet test command that runs them similar to this:

Code:
dotnet test --settings test.runsettings


I couldn't find any place to provide this value like VS or VSTS has, is this supported and if so how?
Back to top
Remco
#2 Posted : Friday, August 25, 2017 12:30:33 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi, thanks for posting.

NCrunch's MSTest support is done through emulation rather than integration. The emulator currently does not have any support for a test settings file.

If possible, I recommend trying system environment variables or a separate resource file to inject data into your test run. Both of these options should work well for both NCrunch and MSTest.
Back to top
CreepyGnome
#3 Posted : Friday, August 25, 2017 1:27:21 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/21/2012(UTC)
Posts: 22
Location: Arizona

Thanks: 2 times
Was thanked: 5 time(s) in 5 post(s)
Yeah we cannot use environment variables because we run these tests in the VSTS Release Tasks where we are required to use this file and then apply overrides based on the environment the task is running in. Everything works in everything but NCrunch basically and we cannot make a change to our tests and how they are implemented to only appease the NCrunch engine when they work in all other runners.

I would like to request a feature to support this and maybe not use emulation if it makes it easier to do so.
Back to top
Remco
#4 Posted : Friday, August 25, 2017 1:31:39 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
CreepyGnome;11064 wrote:
Yeah we cannot use environment variables because we run these tests in the VSTS Release Tasks where we are required to use this file and then apply overrides based on the environment the task is running in. Everything works in everything but NCrunch basically and we cannot make a change to our tests and how they are implemented to only appease the NCrunch engine when they work in all other runners.

I would like to request a feature to support this and maybe not use emulation if it makes it easier to do so.


Understood. I can't really see a future in which NCrunch's MSTest support is integrated and the emulator is gone, but the emulation can always be improved to fill the gaps. You're most welcome to request this on uservoice if you like.
Back to top
CreepyGnome
#5 Posted : Saturday, August 26, 2017 3:07:07 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/21/2012(UTC)
Posts: 22
Location: Arizona

Thanks: 2 times
Was thanked: 5 time(s) in 5 post(s)
I would love to hope the feature request to the creator of the tool I've used for years now via his forums sufficient. I am not of fan of making my requests to companies multiple times, as I feel the best customer service to customers is to make it easy for them, and when they make requests to make sure it gets to the proper place for them, not to ask them to go make the request one more time some other place. To many times a support person says thats great could you please email that here, call these people to tell them, tweet it to these people, and so forth. So I hope my request for the feature is enough here if it is not, so be it.
Back to top
Remco
#6 Posted : Saturday, August 26, 2017 3:41:56 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
CreepyGnome;11070 wrote:
I would love to hope the feature request to the creator of the tool I've used for years now via his forums sufficient. I am not of fan of making my requests to companies multiple times, as I feel the best customer service to customers is to make it easy for them, and when they make requests to make sure it gets to the proper place for them, not to ask them to go make the request one more time some other place. To many times a support person says thats great could you please email that here, call these people to tell them, tweet it to these people, and so forth. So I hope my request for the feature is enough here if it is not, so be it.


I understand that feature voting has a bit of reputation of being a 'graveyard' for such requests, but it does have a purpose.

NCrunch has many thousands of users, and while these forums work well for troubleshooting, they are not an efficient way to track, organise or prioritise feature requests. By placing a feature request in uservoice, you make it easy for others to add their voice to the request, allowing me to better determine the overall value of the feature relative to other requests. Without knowing how many people would benefit from a feature, it is much harder to prioritise it. This is a process I've learnt the hard way, and it's very well established with our industry.
Back to top
CreepyGnome
#7 Posted : Saturday, August 26, 2017 3:48:09 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/21/2012(UTC)
Posts: 22
Location: Arizona

Thanks: 2 times
Was thanked: 5 time(s) in 5 post(s)
This isn't about feature voting, it's about repeating myself. I would think that a well document and recommend by Microsoft feature of MS Test would be supported by any test runner that wants to support MS Test, as far as I can tell yours is the only one that does not. I wouldn't think you need to vote of features like these, as you do not support MS Test properly at this time. Feature voting is for more obscure things that a product manager needs to find out how the users feel about things. I would think that a test runner needs to support all the features of the testing framework it plans to be running tests for. But that just me I guess.

So how about this, lets call it s BUG since you do not support a required feature of MS Test which prevents MS Test tests from running in your test runner? Do bugs need feature voting to see how many users the bug impacts? If it only impacts a few users you just go tough luck not enough users impacted by this bug, better luck next time?
Back to top
yanglee
#9 Posted : Saturday, August 26, 2017 4:08:08 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/25/2012(UTC)
Posts: 89
Location: China

Thanks: 23 times
Was thanked: 19 time(s) in 18 post(s)
Just come across this thread.

I believe uservoice is actually designed to track both feature requests and bugs.

I know it's a little annoying to submit an idea twice. But personally I think it might be better to submit it by yourself on uservoice if you think it's important, because:

We all want our requests to be implemented/fixed. And top rated ideas have higher chance to be implemented/fixed before others.

So when we submit ideas, we may want to carefully choose words to draw voters's attentions, describe the benefits of the feature to convince people to believe my request is more important than some others, so people may move votes to my ideas.

BTW, I'm also waiting for a fix for another bug. And I honestly wish my bug to be fixed before yours :)
Back to top
yanglee
#10 Posted : Saturday, August 26, 2017 4:18:42 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/25/2012(UTC)
Posts: 89
Location: China

Thanks: 23 times
Was thanked: 19 time(s) in 18 post(s)
Actually I personally prefer to call it an issue rather than a bug.
Back to top
Remco
#8 Posted : Saturday, August 26, 2017 4:35:30 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,976

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
CreepyGnome;11072 wrote:

So how about this, lets call it s BUG since you do not support a required feature of MS Test which prevents MS Test tests from running in your test runner? Do bugs need feature voting to see how many users the bug impacts? If it only impacts a few users you just go tough luck not enough users impacted by this bug, better luck next time?


There is, unfortunately, no project on which the definition of a bug could not be considered subjective.

On a project like NCrunch, where everyone knows the bugs get addressed first, I can understand that there is a strong motive to submit every issue as being a critical show-stopping bug that is a massive hole in the product and needs to be immediately addressed.

You need to be aware that for me to treat every request in this manner would be a tragic disservice to the NCrunch user base as a whole, because it prevents any kind of prioritisation from taking place. Issues, compatibility holes and feature-gaps would be addressed in the order of which user yells the loudest, instead of which will add the most overall value. By trying to argue these points, you also take up my valuable time which is in turn taken away from improving the product for others. I stress this because you have a history of reporting things in this manner, which I have previously found to be destructive and personally difficult to deal with.

If you wish to make NCrunch a better product and if you wish for me to do my best to help you with future support issues, then I would politely ask that you follow the correct process for doing so and avoid directing discussion into arguments. If the level of support provided for this product does not meet your needs, then I would strongly suggest finding an alternative tool or consider different ways of working. Future requests from you that I feel are being unreasonable will not be met with a response.
Back to top
 |  Edit by user
yanglee
#12 Posted : Saturday, August 26, 2017 5:18:45 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 6/25/2012(UTC)
Posts: 89
Location: China

Thanks: 23 times
Was thanked: 19 time(s) in 18 post(s)
@Remco, @CreepyGnome,

Relax and take a deep breath, it's time to go batting center, it works for me :)
Back to top
ajbeaven
#13 Posted : Monday, March 22, 2021 7:20:13 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 6/23/2020(UTC)
Posts: 1
Location: New Zealand

Was thanked: 1 time(s) in 1 post(s)
And after all that, no one created the feature request on UserVoice...

I've made one now, so if you're browsing this thread wanting to see this feature, go here and vote.
Back to top
1 user thanked ajbeaven for this useful post.
Remco on 3/22/2021(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.081 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download