The problem I am seeing is that after upgrading to version 2.5.0.12 there is a change in behaviour in regards to running xunit facts. It used to be possible to just add the attribute to any method and this would be picked up by ncrunch and run fine.
The behaviour I am seeing after the upgrade is that facts that are in either a module or a type that is not decorated with any attributes will be added to the pipeline (the little icon at the bottom of vs shows the count and i can see them in the queue window), but are never actually run. When I then add the Trait attribute with a key "Category" and any value these tests will be picked up fine and get run by the engine.
From my understanding it is a specific choice by the developers of xunit to be able to declare tests without the ceremony of having to have a test class or container to house them in. I think this is a step backwards in the scope of ncrunch and would hope for clarification on this matter.
I would like to add that it took me a while to figure this out by myself, so hopefully you could add something to the known issues or some such where google might turn up a result for others.
Build/Test Issues
FSharp Xunit tests in module without traits stuck in pipeline
Started by mschinzpb on 7,241 views
Remco NCrunch Developer
#5588
25 Mar 2014 21:36 UTC
Hi, thanks for sharing this issue.
NCrunch hasn't introduced any changes around xunit integration for a very long time... So I suspect that there is more to this issue that would immediately seem to be the case.
Can you check that you aren't working inside a custom engine mode that has deliberately excluded tests adorned with a particular category?
Does this issue appear for you if you create a blank project and try to create a new xunit test with the structure that you've described?
NCrunch hasn't introduced any changes around xunit integration for a very long time... So I suspect that there is more to this issue that would immediately seem to be the case.
Can you check that you aren't working inside a custom engine mode that has deliberately excluded tests adorned with a particular category?
Does this issue appear for you if you create a blank project and try to create a new xunit test with the structure that you've described?
Hey Remco,
Thanks for getting back about this. I'm a bit swamped atm, but I should have a demo solution with you after the weekend.
To answer your question: Run all tests automatically default engine.
Thanks for getting back about this. I'm a bit swamped atm, but I should have a demo solution with you after the weekend.
To answer your question: Run all tests automatically default engine.
Hey Remco, managed to squeeze 15 min today, so here is the repo with the sample:
https://github.com/mschinz/xunitncrunchtest.git
Let me know if you can reproduce there.
Cheers,
M
https://github.com/mschinz/xunitncrunchtest.git
Let me know if you can reproduce there.
Cheers,
M
Remco NCrunch Developer
#5596
26 Mar 2014 21:42 UTC
Hi,
Thanks for sending through the sample solution.
Unfortunately, the code inside the solution seemed to work for me. All tests were detected and run. I tried it on several machines, but received the same result.
I think that something has changed in the configuration of your machine that may have triggered this problem. You mentioned earlier that this issue appeared for you after you upgraded NCrunch. Does downgrading back to your original version make any difference?
Have you installed any significant updates recently around .NET, VS, or FSharp?
Under the directory 'C:\Users\USER\AppData\Roaming\NCrunch' you'll find a file called v2_globalconfig.crunch.xml. If you delete this file with VS closed, it should restore your global NCrunch settings to their default values. This could also be something worth trying as it would rule out whether there a change in your global configuration may have surfaced this problem.
Thanks for sending through the sample solution.
Unfortunately, the code inside the solution seemed to work for me. All tests were detected and run. I tried it on several machines, but received the same result.
I think that something has changed in the configuration of your machine that may have triggered this problem. You mentioned earlier that this issue appeared for you after you upgraded NCrunch. Does downgrading back to your original version make any difference?
Have you installed any significant updates recently around .NET, VS, or FSharp?
Under the directory 'C:\Users\USER\AppData\Roaming\NCrunch' you'll find a file called v2_globalconfig.crunch.xml. If you delete this file with VS closed, it should restore your global NCrunch settings to their default values. This could also be something worth trying as it would rule out whether there a change in your global configuration may have surfaced this problem.
Hey Remco,
Thank You for your supporting this issue.
I'm quite a bit embarrassed, as I have had a filter for category unit in my settings.
Consider this as resolved, and please accept my apologies,
Martin
Thank You for your supporting this issue.
I'm quite a bit embarrassed, as I have had a filter for category unit in my settings.
Consider this as resolved, and please accept my apologies,
Martin
Post a reply
Log in to reply.