Hi Remco,

I have two tests:

Private m_Tests As New List(of Test)

<Test(Order:=1)> _
Public Sub Prepare(<Factory("TestFactory")> TestObj As Test)
' Do a test
' then add Test data to list.
m_Tests.Add(TestObj)
Gallio.Framework.TestLog.WriteLine("Prepare")
End Sub

<Test(Order:=2)> _
<Rollback()> _
Public Sub Update()
' Do Update on each Test in m_Tests
Gallio.Framework.TestLog.WriteLine("Update")
End Sub

When I run the tests in Debug mode Prepare is run and then Update and the tests pass. This is expected.

At end of running tests NCrunch Tests window shows:
Green clock icon next to Prepare test. (I would have expected green check mark here. So Not sure why I'm seeing green clock.)
Green check mark next to Update test.

I don't see the Gallio WriteLine in the log window


When I run the tests not in Debug mode it appears Prepare is not run but Update runs. Update expects Prepare to add the tests to m_Tests.
But Update discovers m_Tests.Count = 0.


At end of running NCrunch Tests window shows:
Green clock icon next to Prepare.
Red X next to Update test.

I put ExclusivelyUses on the test fixture. I get the same behavior as noted above.

I then added NCrunch serial attribute to each of the tests. I get the same behavior as noted above.

I was going to explain why I divided my tests into these two stages but it was getting complicated. So I'm hoping I've described the problem sufficiently for some feedback.

Thanks, Ed

Hi Remco,

The original test code was a single function (not two as I showed it in my original post). The test code was testing a function which worked like this:

1) Gets values from Green database.
2) Gets values from Yellow database based on the data retrieved from the Green database.
3) Updates the Green database with the values retrieved from Yellow.

I applied the Rollback attribute to the test function to preserve the state of the database.

<Rollback()> _
Public Sub OriginalTest()
End Sub

The problem was that the test code was getting an exception when the code attempted to read records from Yellow database.

Exception: "The operation could not be performed because OLE DB provider "SQLNCLI10" for linked server "Yellow" was unable to begin a distributed transaction."

I'm not the DBA so I can't fix the problem. My workaround was to divide the test code into Prepare and Update functions.

The Prepare test only reads from Yellow and Green databases and then stores the values in a test class member variable.

The Update then picks up the values in the class member variable and uses them to update the Yellow database.

This means Prepare must run first and then Update.

If I put the two tests back together as you suggested I'd have to apply the Rollback attribute to single test and then I'd get the exception again.


I started using Gallio\MbUnit several years ago on this project. I realized several months ago the developers stopped supporting it but
I was reluctant to switch to another test framework. I have hundreds of tests in this project and didn't want to spend a lot of time
making the changes.

What do you recommend for a test framework?

Will NCrunch work with a project that has tests designed for two different test frameworks at the same time?

I'll give it a try when I finish this post.

Also, thanks for the note to use Debug.WriteLine.

Thanks,

Ed

Hi Ed,

This is a very interesting test case, and it's an excellent example of where test ordering can be particularly useful. In similar situations, I've generally split the code between setup/teardown methods and the test methods, or implemented exception traps in stages within the test.

Xunit V2 does have the ability to run tests in a fixed sequence using TestCaseOrdererAttribute. NCrunch will also detect this attribute and will group the tests together so they aren't split between execution tasks (it will also do this with ALL MSpec test contexts/fixtures). Though as you've mentioned you're using VS2010, this probably won't be an option for you.

NUnit at this time doesn't have support for defined test execution order. To my knowledge, MSTest is the same. This means that at the moment you have 3 options:

- Redesign the tests for less granular reporting
- Upgrade to VS2013 and use Xunit V2
- Wait/hope for me to fix MbUnit test orders in NCrunch (please don't wait for this, I have no idea yet if this is actually possible)

Regarding recommended test framework, it's hard for me to really do this as all frameworks have their unique strengths. Personally, I use NUnit, but this is more because I'm a bit old-school. Many people have been moving to Xunit lately. Xunit has seen very extensive development and right now is moving towards supporting multiple platforms (very useful). NUnit V3 is also approaching soon and I expect this will be a large improvement over the original, as Charlie has been very open with the design approach for V3. MSpec is also a very clean product though its syntax does take some getting used to. NCrunch works very well with basic use of MSTest (performance is very good), though MSTest may give you trouble with other toolsets.