Reactive Extensions and ReactiveUI tests failures

Hi,

I'm using Reactive Extensions and ReactiveUI (http://rx.codeplex.com/ & http://www.reactiveui.net/).

When I run my tests using the VS2010 test runner they all pass.
When NCrunch runs them they sometimes fail and sometimes pass (at random it seems).
This will always fail in NCrunch and always pass in VS2010.

Steps to reproduce:

    using System;
    using System.Text;
    using System.Collections.Generic;
    using System.Linq;
    using Microsoft.VisualStudio.TestTools.UnitTesting;
    using ReactiveUI;
    using System.Reactive.Concurrency;
    using Microsoft.Reactive.Testing;
    using System.Reactive.Linq;

    public class NCrunchBug : ReactiveObject
    {        
        IScheduler scheduler;
        List<NCrunchMessage> messages = new List<NCrunchMessage>();
        public List<NCrunchMessage> Messages
        {
            get { return messages; }
        }

        public NCrunchBug(IScheduler sched = null)
        {
            if (sched == null)
                scheduler = Scheduler.Default;
            else
                scheduler = sched;
            RegisterBusListener();
        }

        private void RegisterBusListener()
        {
            RxApp.MessageBus.Listen<NCrunchMessage>()
                .ObserveOn(scheduler)
                .Subscribe(message => messages.Add(message));
        }
    }

    public class NCrunchMessage
    {
        public string Message { get { return "NCRUNCH!!!"; } }
    }

    [TestClass]
    public class NCrunchTester
    {
        [TestMethod]
        public void MessageTest()
        {
            NCrunchBug bug = new NCrunchBug(Scheduler.CurrentThread);
            RxApp.MessageBus.SendMessage<NCrunchMessage>(new NCrunchMessage());
            Assert.AreEqual(1, bug.Messages.Count);
        }
    }

Also the send Report from the extension doesn't work :)

Cheers,
nisbus

Hi Nisbus,

Thanks for sharing this issue.

Am I correct in my understanding that this test is executing code across multiple threads?

I'm afraid that I have no knowledge of the how this framework behaves, although I suspect that this is a problem with the way the test is designed, rather than NCrunch itself. Because NCrunch executes the test much more frequently (and in a very different environment) to other test runners, there is a much high probability that race conditions within the test will be noticeable with NCrunch but not other test runners.

Intermittent failure of tests that execute code across multiple threads is highly likely to be the result of race conditions within either the test, or the code under test.

If this message bus listens for messages on a background thread, then there is a suspect race condition between the dispatch of the message and where it is received, as this test is checking for the existence of a received message on a different thread to that response for receiving the message.

If you aren't executing this code across multiple threads, check that the test is not somehow sequence dependent (i.e. can be confused by leftover state from previous execution runs of itself or other tests).

Relevant documentation links that you might want to look through: Multi-threaded tests, Test atomicity.


Cheers,

Remco

Hi Remco,

Just in case others run into the same problem.
I figured this out and the solution is to put in the ClassInitialize this line:

RxApp.DeferredScheduler = Scheduler.CurrentThread;

This makes all the tests run successfully.

cheers,
nisbus