I've had a go at trying to reproduce this problem using the same structure and configuration as you've described above. Unfortunately, the tests seem to work correctly every time for me.
Are you able to reproduce this with a sample solution that you can share with me? You can submit code in ZIP form through the
contact form.
My guts says that this may be a sequencing issue in Xunit itself, perhaps surfaced by something in the environment or solution. We don't do any messing with test collection in NCrunch - we simply give Xunit a list of tests to run. If you haven't already, I suggest upgrading to the latest version of Xunit to see if this solves the problem (I think we're up to 2.4 now).