Hi,

I'm experiencing some issues when mocking static methods using typemock and running the tests in ncrunch. When running the tests in a separate process everything works fine but when ncrunch runs the tests automatically they fail randomly. This seems to be caused by mocking the static method's return value (some tests expect fields on the returned object to have different values) and the order in which the tests are executed.

Is there any way that each test can be run in a separate process by default to avoid problems with mocked static methods?


Thanks.

Hi, thanks for posting!

NCrunch won't ever run tests concurrently within the same process, but it will re-use processes between execution runs. Traditional serial test runners don't do this, so not all tests have been engineered to consider it. Broadly, there are two ways to do what you're asking:

1. Adorn the tests involved with the IsolatedAttribute. Each test adorned with this attribute will be executed within a specially created process that is torn down as soon as the test has finished executing.

2. Set the Test process memory limit configuration setting to '1'. This will force NCrunch to terminate each test process after the process has finished executing a batch of tests.

Unfortunately, both of the above options will have a serious impact on NCrunch's performance - especially if you have a large number of small tests. I'm afraid I don't know enough about the behaviour of TypeMock around how it mocks static members to know for certain whether or not the random failures are being caused by a TypeMock/NCrunch clash, or something else. I recommend trying to narrow down the problem to make sure it isn't an issue caused by leftover state being re-used accidentally between tests. It only takes 1 erroneous test to corrupt the state of a test process and cause some very bizarre behaviour downstream in the test pipeline, and often these sorts of problems don't become visible until the project is run under NCrunch. You may find the troubleshooting guide helpful in further analysing this issue.

Cheers,

Remco

Do you notice any difference if you set up NCrunch in compatibility mode? This would help to narrow down if the issue is being caused by NCrunch's environment, or something with TypeMock Isolator.