Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Conditional expression evaluated wrong by NCrunch
Options
Previous Topic Next Topic
Taschentiger
#1 Posted : Tuesday, August 14, 2012 8:23:37 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 8/14/2012(UTC)
Posts: 7
Location: Germany

Thanks: 1 times
I have the following test:

Code:
        [TestMethod]
        public void OSIs64Bit_ReturnsCorrectResult()
        {
            Assert.AreEqual(IntPtr.Size == 8, OracleHelpers.OSIs64Bit);
        }


I am running on a 64-bit machine, so IntPtr.Size is 8 (verified), so this expression is true. OSIs64Bit returns correctly true. However this test fails in NCrunch because the comparison expression is incorrectly evaluated to false:

Expected: False
But was: True

With NUnit as TestRunner, this test passes correctly.

This bevavior is true for version 1.40b and 1.41b for both VS2010 and VS2012RC.
Back to top
Taschentiger
#2 Posted : Tuesday, August 14, 2012 8:30:01 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 8/14/2012(UTC)
Posts: 7
Location: Germany

Thanks: 1 times
Interesting supplement:

Just adding a line of code causes this test to pass:

Code:
        [TestMethod]
        public void OSIs64Bit_ReturnsCorrectResult()
        {
            int x = IntPtr.Size;   // This line added for debugging
            Assert.AreEqual(IntPtr.Size == 8, OracleHelpers.OSIs64Bit);
        }

Back to top
Taschentiger
#3 Posted : Tuesday, August 14, 2012 8:38:04 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 8/14/2012(UTC)
Posts: 7
Location: Germany

Thanks: 1 times
To be exact on this supplement:

1. Add the additional first line --> Test still fails.
2. Set a breakpoint on the Assert statement and run in debugger.
3. Verify that IntPtr.Size is indeed 8.
4. Continue with the test --> Test passes, but the Assert line is marked with a green cross??? Passed or failed?
5. Now you can remove the additional first line --> The test still passes and is now shown correct.
Back to top
Remco
#4 Posted : Tuesday, August 14, 2012 8:59:09 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,986

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
When working with MSTest, NCrunch relies on a project-level configuration setting to determine whether the test environment should be set up for x64. This configuration setting is used when the test project itself has its output architecture set to 'AnyCPU', as the architecture to be used by the test runner is thus ambiguous.

Try setting the NCrunch configuration setting, 'Use CPU Architecture' to 'x64' for your test project. Hopefully this should solve the problem.
Back to top
Taschentiger
#5 Posted : Tuesday, August 14, 2012 9:18:55 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 8/14/2012(UTC)
Posts: 7
Location: Germany

Thanks: 1 times
This will unfortunately not work out, as the tests must pass on several machines (Continuous Integrations servers), some of them might well be 32-Bit. This is not under my control.

I managed to get the test to pass (see post #3). I do not have the slightest clue, why this worked and changed the behaviour, but it worked excatly as described.

Also, if NCrunch assumed to setup a 32-Bit environment, wouldn't then also run the code under test in this environment? In this case the method OSIs64Bit would return false in the same way as the comparison expression in the Assert statement. Both arguments would be false and the test passes.

The OSIs64Bit() method is nothing else than:

Code:
public static bool OSIs64Bit()
{
    return IntPtr.Size == 8;
}


So, for me it's working (at least) now. So no problem at the moment.

This post is more or less meant as a hint for you that there is something strange going on...
Back to top
Remco
#6 Posted : Tuesday, August 14, 2012 11:27:34 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,986

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
I have a bit of a theory about what may be happening. It's just a theory, and at this stage I don't have much hard evidence to back it up ... but a theory is better than nothing!

Is the 'OSIs64Bit' method defined in a different assembly to the test project containing the failing test? If so, what processor architecture are both of these two projects configured to use? Are they consistent with each other?

I'm wondering if there is some kind of bitwise consistency issue surfacing that may be related to the sequence in which relevant assemblies are being loaded into the test process's application domain. NCrunch can create different behaviour here compared with other test runners, as it can execute tests in a very different order. I would think that to have IntPtr.Size == 8 return different values from inside the same process should be impossible unless the CLR was in some kind of corrupted state.

You mentioned earlier that when you debug the test it passes. Have you ever managed to have the test fail while the debugger is attached? I think it would be very interesting to see a debugger dump of the modules that are loaded into the process. Note that you can also debug using an existing process (by going through the 'Advanced' context submenu in NCrunch) - this may help with producing a failure with the debugger attached.
Back to top
Taschentiger
#7 Posted : Tuesday, August 14, 2012 11:42:10 AM(UTC)
Rank: Newbie

Groups: Registered
Joined: 8/14/2012(UTC)
Posts: 7
Location: Germany

Thanks: 1 times
Are the test and the code under test in different assemblies: yes

Both assemblies are configured for "Any CPU" and are targeted to .Net 4.0. The test assembly is configured to use NUnit instead of MSTest. See the following code excerpt:


Code:
#if NUNIT
using NUnit.Framework;
using TestClass      = NUnit.Framework.TestFixtureAttribute;
using TestMethod     = NUnit.Framework.TestAttribute;
using TestInitialize = NUnit.Framework.SetUpAttribute;
using TestCleanup    = NUnit.Framework.TearDownAttribute;
#else
using Microsoft.VisualStudio.TestTools.UnitTesting;
#endif

The constant NUNIT is defined in the test project settings.


Unfortunately, I cannot reproduce the wrong behavior anymore. Once the test has made it through the debugger it passes! I could verify the steps in my post #3 with the counterpart of OSIs64Bit() called OSIs32Bit() which tests for a pointer size of 4 byte and behaved exactly the same. But also here: once through the debugger and it passes...
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.048 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download