Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Inconsistent tests results
Options
Previous Topic Next Topic
emgergensys
#1 Posted : Monday, August 3, 2015 2:34:22 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/11/2013(UTC)
Posts: 3
Location: Canada

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
Hi,

I'am having some issues when I'am using the library FakeDbSet. Using my unit test suite (NUnit), my tests goes well, but Ncrunch keeps showing me that there are errors, so if I change my test by adding space anyhere in the code, my tests goes green. Is this problem known?

You can take a look at the picture below:



[img=(- BROKEN LINK -)]Both test case[/img]
Back to top
Remco
#2 Posted : Monday, August 3, 2015 10:35:54 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,144

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this issue.

My first thought is that the code under test may be providing an intermittent pass/fail result, depending upon which tests have been executed before it inside the process. See here for more details about this problem - http://www.ncrunch.net/documentation/considerations-and-constraints_test-atomicity.

Marking the test with the NCrunch.Framework.IsolatedAttribute is often a good way to troubleshoot this. If the test passes consistently with the attribute applied, this provides fairly solid evidence that the test is somehow sequence or state dependent.
Back to top
1 user thanked Remco for this useful post.
emgergensys on 8/4/2015(UTC)
emgergensys
#3 Posted : Tuesday, August 4, 2015 5:19:08 PM(UTC)
Rank: Newbie

Groups: Registered
Joined: 2/11/2013(UTC)
Posts: 3
Location: Canada

Thanks: 1 times
Was thanked: 1 time(s) in 1 post(s)
Remco;7586 wrote:
Hi, thanks for sharing this issue.

My first thought is that the code under test may be providing an intermittent pass/fail result, depending upon which tests have been executed before it inside the process. See here for more details about this problem - http://www.ncrunch.net/documentation/considerations-and-constraints_test-atomicity.

Marking the test with the NCrunch.Framework.IsolatedAttribute is often a good way to troubleshoot this. If the test passes consistently with the attribute applied, this provides fairly solid evidence that the test is somehow sequence or state dependent.


Hi,

We found the problem and you are right. For some reason, FakeDbSet keeps data shared in its implementation. Using hte IsolatedAttribute fixed the problem but we also found that there is an onverload to force the data cleanup in it's constructor (weird, but works).

Thank you.
Back to top
1 user thanked emgergensys for this useful post.
Remco on 8/4/2015(UTC)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.040 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download