Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
2 Pages<12
Approvals and Distributed Processing
Options
Previous Topic Next Topic
MaxRonin
#21 Posted : Sunday, March 20, 2016 11:54:44 AM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2016(UTC)
Posts: 32
Location: Austria

Thanks: 11 times
Was thanked: 3 time(s) in 3 post(s)
Besides, let me take the chance to thank both of you, Remco and Llewellyn, not just for your help with this issue, but for providing both these tools.

I've been using NCrunch for years, have also recommended it to some of my client teams, and can say C# development without it just doesn't feel productive anymore.

As for Approvals, I've seen it used in the past a few times, but in one of my current projects I first started using it myself because the use-case scenario is an ideal fit: I'm writing a transpiler for C# to Typescript, Java and C++ (it's specialized, so it's only gotta support the runtime features used by the C# source of one my client) and I use approvals in my integration and regression tests to check the output code. In this scenario, Approvals is... ideal. Without it, I could not imagine making those tests properly readable, let alone easily maintainable.

So, once more, a big thanks and a huge thumbsup to both of you guys :)

Max
Back to top
1 user thanked MaxRonin for this useful post.
Remco on 3/20/2016(UTC)
JayBazuzi
#22 Posted : Sunday, March 20, 2016 10:15:33 PM(UTC)
Rank: Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 10
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
What if ApprovalTests switched away from using the call stack, to using [CallerMemberName] and [CallerFilePath]?
Back to top
MaxRonin
#23 Posted : Sunday, March 20, 2016 10:21:05 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2016(UTC)
Posts: 32
Location: Austria

Thanks: 11 times
Was thanked: 3 time(s) in 3 post(s)
JayBazuzi;8442 wrote:
What if ApprovalTests switched away from using the call stack, to using [CallerMemberName] and [CallerFilePath]?

I think these do not use reflection, but are replaced by the compiler at compile time. Not sure, though.
Back to top
MaxRonin
#24 Posted : Monday, March 21, 2016 12:43:23 PM(UTC)
Rank: Advanced Member

Groups: Registered
Joined: 3/18/2016(UTC)
Posts: 32
Location: Austria

Thanks: 11 times
Was thanked: 3 time(s) in 3 post(s)
Hi again,

it seems from the rewriting of the source path stems a second problem:

Since I've added the "include additional items..." options:

If perform these steps:

1. Write new test
2. Run it with a reporter popping up a merge tool in case of failure.
3. The merge tool pops up. The paths of both files are NOT pointing to my actual source location, but to NCrunch's temporary working directory.

Now here's the first issue - if, inside the merge tool, I now copy from left (received) to right (approved) and save, the approved file will be saved in the wrong location. I can work around this:

4. Save a copy of approved file to the right location.

But that's obviously very error prone and it goes even further:

5. Close merge tool.
6. Open approved file in the right location and edit it with text editor, make some minor change.
7. Run the test again
8. NCrunch reports a success although I would expect a failure now as the change in the approved file has no corresponding change in code.

The reason for 8 seems to be that NCrunch takes the approved copy from the temporary location rather than from the source location.

Together, these two are now really hindering my workflow.

I should add that at some point in the past I googled how to make NCrunch use the regular Merge reporter instead of its default silent reporter. Unfortunately, I cannot remember anymore how I did that - it was a few months ago - and the only relevant hit I've found again mentions an assembly attribute about frontloading, but I checked my properties/AssemblyInfo.cs and there's no such attribute. So really I got no idea how I would go back to NCrunch's default behavior.
Maybe this is somehow related to the part of the described behavior in 8?

Also, albeit slightly offtopic, @Remco:
it's sort of unpleasant to more or less statically decide whether NCrunch uses the quiet or the merge reporter. I tend to turn off Resharper's unit-test support because if it's turned on, the visual markers of R# create clutter I really don't need. Besides, all in all, I don't want to use two tools for running my tests.
But when I'm using only NCrunch with Approvals and I keep NCrunch's default quiet reporter, then I can't easily get to the comparison of the files in case of a failure.

Ideally, NCrunch would use the quiet reporter, but when debugging a test, it would use the regular merge reporter.

Thanks,
Max
Back to top
Remco
#25 Posted : Monday, March 21, 2016 9:23:03 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,000

Thanks: 932 times
Was thanked: 1259 time(s) in 1172 post(s)
Hi Max,

We will need some feedback from Llewellyn on how to tackle these issues. As NCrunch is only a base runner calling into user code, it isn't able to change any of the behaviour you've just described (this is effectively handled further up the tool stack in Approvals itself). Being able to rig up the tooling to only use the merge reporter when a debugger attached does make sense to me - I have some tests myself that work in a similar way. I'm happy to help support Llewellyn here in any way I can.

One way to work around the distributed processing issues here might be to mark the tests with a NCrunch.Framework.RequiredCapabilityAttribute so that they are set to only run on the local machine. I appreciate that this takes away the benefit of distributed processing for these tests, but if they make up only a minority of your test suite, it could be a quick fix that would allow you to continue to work while we address this.
Back to top
1 user thanked Remco for this useful post.
MaxRonin on 3/22/2016(UTC)
JayBazuzi
#26 Posted : Tuesday, March 22, 2016 1:16:02 AM(UTC)
Rank: Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 10
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
What if NCrunch recreated the original solution at the original path on the node? Since it would match the local machine, things would just work, right?
Back to top
Remco
#27 Posted : Tuesday, March 22, 2016 3:11:42 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,000

Thanks: 932 times
Was thanked: 1259 time(s) in 1172 post(s)
JayBazuzi;8464 wrote:
What if NCrunch recreated the original solution at the original path on the node? Since it would match the local machine, things would just work, right?


It already does :)

NCrunch transfers the entire known solution to the grid node before it begins any processing. However, the paths in the PDB still need to point to the original solution path on the client, because this is where the results are interpreted and read by the user.

It's possible to access the path of the solution on the grid node or the local machine through the NCrunch.OriginalSolutionPath environment variable.
Back to top
JayBazuzi
#28 Posted : Tuesday, March 22, 2016 4:43:15 AM(UTC)
Rank: Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 10
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
Remco;8465 wrote:
JayBazuzi;8464 wrote:
What if NCrunch recreated the original solution at the original path on the node? Since it would match the local machine, things would just work, right?


It already does :)



Really?

What I mean is: if I have my sources at C:\Projects\Foo on my local machine, that NCrunch puts that at C:\Projects\Foo on the node.
Back to top
Remco
#29 Posted : Tuesday, March 22, 2016 4:52:46 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,000

Thanks: 932 times
Was thanked: 1259 time(s) in 1172 post(s)
JayBazuzi;8466 wrote:

What I mean is: if I have my sources at C:\Projects\Foo on my local machine, that NCrunch puts that at C:\Projects\Foo on the node.


Oh, sorry .. I misunderstood. No, it doesn't do this. Such an approach wouldn't work outside a very narrow use case, as the nodes need the potential to store many different snapshots/solutions on behalf of multiple connected clients. For performance reasons, NCrunch will store solutions on the node over a long term period to prevent needing to re-copy files again and again. It naturally needs somewhere to place these solutions where they can be adequately managed.

There is also no guarantee that file paths can be made the same on the node as they are on the client. For example, if the solution is stored at D:\Projects\MyProject on the client, but there is no D: drive on the node.

The extension of this argument is to make it configurable where to place the solution on the grid node. This is already possible to a certain extent (using the 'Snapshot storage directory' setting on the node), but this system still needs to place solutions in separate compartments, so the path would never be the same as it is on the client. You could argue that it should be made possible to define (via configuration) the storage directory for each solution on a connected client, but this is terribly messy and requires considerable effort ... and would likely be confusing to many people. Compared with a relatively small change to Approvals to have it look for the .txt file using the NCrunch environment variable, such an approach doesn't make much sense.
Back to top
JayBazuzi
#30 Posted : Tuesday, March 22, 2016 2:58:10 PM(UTC)
Rank: Member

Groups: Registered
Joined: 12/19/2013(UTC)
Posts: 10
Location: United States of America

Was thanked: 1 time(s) in 1 post(s)
Obviously it would not be trivial. All of the concerns you raise are valid and would need to be addressed.

One option would be virtualization/containers. Then each run thinks it's running in the "correct" file layout, but they are still isolated.
Back to top
Users browsing this topic
Guest
2 Pages<12
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.065 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download