Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Run pre/post-build events doesn't resolve macros correctly
CreepyGnome
#1 Posted : Sunday, November 17, 2013 10:23:17 PM(UTC)
Rank: Member

Groups: Registered
Joined: 8/21/2012(UTC)
Posts: 22
Location: Arizona

Thanks: 2 times
Was thanked: 5 time(s) in 5 post(s)
Greetings,

When you enable pre/post build events with NCrunch and you are using xcopy with build in macro's it should resolve it as if it was running within the solution and project path, not from within wherever it decides to copy files to spacfe.

So if you have a macro like so:

xcopy /Y "$(SolutionDir)Library\Sqlite3\sqlite3.dll" $(TargetDir)

Which is required to copy the NATIVE library into a C# build output so that P/Invokes and use it. I cannot reference this library and I need this file to show up prior to any code being expected.

When using NUnit test runner, or R# test runner they all handle these pre/post builds correctly. However NCrunch is the only test runner that resolves them incorrectly.

You get an error because it can't find the file.

xcopy /Y "C:\Users\<user>\AppData\Local\NCrunch\6236\22\Library\Sqlite3\sqlite3.dll" C:\Users\<user>\AppData\Local\NCrunch\6236\22\Tests\TestSqliteWpf\bin\Debug\" exited with code 4.

This is because it is looking in "C:\Users\<user>\AppData\Local\NCrunch\6236\22" as the SolutionDir which is not correct. So either NCrunch needs to copy everything in the realy SolutionDir to its special location or it needs to translate these so they go to the correct place. I think translating works best, Or have a setting to just run the tests from their really build location like every other test runner. I understand you copy them out because you are running them in the background and probably other reasons. However because of this you have a lot more work that should be done to allow normal things like build events to occur and to occur safely.

This has been killing me lately on a few projects and makes me have to disable NCrunch altogether as it is to much work to deal with and I resort to R# instead. However I rather keep R# test runners disabled and have background testing going.

Remember one of the goals is to have tests portable so that any developer can run them easily. Keeping everything withingthe solution folder is a good way to stay portable. If you keep your pre/post build events using only build in macros that never leave your solution directory then all other developers you are working with can also easily run the test.
Remco
#2 Posted : Monday, November 18, 2013 12:28:53 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,979

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Hi,

Thanks for posting and sharing this issue.

I believe this issue should be easy to resolve by adding the sqlite3.dll file to the Additional files to include setting. This will inform NCrunch of the dependency on the file so that it can copy it to the workspace, allowing your build to function normally (see Implicit file dependencies for more information on this).

Because NCrunch performs background build and execution tasks, the shadowing is unfortunately necessary for about 90% of the features in the product. There is no way to build and run tests reliably against a project as it exists in its original location on disk, as there is simply too much interference from Visual Studio and other processes to allow this to happen. To rely on foreground solution test running is basically the same as going back to manual test runs with a code coverage tool, which would sadly defeat the purpose of the product.

I definitely recommend having a read of the 'Considerations and Constraints' section of the documentation. I realise that many of these constraints will likely feel artificial when working with an existing solution that hasn't been designed with them in mind, but being aware of them may help save you from much frustration.

Cheers,

Remco
1 user thanked Remco for this useful post.
CreepyGnome on 11/18/2013(UTC)
CreepyGnome
#3 Posted : Monday, November 18, 2013 1:00:38 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/21/2012(UTC)
Posts: 22
Location: Arizona

Thanks: 2 times
Was thanked: 5 time(s) in 5 post(s)
Hi Remco,

I am fully aware of they additional files ot include feature of NCrunch however that removes the portability of the tests and it is duplicating "code" that shouldn't be duplicated. Not all developers use NCrunch that I share code with, so I don't check in any NCrunch settings, I expect the NCrunch to work command standard out of the box settings.

I have read the considerations and Constraints documentation in the past and I don't think it has changed much since. I also stated I understand why you need to run from a copied to location. I think a goal should be trying to over come as many considerations and constraints as you can over time to make it work as much like any other test runner out there but with all your additional features they don't. Otherwise other test runners will always have something they can do that NCrunch can't.

This isn't asking how I can workaround what is a bug from the customers perspective it is me reporting it as a bug and asking for it to be fixed. This seems like it could be supported. It seems like you could support many of the targets by mapping them to the correct locations. Target stuff should work as is, but things like Project and Solution related ones may not especially ones that are folder based.

If mapping doesn't work why not copy all non-project folders and files in a solution to your mirrored locations? This could be a boolean setting to enable and disable, this may be much easier than, and these types of files are typically static so you would only have to copy them if they change.
Remco
#4 Posted : Monday, November 18, 2013 2:20:49 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,979

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
If the Additional Files To Include configuration setting doesn't meet your needs, another option that may be worth trying in your situation is to include a reference from the project XML file to the DLL file being referenced. You can do this simply with a 'None' XML element, for example:
<ItemGroup>
<None Include="$(SolutionDir)Library\Sqllite3\Sqllite3.dll" />
</ItemGroup>

NCrunch will then detect the file as being used by the project and will copy it, allowing your build to function as normal in the NCrunch workspace.

I realise that what you are asking for is effectively for me to provide you with a feature set that will allow NCrunch to work without any configuration required, but I'm afraid that with the current design of the tool this is simply impossible. The features provided by NCrunch bring with them an inherent compatibility penalty that can affect solutions that haven't been designed with the tool in mind. It's almost always possible to get the tool to work with a solution, but the effort involved in this is always variable and depends upon the situation.

The simplification of NCrunch configuration and resolution of compatibility issues is an ongoing exercise that so far as seen many years of effort behind it. There will also be many more years further spent continuing to evolve the tool to make it as easy to 'switch on' as possible. Believe me, there is plenty of incentive to continue to make these improvements as supporting the environments that NCrunch operates in is an expensive and time consuming exercise. The current model has had considerable thought put behind it along with several years of trial and error. It may not always seem like the easiest approach in every niche scenario, but it does work.

The current model of shadowing individual projects is as fundamental to the tool as the language it's written in. The work that NCrunch does needs to be carefully sandboxed with all its dependencies controlled. The sorts of issues that can appear from having references from the NCrunch workspace to the foreground solution are much more numerous and much harder to resolve than the dependency issues you're currently facing.

I agree that there are several ways of handling the identification of solution or project dependencies. The approach you've suggested also has its own caveats - for example, where would the files be copied from? The root solution directory? This would then leave out any files referenced relatively above the directory of the .sln file, which is quite a common situation. NCrunch also requires knowledge of which projects use which files, as it does in fact work at project level (not solution level), so there would need to be consistent way to identify these dependencies. The shadowing mechanism is not as simple as it might immediately seem, and there are many reasons for it existing in its current form.

I'm sorry that I can't offer you a solution that will take your frustration away and make the tool work in the simplest possible way in your environment. The areas of NCrunch that you've called into question are too fundamental to ever change, even if a better implementation can actually be found.
1 user thanked Remco for this useful post.
CreepyGnome on 11/18/2013(UTC)
CreepyGnome
#5 Posted : Monday, November 18, 2013 5:44:52 AM(UTC)
Rank: Member

Groups: Registered
Joined: 8/21/2012(UTC)
Posts: 22
Location: Arizona

Thanks: 2 times
Was thanked: 5 time(s) in 5 post(s)
I understand that if I add a file to the solution as a linked solution item it will be copied, however you shouldn't add DLL's as a solution item really. But that would be the best of all the options so far as it is at least portable.

I am not asking you to add a feature that would allow NCrunch to work without any configuration required. I am asking for NCrunch to allow the tests to be portable. To allow them to run just like they do in every other test runner but NCrunch. The scenario I bring up really isn't a niche scenario either.

So you are intentionally complicated or at best over thinking something I stated that is actually rather simple. Any files OUTSIDE of the solution directory should not ever be considered. Solutions should be self-contained, this becomes practically a requirement when working with Microsoft TFS and their Workspace concepts. However even when working with other source control providers keeping a solution self contained within its solution folder is a good practice to have.

Thanks for your time, I will drop the issue as I see there is no point to trying to convince you of the merritts of it.
Remco
#6 Posted : Monday, November 18, 2013 5:54:55 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,979

Thanks: 931 times
Was thanked: 1257 time(s) in 1170 post(s)
Thanks for your understanding. I'm not trying to shoot down your idea - I just have trouble fitting it with the current design of the product in a way that I think will achieve its desired outcome. I really do appreciate the effort you've taken to share the concept with me and I do understand that you do so with the intention of making NCrunch a better and more usable product .. which is something that I really respect and am grateful for. Though I guess as is often the case in the industry we work in, nothing is ever as simple as it seems :)
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.072 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download