Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error
Solution without metrics
Options
Previous Topic Next Topic
fsdschmidt
#1 Posted : Wednesday, June 29, 2016 11:14:34 AM(UTC)
Rank: Member

Groups: Registered
Joined: 6/29/2016(UTC)
Posts: 24
Location: Germany

Was thanked: 2 time(s) in 2 post(s)
We do have an existing solution with multiple projects and tests. Usually the binaries were signed, but we activated the prevent signing flag of NCrunch for all assemblies. The assemblies successfully built and the tests run but NCrunch do not reports any metrics. Also the code markers does not show (not black just missing). We use NCrunch also with other solutions and it works very well but we do not understand where the problem of missing metrics. Can anybody help us?
Back to top
Remco
#2 Posted : Wednesday, June 29, 2016 11:42:45 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Hi, thanks for sharing this issue.

If the markers are missing, this would suggest there is no instrumentation taking place.

Check that the 'Instrument output assembly' setting has been turned on for all the projects in your solution.
Back to top
fsdschmidt
#3 Posted : Wednesday, June 29, 2016 12:58:42 PM(UTC)
Rank: Member

Groups: Registered
Joined: 6/29/2016(UTC)
Posts: 24
Location: Germany

Was thanked: 2 time(s) in 2 post(s)
Thanks for your fast reply. Yes the instrument flag is enabled, I tried also disabling and enabling the flag for all assemblies.
Back to top
Remco
#4 Posted : Wednesday, June 29, 2016 10:49:30 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
fsdschmidt;8898 wrote:
Thanks for your fast reply. Yes the instrument flag is enabled, I tried also disabling and enabling the flag for all assemblies.


Thanks. Something must be going wrong here then ...

Can you tell me anything more about these projects? What kind of project are they? (i.e. standard .NET framework class library, Silverlight, MVC, .NET Core/DNX, etc).

Can you also confirm that the problem started happening after changing the assembly signing? Does changing it back 'fix' the problem?
Back to top
 |  Edit by user
fsdschmidt
#5 Posted : Friday, July 1, 2016 7:22:06 AM(UTC)
Rank: Member

Groups: Registered
Joined: 6/29/2016(UTC)
Posts: 24
Location: Germany

Was thanked: 2 time(s) in 2 post(s)
I´ve disabled the signing now but there is no difference, no metrics. The solution has 30 projects (all C#) and one of them is the test project. The 29 projects are all .NET class libraries some of them has XAML files for WPF. I´ve also enabled all of your compatibility mode settings but can´t find any further info. The NCrunch Diagnostic Output window is empty. Is there any information or log about your assembly adjustments which enables the metrics?
Back to top
Remco
#6 Posted : Friday, July 1, 2016 7:33:00 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
Ok, I think we need to do a complete reset of all your configuration and cached data to default settings.

Close the IDE and remove the following files from the solution (rename or delete according to your preference):

- The _NCrunch_solutionname directory adjacent to your solution file
- The .ncrunchsolution file
- The .ncrunchsolution.user file
- All .ncrunchproject files underneath your solution directory

Also remove the globalconfig.crunch.xml file under C:\Users\YOURUSERNAME\AppData\Roaming\NCrunch

If this doesn't restore your inline instrumentation and metrics, could you try submitting a bug report for me? This will allow me to inspect whether the missing data is a result of an internal problem.
Back to top
fsdschmidt
#7 Posted : Friday, July 1, 2016 8:24:11 AM(UTC)
Rank: Member

Groups: Registered
Joined: 6/29/2016(UTC)
Posts: 24
Location: Germany

Was thanked: 2 time(s) in 2 post(s)
Ok I have updated to the newest version now and cleared all the configuration and cached data but unfortunately I have still no metrics. I submitted a bug report with link to this thread and looking forward to your results.
Back to top
Remco
#8 Posted : Friday, July 1, 2016 10:32:10 AM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
fsdschmidt;8910 wrote:
Ok I have updated to the newest version now and cleared all the configuration and cached data but unfortunately I have still no metrics. I submitted a bug report with link to this thread and looking forward to your results.


Thanks for sending through the report. Unfortunately, it doesn't look like this is being caused by an internal problem ... at least, not one that can be recognised in the report.

We need to find out what is different about these projects that is preventing them from being instrumented.

When you say that you see no metrics, does this mean for the entire solution? Does the metrics window report any code at all, or is it just that the coverage percentages are at zero?

What happens if you create a new test project in this solution that isn't attached to any of the other projects? Does this test project show up in the Metrics window? Does it have inline coverage markers?
Back to top
fsdschmidt
#9 Posted : Friday, July 1, 2016 11:34:37 AM(UTC)
Rank: Member

Groups: Registered
Joined: 6/29/2016(UTC)
Posts: 24
Location: Germany

Was thanked: 2 time(s) in 2 post(s)
The new test project is a good approach, the metrics are fine for this project. But when I reference one of the other assemblies and test through real code, I see no metrics and no markers inside our business logic. But the calling test has still markers! I will compare the project files from both test projects, the values of the properties dialog are nearly identical.

In the metrics window I see all assemblies and it reports also the code lines, but all other columns are zero apart from code coverage which has no value. This applies for the entire solution with exception of the new test project.
Back to top
Remco
#10 Posted : Friday, July 1, 2016 12:13:31 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
fsdschmidt;8912 wrote:
The new test project is a good approach, the metrics are fine for this project. But when I reference one of the other assemblies and test through real code, I see no metrics and no markers inside our business logic. But the calling test has still markers! I will compare the project files from both test projects, the values of the properties dialog are nearly identical.

In the metrics window I see all assemblies and it reports also the code lines, but all other columns are zero apart from code coverage which has no value. This applies for the entire solution with exception of the new test project.


Ok, we're making progress here! Now we just need to discover the point of difference between the test project you've added and any one of your other projects in the solution. Examining the XML of the project file using a text editor may help with this - you can copy/paste across the differences until the markers disappear.
Back to top
fsdschmidt
#11 Posted : Monday, July 4, 2016 1:36:39 PM(UTC)
Rank: Member

Groups: Registered
Joined: 6/29/2016(UTC)
Posts: 24
Location: Germany

Was thanked: 2 time(s) in 2 post(s)
So I´ve found the problem and now NCrunch works normal. In this solution we set the PdbFile property for all projects to custom paths:

<PropertyGroup>
<PdbFile>$(PATHVAR)..\Symbols\$(AssemblyName).pdb</PdbFile>
</PropertyGroup>

But unfortunately VS writes the symbol in addition also to the default path. And because of this strange behavior we do also remove the symbol from the default path:

<Delete Files="$(OutputPath)\$(AssemblyName).pdb"/>

I assume that you expect the pdb at the default path and can´t find it. So may be you could analyze the project for the PdbFile property or you could also read the pdb from the output assembly. For now I extended our Delete build step with a NCrunch condition:

<Delete Condition="'$(NCrunch)' == '0'" Files="$(OutputPath)\$(AssemblyName).pdb"/>

Thanks for your help!
Back to top
Remco
#12 Posted : Monday, July 4, 2016 10:40:57 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 7,145

Thanks: 959 times
Was thanked: 1290 time(s) in 1196 post(s)
fsdschmidt;8917 wrote:

I assume that you expect the pdb at the default path and can´t find it. So may be you could analyze the project for the PdbFile property or you could also read the pdb from the output assembly.


This is correct. NCrunch always assumes that the PDB is co-located with the output assembly. If the PDB doesn't exist, it won't be able to read the debug metadata for the DLL/EXE and there won't be any way to cross-reference your code with the IL in the assembly. The result is no coverage tracking of any kind.

This was a good find. Nice work on figuring it out! Your fix is also a good solution and it is the approach that I would recommend.
Back to top
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.066 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download