Welcome Guest! To enable all features please Login or Register.

Notification

Icon
Error

Provide teamcity metrics in Console Tool
albertomonteiro
#1 Posted : Thursday, July 23, 2015 12:51:35 PM(UTC)
Rank: Member

Groups: Registered
Joined: 5/28/2012(UTC)
Posts: 17
Location: Fortaleza, Ceara - Brasil

Thanks: 11 times
Was thanked: 3 time(s) in 3 post(s)
I am using NCrunch Console Tool.

Since that doesnt offer a result in xml with final coverage result I am using "RawCoverageResults.xml" to discover that, and using values to report to TeamCity writing on "teamcity-info.xml" file.
You can check TeamCity documentation here: https://confluence.jetbr...hart-CustomBuildMetrics

But my ScriptCs script doesnt calculate the same final value that is generated in ncrunch html report, I wont use html to fill this info.
Whats is wrong with my calculation code? There is some way to NCrunch Console write the teamcity-info.xml file with the code coverage statistic of teamcity?

Code:
using System;
using System.Globalization;
using System.IO;
using System.Linq;
using System.Xml.Linq;
using System.Text.RegularExpressions;

var path = Path.Combine(Env.ScriptArgs[0].ToString(), @"\RawCoverageResults.xml");

const string CLASS_IGNORE_PATTERN = @"(?<class>.+\.cs):";
const string PROJECT_IGNORE_PATTERN = @"^(?!.+?\.cs:).*\\(?<project>.+)\.csproj$";
const string FOLDER_IGNORE_PATTERN = @".+\\(?<folder>.+\.csproj\\.+)";

var mainElement = XElement.Load(Path.Combine(Env.ScriptArgs[1].ToString(), @"Nash\OnlyNash.v2.ncrunchsolution"));

var exclusoes = mainElement.DescendantsAndSelf("MetricsExclusionList").FirstOrDefault();

var blockedProjects = new List<string>();
var blocked = new List<string>();

if(exclusoes != null)
{
    var content = exclusoes.Value.Trim();
    var classesToIgnore = Regex.Matches(content, CLASS_IGNORE_PATTERN).Cast<Match>().Select(m => m.Groups["class"].Value).Distinct();
    var projectsToIgnore = Regex.Matches(content, PROJECT_IGNORE_PATTERN, RegexOptions.Multiline).Cast<Match>().Select(m => m.Groups["project"].Value).Distinct();
    var foldersToIgnore = Regex.Matches(content, FOLDER_IGNORE_PATTERN).Cast<Match>().Select(m => m.Groups["folder"].Value.Replace(".csproj","")).Distinct();
    

    blockedProjects.AddRange(projectsToIgnore);
    blocked.AddRange(classesToIgnore);
    blocked.AddRange(foldersToIgnore);
}

var sourceFileElements = XElement.Load(path)
    .DescendantsAndSelf("project")
    .Where(x => !blockedProjects.Contains(x.Attribute("name").Value))
    .DescendantsAndSelf("sourceFile")
    .Where(x => !blocked.Any(b => x.Attribute("path").Value.Contains(b)))
    .ToList();

var lineElements = sourceFileElements
    .DescendantsAndSelf("line")
    .ToList();

var totalLines = lineElements.Count;
var totalCoveredLines = lineElements
    .Select(x => Math.Min(1, int.Parse(x.Attribute("coveringTests").Value)))
    .Sum();

var totalClasses = sourceFileElements.Count;
var totalCoveredClasses = sourceFileElements
    .Count(x => x.DescendantsAndSelf("line").Any(ele => ele.Attribute("coveringTests").Value != "0"));

var linesCoverage = (totalCoveredLines * 1.0) / (totalLines * 1.0);
var classesCoverage = (totalCoveredClasses * 1.0) / (totalClasses * 1.0);

var xElement = new XElement("build",
                            new XElement("statusInfo",
                                         new XElement("text", string.Format("Coverage: {0:P}", linesCoverage),
                                                      new XAttribute("action", "append"))),
                            CreateStatisticElement("CodeCoverageC", (classesCoverage * 100).ToString("F", new CultureInfo("en-US"))),
                            CreateStatisticElement("CodeCoverageL", (linesCoverage * 100).ToString("F", new CultureInfo("en-US"))),
                            CreateStatisticElement("CodeCoverageAbsLCovered", totalCoveredLines.ToString()),
                            CreateStatisticElement("CodeCoverageAbsCCovered", totalCoveredClasses.ToString()),
                            CreateStatisticElement("CodeCoverageAbsLTotal", totalLines.ToString()),
                            CreateStatisticElement("CodeCoverageAbsCTotal", totalClasses.ToString()))
                            );

File.WriteAllText("teamcity-info.xml", xElement.ToString());
Console.WriteLine(xElement.ToString());

public XElement CreateStatisticElement(string key, string value)
{
    var keyAttribute = new XAttribute("key", key);
    var valueAttribute = new XAttribute("value", value);
    return new XElement("statisticValue", keyAttribute, valueAttribute);
}



TeamCity code coverage stats:


  • CodeCoverageB Block-level code coverage %
  • CodeCoverageC Class-level code coverage %
  • CodeCoverageL Line-level code coverage %
  • CodeCoverageM Method-level code coverage %
  • CodeCoverageAbsLCovered The number of covered lines int
  • CodeCoverageAbsMCovered The number of covered methods int
  • CodeCoverageAbsCCovered The number of covered classes int
  • CodeCoverageAbsLTotal The total number of lines int
  • CodeCoverageAbsMTotal The total number of methods int
  • CodeCoverageAbsCTotal The total number of classes int
Remco
#2 Posted : Thursday, July 23, 2015 10:34:06 PM(UTC)
Rank: NCrunch Developer

Groups: Administrators
Joined: 4/16/2011(UTC)
Posts: 6,974

Thanks: 929 times
Was thanked: 1256 time(s) in 1169 post(s)
Hi,

It's great to see that you've come up with a way of exporting results to Team City in a format that it can recognise. Does the export work correctly? I made an attempt at something similar when first introducing the NCrunch Console Tool, but couldn't find a way to integrate with Team City. Is this Team City code coverage import feature new?

Trying to align coverage export algorithms can be a tremendously fiddly thing. Unfortunately I am not set up correctly to troubleshoot your algorithm directly against the NCrunch one (which is quite integrated into the product and therefore more complex). Looking at the code, however, I wonder if there may be a misalignment in how the metrics exclusions are being applied to the calculation. The exclusions were a major point of complexity when trying to build the NCrunch code itself because of their hierarchical nature. Excluding a directory would also mean excluding all files within directories held within the directory, etc.

I am not at the moment aware of any 'magic' performed by the NCrunch routine that you haven't in some way tried to account for.

Do you receive the correct result when working in a simple project with no metrics exclusions? The best way to align coverage between algorithms is to avoid troubleshooting it against large solutions with tons of data. Try building some small sample solutions that each make use of a variation on their coverage - for example, one has a metrics exclusion at project level, another at directory level, another at file level, etc.

With NCrunch, the HTML report metrics should be identical to the metrics shown in the Metrics Window. Relying on this may help you to troubleshoot your routine faster as you won't need to export HTML over and over again.

Finally, its worth mentioning that NCrunch collects/reports metrics on a file-by-file basis, rather than class-method basis. I think you're already aware of this, but thought I'd mention it to be sure. Tools like Team City weren't really intended to work this way and the only way to provide data to them is to discard method-level data and pretend that files are classes.
Users browsing this topic
Guest
Forum Jump  
You cannot post new topics in this forum.
You cannot reply to topics in this forum.
You cannot delete your posts in this forum.
You cannot edit your posts in this forum.
You cannot create polls in this forum.
You cannot vote in polls in this forum.

YAF | YAF © 2003-2011, Yet Another Forum.NET
This page was generated in 0.034 seconds.
Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download