Console tool, custom environment variables with connection string not picked up

[NCrunch Console Tool v3.17.0.2

I cannot get it to work with supplying custom environment variables via a generated config file, it does pick up the other two though. What am I doing wrong? :|

[spoiler]
[13:33:20][Step 6/8] Removing old 'X:\_tmp\ncrunchConfig.xml'.
[13:33:20][Step 6/8] Writing X:\_tmp\ncrunchConfig.xml.
[13:33:20][Step 6/8] NCrunchXmlConfig
[13:33:20][NCrunchXmlConfig] <GlobalConfiguration>
[13:33:20][NCrunchXmlConfig] <Settings>
[13:33:20][NCrunchXmlConfig] <GridServerReferencesForComputer>
[13:33:20][NCrunchXmlConfig] <Value>NODE&gt;zzz=</Value>
[13:33:20][NCrunchXmlConfig] </GridServerReferencesForComputer>
[13:33:20][NCrunchXmlConfig] <CustomEnvironmentVariables>
[13:33:20][NCrunchXmlConfig] <Value>TestsConnectionString = Server=(local);Initial Catalog=NCrunch_Proj_TCAgent-Agent2;Integrated Security=SSPI</Value>
[13:33:20][NCrunchXmlConfig] <Value>IgnoreTestsResettingDb = True</Value>
[13:33:20][NCrunchXmlConfig] <Value>ResetDbIfSchemaIsOutdated = True</Value>
[13:33:20][NCrunchXmlConfig] </CustomEnvironmentVariables>
[13:33:20][NCrunchXmlConfig] </Settings>
[13:33:20][NCrunchXmlConfig] </GlobalConfiguration>
[13:33:20][Step 6/8] &"X:\_tmp\NConsole\NCrunch.exe" Proj.sln /o "X:\_tmp\NCResults" -NCrunchCacheStoragePath "X:\_tmp\NCCache" -WorkspaceBasePath "X:\_tmp\NcWc" /vs "2012" -ConsiderInconclusiveTestsAsPassing True -MaxNumberOfProcessingThreads 5 /TeamCityDisableTestNotRunFailureReporting -AnalyseExecutionTimes "False" -PipelineOptimisationPriority "Throughput" /C "X:\_tmp\ncrunchConfig.xml" -TestsToExecuteAutomatically "((DoesNotHaveCategory 'LongRunningTest' AND DoesNotHaveCategory 'HttpIntegrationTest' AND DoesNotHaveCategory 'SystemTimeModifyingTest') AND (HasNotBeenRun OR IsImpacted OR IsFailing OR FullNameMatchesRegex '\.SchemaOnDiskIsLatest$'))" -AlignOutOfDateStatusWithImpactStatus "True"
[13:33:21][Step 6/8]
[13:33:21][Step 6/8] NCrunch Console Tool v3.17.0.2
[13:33:21][Step 6/8] Copyright ¸ 2010-2018 Remco Software Ltd
[13:33:21][Step 6/8] Usage of this tool is permitted only under the terms described in License.rtf
[13:33:21][Step 6/8]
[13:33:21][Step 6/8]
[13:33:21][Step 6/8]
[13:33:21][Step 6/8] This product is licensed to xxx (support/maintenance expires 13-Mar-2019)
[13:33:21][Step 6/8]
[13:33:21][Step 6/8] [?-1] Initialising UI Services
[13:33:21][Step 6/8] [?-1] NCrunch Services Initialised
[13:33:21][Step 6/8] [?-1] Publishing Event: [EngineModesChangedEvent]
[13:33:22][Step 6/8] [Core-7] Console tool is using engine mode: [Undetermined]
[13:33:22][Step 6/8] [Core-7] Beginning end-to-end run of solution using tools from VS version 'VS2012'
[13:33:22][Step 6/8] [Core-7] Now dumping raw settings. Note that not all of these settings are used by the console tool:
[13:33:22][Step 6/8] [Core-7] Check for updates = 'True'
[13:33:22][Step 6/8] [Core-7] Project config file storage path = ''
[h][13:33:22][Step 6/8] [Core-7] Custom environment variables = 'nCrunch.Core.CustomEnvironmentVariable[]'
[13:33:22][Step 6/8] [Core-7] ResetDbIfSchemaIsOutdated = true
[13:33:22][Step 6/8] [Core-7] IgnoreTestsResettingDb = true[/h]
[13:33:22][Step 6/8] [Core-7] Tests to execute on this machine = 'True'
[/spoiler]

Thanks

Array-based settings like the CustomEnvironmentVariables are handled by the configuration system in a simplified way, where the values for the settings can only be declared at one level.

For example, you may have some CustomEnvironmentVariables declared at global level, and some more declared at solution level. When the setting is defined at solution level, it will override anything declared at global level. My guess would be that you have this defined at multiple levels and the bottom level one is overriding the other values.

Hrm so if I understand it correctly NCrunch picks up settings in the following order

global config or "/c configFile" switch to console -> solution config -> "-setting val" switch to console or solution .user config.

But since there are variables I cannot declare via console parameters, I must supply them via "/c configFile", but then the team settings override it.
So basically every value that I cannot specify via "-setting val", I must ensure developers are not adding in solution config because then it breaks the CI build settings? That doesn't sound right?

Thanks.

GreenMoose wrote:
But since there are variables I cannot declare via console parameters, I must supply them via "/c configFile", but then the team settings override it.
So basically every value that I cannot specify via "-setting val", I must ensure developers are not adding in solution config because then it breaks the CI build settings? That doesn't sound right?

That's correct. Putting it that way, there does seem to be a deficiency here. Which setting is being declared in the solution config that is causing problems for you? Could it be declared in the .user files instead?

Edit: One other option would be to provide the setting inside the engine mode being used by the console tool. Engine mode settings will take precedence over those declared at solution level.

Its environment variables that are defined in solution file. Currently for me I can simply remove them from there since I am currently the only one in the team for this project :)
(But those settings override default app.config which determines "Ncrunch or not behavior", that's why it's in solution file so when someone runs without Ncrunch they get correct behavior as well as running with Ncrunch.

* Edit: I'll look into engine mode

So I added the custom environment variables as an engine mode and it now correctly picks it up


My current config is as follows:
[spoiler]
<GlobalConfiguration>
<Settings>
<GridServerReferencesForComputer>
<Value>node1&gt;ZZZ=</Value>
</GridServerReferencesForComputer>
</Settings>
<EngineModes>
<EngineMode>
<Name>TeamCityEngineMode-2018-06-25T09:29:57</Name>
<Settings>
<CustomEnvironmentVariables>
<Value>TestsConnectionString = Server=(local);Initial Catalog=yada;Integrated Security=SSPI</Value>
<Value>IgnoreTestsResettingDb = True</Value>
<Value>ResetDbIfSchemaIsOutdated = True</Value>
</CustomEnvironmentVariables>
</Settings>
</EngineMode>
</EngineModes>
</GlobalConfiguration>
[/spoiler]
1) What happens is solution config also has engine mode with same name? I guess that engine mode overrides the "global" one above ? (which is why I always generate a new name for it above)
2) What about grid nodes? If those are defined in solution config I guess those overrides the ones I have set in my global config file? (And since they cannot be set in an engine mode it seems like a dead end on that one?)

Thanks.

You've won the argument on specifying additional config settings via the command line. We're trying to work out how to proceed with this at the moment.

1) NCrunch identifies the engine modes internally using a combination of their name and the scope they're applied in (i.e. global/solution), so internally there's no weird stuff that would happen with the engine modes having the same name. However, the console tool command line can only identify an engine mode by its name, so there is ambiguity here. It looks like the code will use the first engine mode it will find with a matching name. Based on the sequence the configuration gets loaded, I would expect this will probably be the global one. To avoid confusion, it is recommended that you do not create engine modes with the same name.

2) I can't think of a way to override the grid nodes for a console run, if they've been defined in a solution-level file. This would need to be handled by changing where you declare your configuration settings. Or wait for us to implement the extensions for parsing more complex settings via the command line :)