Forum member

samholder

100 posts

Posts by samholder

  1. Specflow [BeforeTestRun] and [AfterTestRun] methods are being called multiple times

    Hi. We are using Specflow 3.5 and XUnit as the test flavour to generate the tests. We are on NCrunch 4.3.0.13 currently, awaiting new licenses before we can upgrade... I thought I understood the parallisation model used by NCrunch, but I'm seeing behaviour which conflicts with that understandi…

    26 Nov 2020 13:50 UTC

  2. Grid is not always running all tests on team city

    Thanks Michael. That seems to work, at least after 1 test run. What is the default engine mode if not specified on the command line?

    18 Aug 2020 19:44 UTC

  3. Grid is not always running all tests on team city

    we're on: NCrunch Console Tool v4.3.0.13 do I need a newer version? as it seemed to complain when I added it

    18 Aug 2020 09:46 UTC

  4. Grid is not always running all tests on team city

    looking at the build log, its large, 13mb. I see this: NCrunch: The node responsible for executing this test has truncated the trace output because it exceeds the configured limit of 50000 characters so I'm wondering if the logs elements that tell TC that tests passed are not being reported…

    17 Aug 2020 16:59 UTC

  5. Grid is not always running all tests on team city

    Thanks for the suggestions. Checking the NCrunch report and i see this: Tests passed: 2090, failed: 0, not run 0 which indicates that all the tests are running, but just TC is not picking them up. The timeline report looks ok (ie its all green, and I don't see any connection issues) anythin…

    17 Aug 2020 16:56 UTC

  6. Grid is not always running all tests on team city

    We have an issue where sometimes not all tests are run on the grid nodes. Our solution has about 2,000 tests, and I see test runs which TeamCity reports as having tests passed of 897, 2091, 1484, 2090, 129 in subsequent runs. When I look in the build log for the latest run (with 897 tests reported…

    13 Aug 2020 13:06 UTC

  7. .NET Core test project does not pick up settings from app.config or appsettings.json

    I've been able to load up the appsettings.json during the tests by doing it manually, which is sufficient for me for now.

    17 Jun 2020 15:44 UTC

  8. .NET Core test project does not pick up settings from app.config or appsettings.json

    I'm migrating a project that was on .net framework to .net core 3.1. I'm used to behaviour where if a test project has settings in the App.config, those settings get 'transferred' to the app.config of the NCrunch test execution application so that they are available if the tests do a Configurati…

    09 Jun 2020 10:43 UTC

  9. NCrunch reports tests as failing even though all tests pass, and no indication why.

    Thanks! yeah that shows which test is failing, though no exception details or anything, just a single line of output, but that at least gives us somewhere to start...

    13 Mar 2020 07:24 UTC

  10. NCrunch reports tests as failing even though all tests pass, and no indication why.

    We have a build which runs some specflow (v3.1.74 MSTest) and selenuim based UI tests, and runs them on a few grid nodes in teamcity. We had a problem recently where the build would report all tests as passing but then output this in the log: [17:53:40]E: [Step 10/13] NCrunch result: TestFailure…

    12 Mar 2020 19:49 UTC

  11. Tests not distributed evenly across grid nodes

    thanks for the reply. The tests don't use the AtomicAttribute. We are using Specflow which is using MSTest as the testing framework. The [BeforeTestSetup] methods generally do some process killing to make sure any process hanging around from a previous test run is killed before we start. they al…

    03 Mar 2020 10:22 UTC

  12. Tests not distributed evenly across grid nodes

    Any ideas on this. We have updated to latest builds and are still seeing this behaviour. The causes a problem in some long running smoke/integration tests we have as we have tried to pick tests such that they take about 15 mins to run when evenly distributed, but this means that sometimes they c…

    02 Mar 2020 16:47 UTC

  13. exception running tests on the grid

    I'll try and do this but not super easy as lots of machines to update, and happens infrequently. Will this speculative fix be included in the main NCrunch release? It might be easier to schedule the update at some later date with a released version

    27 Feb 2020 10:23 UTC

  14. Tests not distributed evenly across grid nodes

    Hi, I'd like to try and understand some strange behaviour we see occasionally. We are running some UI tests that take a while (some minutes each) and have 5 nodes in the pool of available grid nodes. In this one test case there were 6 tests, but despite all nodes doing nothing, the distribution w…

    30 Jan 2020 15:26 UTC

  15. Performance enhancement for grid nodes

    Thanks for the reply, yeah now we are aware of the issue then we can mitigate it. Glad to know its been considered and rejected for valid reasons.

    23 Jan 2020 14:11 UTC

  16. Performance enhancement for grid nodes

    We have seen an issue which I think could be optimised. We have some unbalanced grid nodes in terms of number of max concurrent processing they can do. We have some nodes that have 5 threads allowed and some that have 1 thread allowed. We have some builds that can only use the single thread nodes…

    22 Jan 2020 13:44 UTC

  17. exception running tests on the grid

    Sorry, missed the notification about the response. This is happening on a grid node, so we see it in the build log on teamcity.

    17 Jan 2020 17:08 UTC

  18. exception running tests on the grid

    When running tests we sometimes get all tests pass but the run fails with this exception [14:17:49][Step 8/12] [?-14] ERROR (Internal): System.IO.IOException: The process cannot access the file 'C:\Users\username\AppData\Local\NCrunch\hashcache' because it is being used by another process. [14:1…

    10 Jan 2020 14:40 UTC

  19. IndexOutOfRange exception when running project on build server

    Having looked at the changes it looks to me like someone added a temporary test ('fdfdg' is a bit of a give away), then probably deleted the test from the feature file and committed before the feature.cs was regenerated, so I suspect the lines with the high numbers no longer existed in the feature f…

    09 Oct 2019 09:38 UTC

  20. IndexOutOfRange exception when running project on build server

    Thanks for the pointers. I have cleared the cache for all the machines running the console tool, and cleared the grid node machines working folders and set the option to not instrument. It doesn't seem to make any difference. This is the log from the build server (up to the point of the error): …

    20 Sep 2019 13:58 UTC