MCP server

I use AI as a support tool. After each change the AI implements, it runs the relevant unit tests. However, ncrunch does the same thing in parallel. It would be helpful to have an ncrunch MCP server that the AI could query.

isb-ik wrote:I use AI as a support tool. After each change the AI implements, it runs the relevant unit tests. However, ncrunch does the same thing in parallel. It would be helpful to have an ncrunch MCP server that the AI could query.

Yes, I have been looking at this. Allowing AI to query NCrunch has much potential, especially with RDI.

Allowing AI to query NCrunch has much potential

+1!
Nowadays it feels there is a big traction to "sell in" NCrunch internally with colleagues. I used to use NCrunch to demonstrate TDD, but nowadays when AI agents "can write" the tests for you trying to persuade devs to do "TDD manually" feels a bit outdated.

If one could let AI agents use NCrunch engine via MCP; Maybe we can get argue that this will result to even better code, e.g. better "TDD cycle" logic, easily add tests where code is missing coverage, troubleshoot test issues with help of the RDI etc., I think this would be a really big sellpoint.

I hope it happens :)

That would be a MASSIVE +1 !!

Running tests the standard way is so slow. An agent that could benefit from NCrunch would greatly increase my productivity.

@Remco: need to expose some APIs for MCP to consume ?

Yes, in concept this isn't too complicated. NCrunch just needs to expose an MCP server containing the APIs that allow the agent to pull data directly from the engine, and give it basic instructions (i.e. run specific tests, etc).

Where I expect it will be more complicated is in finding sensible ways to represent the data so that the models can make sense of it. This is going to require trial and error to get right, but I'm excited about it.

When you consider how much of a productivity improvement we get from running tests continuously and working with features like RDI, it's reasonable to expect that an intelligent agent could see the same benefits. Building more reliable applications for a fraction of the tokens would be a big win.

I came specifically to ask about NCrunch exposing an MCP server and found this post.
I can hardly tell you how much I want this. Even if it is far from perfect to start with, getting anything out there would probably be revolutionary when working with an agent.

I think a minimal, hugely useful, first release would only need something like this:

1. Current overall status. Number of tests per category [Detected, Successful, Pending, Failed]
2. List failed test.
3. Get the failure message for a specific failed test.

Important to me but probably not to most developers:
1. Enabling the agent to disambiguate by open solution path. (I often work in multiple git worktrees for the same solution at the same time....)

It would be fantastic if something like the above was to turn up!

Really hope this happens.

It's happening. Right now as I type this I have an OpenClaw agent adding tool methods to the MCP server. Using the NCrunch MCP server.

I underestimated this task though, both in its scope and its potential. An MCP server is actually a full blown API that harnesses the engine and allows an agent to use it the same as a user would in an IDE. When you think about what NCrunch offers (build system, test runner, metrics, in-memory solution model), it's actually pretty close to a full blown headless IDE. So an MCP server is actually about empowering an agent to work with a live executed in-memory solution, with ability to make code changes, examine build issues, run tests, read performance info and metrics, and even troubleshoot tests using RDI.

Current plan involves two runtime modes:

1. IDE integrated mode that exposes tools to an IDE-hosted agent (i.e. Copilot, JB AI assistant, etc) so that they can read things from NCrunch and steer it. I think this represents the original intention in this thread. It seems silly that an agent can't run tests with NCrunch or read their results. So this is basically a stdio based system that hooks into the existing NCrunch session.

2. Headless engine exposed via JSON HTTP RPC. A multi-session capable server that exposes MCP capabilities to an agent with the ability to connect to it. This is designed for server-hosted agents like OpenClaw and Hermes. They can talk direct with the MCP server, telling it to start an NCrunch engine session for a given solution file, then fully interact with that engine, telling it to run tests, reading full engine state (build issues, tasks, metrics, etc). No IDE required. Because the engine auto syncs with files on disk, the agent can freely develop the source code and the engine will continuously run tests. The agent can then poll the various APIs for info on when it breaks things. The server supports multiple engines running 24/7, so in theory you can have an always-on coding agent equipped with a script to automatically pull git changes. You could give it instructions to automatically review or refactor code as it gets committed, or run other tasks that involve using NCrunch for verification steps.

I have most of 2 already implemented. I still need to surface the tool methods to the IDE to make 1 a reality.

Current challenges include trying to explain to the agent that NCrunch is an engine, not a static test runner. It seems to want to treat it like 'dotnet test', when NCrunch needs to load a whole solution and build a bunch of stuff before it can even discover the tests and how to run them. It's getting better. My current coding agent has made this project a world easier. It's both exciting and rather strange to watch it improve its own capabilities through the MCP interface while it extends it.

Right now the source code is read only, and the agent can't change configuration. I think that both of those things will need to be configurable when they eventually get added, since there are clear trust limits with this sort of thing.

I also still need to figure out how to expose RDI data. I think I will do this in a later update.

It's hard to make promises on timeframes. I wanted this to be out as part of the 5.22 release, but the pending Rider 2026.2 release might force me to push 5.22 before the MCP system is fully production ready. It would also be nice if the first build is somewhat battle tested before everyone points their agents at it, since I have watched some remarkable stupidity happen around it too :) We'll have to see how it goes over the next week. Maybe 5.23 will be released shortly after 5.22.

That all sounds absolutely fantastic!

I can't wait to use it :)

The MCP server is now available in v5.22 (released 5 minutes ago).

Hi @Remco, thank you for this amazing work on the MCP server.

Question: once the MCP server is running on http, how can a client use the secret/password to authenticate ?

arkiaconsulting wrote:Question: once the MCP server is running on http, how can a client use the secret/password to authenticate ?

Sorry about this! It was supposed to be in the documentation. I've updated the MCP server page with these details now.

Trying out the MCP. Got this error (long paths are enabed, and reboooted after the registry change)

\[NCrunch.Engine] [PID:10112 12:16:08.3197 InitialisationQueueTask-18] Build sandbox: D:\cachefiles\ncrunch_workspace\BuildSystem\10112\NotApplicable\nCrunch.BuildIntegration.Net6.dll
[NCrunch.Engine] [PID:10112 12:16:08.3197 InitialisationQueueTask-18] Build sandbox: D:\cachefiles\ncrunch_workspace\BuildSystem\10112\NotApplicable\nCrunch.TaskRunner.Net6.x64.deps.json
[NCrunch.Engine] [PID:10112 12:16:08.3197 InitialisationQueueTask-18] Build sandbox: D:\cachefiles\ncrunch_workspace\BuildSystem\10112\NotApplicable\nCrunch.TaskRunner.Net6.x64.dll
[NCrunch.Engine] [PID:10112 12:16:08.3197 InitialisationQueueTask-18] Build sandbox: D:\cachefiles\ncrunch_workspace\BuildSystem\10112\NotApplicable\nCrunch.TaskRunner.Net6.x64.runtimeconfig.json
[NCrunch.Engine] [PID:10112 12:16:11.2107 InitialisationQueueTask-22] ERROR (Load): Error occurred during load of component at D:\dev\xyz\xyz1\backend\test\IntegrationTests\IntegrationTests.csproj: NCrunch was unable to load this project because it contains a file path that exceeds the internal file path or directory length limit. The fully qualified file name must be less than 260 characters, and the directory name must be less than 248 characters. The first file in the project to cause this error can be found at: D:\dev\xyz\xyz1\backend\test\IntegrationTests\Features\ContractDeals\UploadPayAsNominatedTransferTests.PayAsNominatedTransfer_CreatesCorrectContractDeals_testCaseName=xyz2De_contractPortfolioGroup=None_negativeSpotPriceCurtailment=Buy_contractDirection=0.3.verified.txt
[NCrunch.Engine]

Hi, thanks for sharing this.

Proper support for long file names in NCrunch is effectively impossible, because large areas of the underlying platform still don't support it.

Something looks wrong with this file name. Does it exist under this name in your foreground solution too? It looks like the output from a malfunctioning build step.

The MCP HTTP host exits if a client disconnects while a tool call is still running, taking every session with it.

McpHttpServer.HandleRequest catches the HttpListenerException from writeJsonResponse correctly, but the recovery path faults: the write threw at OutputStream.Write before reaching Close(), so CanWrite is still true, and writeEmptyResponse then sets ContentLength64 on an already-submitted response. That InvalidOperationException comes from inside the catch block, and Run() has only try/finally, so it reaches Main and the process exits - with _sessionHost.Shutdown() running on the way out.

Repro: StartSession on a solution whose load outlasts the client's per-request timeout, then call any tool while it is loading.

Thanks for flagging this. I'll check it out.