Available for NCrunch is a console tool that can operate without the need for Visual Studio. This tool has been designed primarily for use on build servers, though it can also be used in other scenarios as a replacement for build runs and testing tools. There are many advantages to using this tool, as it enables many of NCrunch's features (such as parallel execution and distributed processing).
This guide deals primarily with setting up the console tool on a team build server for continuous integration purposes. For more detailed information about the tool, please refer to the console tool documentation.
The NCrunch console tool is essentially a headless version of NCrunch. It performs the same build and test actions in the same manner as the Visual Studio plugin. This means that the build server using it must have installed all the necessary SDKs and toolsets required to build the target solution using MSBuild, along with any dependencies required by test code that will be running on the build server.
Most teams already have an existing build server that meets these requirements, as this server is likely to already be building their solutions and running tests. If you're uncertain whether your server is able to build your solution, it's often worth copying over the solution and running MSBuild.exe against it to verify that the server is correctly prepared. You may also want to try running your tests using a console version of your test framework to check for additional dependencies.
To use NCrunch with CI, you'll need to already have installed your CI server (i.e. TeamCity, CruiseControl .NET, etc).
The console tool can be installed either via MSI, or a ZIP file. The MSI doesn't include any special installation steps. Installation itself is essentially an xcopy deployment.
Once the tool has been installed, you'll need to create a configuration file in order to use it. This can be skipped if you want to use the default configuration, though note that the default NCrunch configuration is fairly minimal and isn't prepared for parallel execution or distributed processing.
The configuration file used by the tool is the same as the global configuration file used by the NCrunch Visual Studio plugin. On any machine with the Visual Studio plugin installed, the file can usually be found at:C:\Users\[USER]\AppData\Roaming\NCrunch\globalconfig.crunch.v3.xml
If you have a pre-configured workstation with configuration that is similar or identical to that which should be used on the build server, you can freely copy this file over to the build server.
Alternatively, running the console tool (ncrunch.exe) on the build server without any parameters will generate a new configuration file at the same above location containing default settings.
Once you have a configuration file on the build server, you can freely edit this file using any text editor. Most of the settings in this file are irrelevant to the console tool. The most important setting in this file is 'MaxNumberOfProcessingThreads' which should be set to a sensible value (often equal to the number of CPU cores on the machine). As there's no need to reserve fast lane threads on a build server, it's also worth setting the 'FastLaneThreads' setting to '0'.
Because the 'GridNodeAddresses' setting contains encrypted password information, you'll need to copy this setting over from a pre-configured workstation if you wish the console tool to use distributed processing.
It's often worth storing your configuration file in a well known location (i.e. 'C:\NCrunchConfig\GlobalConfig.xml') instead of leaving it under the user profile. This is because many CI tools tend to run under a different user profile to the server's desktop user, leading them to resolve configuration from unexpected places. By storing your config file in a known location and specifying this location using a command line parameter, you can always be sure of which configuration is being used.
An important configuration setting worth investigating when using the console tool is the 'NCrunch cache storage path' solution-level setting. Because most CI systems will regularly perform a fresh checkout from the VCS, they may often end up running NCrunch with no cached data present. This cached data is important, as it contains timing information from previous test runs that is used by the engine to optimise future execution. A way to work around this problem is to set the 'NCrunch cache storage path' to an absolute path on the build server. In this way, it will always be present any fresh checkout from VCS. It's possible to specify an environment variable inside the setting if you need cached data to be placed differently when the solution is opened on regular development workstations.
Before the tool can be used, you'll need to specify a license for it. This is simple to do - just run the following command:ncrunch.exe /license
You'll receive a popup dialog allowing you to choose a license. Note that the tool also supports a 30-day evaluation license.
If you're already a licensed NCrunch user, you won't need to buy a new license to use this tool. Simply place your existing license into the license dialog. See here for more information about how licensing works for the NCrunch console tool.
Before hooking the tool up to your CI system, it's important to give it a quick test via the command line. This can help save time later, as CI tools add an extra layer of complexity that can make troubleshooting difficult if the tool is behaving incorrectly.
If you don't already have a copy of your solution on the build server, copy it over or VCS check out to a temporary location.
Now run the following command:ncrunch.exe SOLUTION.sln /c CONFIGFILE /o NCrunchResults
For example:ncrunch.exe c:\MySolution\MySolution.sln /c c:\NCrunchConfig\GlobalConfig.xml /o NCrunchResults
You should see the tool kick off and run a full build and test pass of the solution using the configuration you've specified. The results of this test run will be written to reports stored inside: c:\MySolution\NCrunchResults\
Open NCrunchResults\AllResults.html using a web browser for detailed information about what happened during the execution run. Check for build and load failures. If you hit a failure, you may need to examine the state of the solution and check whether you have any SDKs missing from the build server.
Assuming all is OK, you're now set to hook the tool up to your CI system.
Because you're able to run MSBuild indirectly through NCrunch, there is often less need to have a separate MSBuild step inside your CI.
Sometimes it can be useful to disable these separate build steps to improve the speed of your build. Note that NCrunch is not designed to produce build artifacts from release builds, so you should only do this if you are using your CI for verification only and not for producing installers or deployment packages.
Configuring the NCrunch console tool with Jetbrains' Team City is a simple process. Once you've identified which build you'd like to run with NCrunch, enter the following settings:Build Configuration Settings -> General Settings -> Artifact paths
Add: NCrunchResultsBuild Steps -> Add build step
Runner type: Command Line
Step name: NCrunch
Command executable: C:\Program Files\Remco Software\NCrunch Console Tool\NCrunch.exe (or wherever you installed the tool)
Command parameters: %system.teamcity.build.workingDir%\MySolution.sln /o NCrunchResults (.sln must be relative to checkout directory)
All other settings should be left to their default values.
That's it! NCrunch will automatically detect Team City when it runs, and will output real-time information about running tests, builds, failures, etc. This allows Team City to track NCrunch results in a similar way to other build/test steps.
Once a build has run, you'll be able to find the report data inside the build artifacts.
The design of TFS makes it difficult to cleanly integrate the NCrunch Console Tool into the continuous integration system. However, it is still possible to do this by piggybacking the console tool execution on MSBuild. The solution involves adding a custom project to the solution that will execute the console tool as part of the build, then storing the output from the tool inside the TFS drop folder.
Please note that the following instructions have only been tested with TFS 2013, although in principle it should be possible to use the same approach with other versions as all versions of TFS invoke MSBuild.
To use the NCrunch Console Tool with TFS, follow these steps:
Install the console tool on the machine TFS is to use for executing builds. If installing the tool manually from ZIP, don't forget to set up the environment variable.
Add a new project to your solution. You can use any of the standard project templates available in VS. A C# class library is fine.
Remove all references and files from the newly added project. Unload it by right-clicking it in the solution explorer.
Edit the project file by right clicking it in the solution explorer, or by using any text editor. Replace the contents of the file with:
Right click the project file in VS and choose to reload it.
Go to your NCrunch configuration for the newly added project. Turn on the 'Ignore this project completely' setting so that NCrunch doesn't give you noisy errors about it. NCrunch shouldn't try to build this project because it isn't a supported project type, and if it actually did work, it would be a recursive nightmare.
Go to the Visual Studio Configuration Manager (from the build configuration toolbar drop-down).
Ensure that the newly added project is not selected to be built from any of your standard build configurations - it isn't designed to be built outside TFS.
Create a new build configuration. You can name it something like 'TFS NCrunch Build'. DON'T choose to update your project files.
Under the new build configuration, turn off the build for ALL projects in your solution except for the new one you've added. This configuration will be used for TFS, and we usually want TFS to avoid building anything but the NCrunch project, as NCrunch will build the other projects in its own sandbox anyway. If you are using TFS to produce artifacts for release builds, then you should skip this step.
Open up your TFS server configuration. Edit your TFS build definition. Go to Process -> Build -> Configurations and make sure that you choose your newly added build configuration for the TFS build.
Check your new code/project into TFS. Make sure you also check in the .ncrunchproject file, as this contains the configuration entry that will tell NCrunch to avoid crunching the build runner project. TFS should now execute the NCrunch Console Tool using MSBuild. You can retrieve the build and test results from the TFS drop folder.