V2 Documentation
Search

Quick Start Guide

Installation

If you haven't already, first go and download an installer for the version of Visual Studio you are working with.

There are four separate installers for four different versions of Visual Studio (2008/2010/2012/2013). Installing NCrunch for one version of Visual Studio should have no impact on other versions you may have running on your machine.

Once you've downloaded and stepped through the MSI, NCrunch should be ready to go. Note that if you had Visual Studio open during the installation, you'll need to restart your IDE before NCrunch will load.

First Time Run

For your first run with NCrunch, it's often a good idea to try it out with a small, well tested solution that you understand well. This should give you a bit more space to get your head around how the background processing works. NCrunch also works very well with large solutions, but often requires a bit more configuration to reach the best possible experience.

By default, NCrunch will always start as disabled for a solution before it is explicitly turned on from the top menu.

Once you've opened up your solution, select Enable from the NCrunch Visual Studio menu. NCrunch should wake up and show a wizard to help you configure it for your machine, and the solution you're working in. It's worth stepping through this wizard and answering the questions, as this will save you from needing to configure the individual settings yourself.

After running through the wizard, you can get a handle on what's going on under the hood by opening up the Processing Queue from the top menu. This should open up a dockable tool window that shows the inner workings of the NCrunch engine.

The processing queue is a useful way to get detailed information about the background processing, such as finding out which tests are being run, how the work is prioritised, etc. You could consider it your 'what the hell is going on?' view that you can check any time while you work.

If all is well, you should have a nice long list of blue ticks down the side of the queue by the time the engine has finished its first run. If you have any red crosses, possibly this means you are using some kind of custom build configuration that will need some investigation. Have a look at troubleshooting project build issues for more information on how to fix situations like this.

During its first full build and test cycle, NCrunch will keep track of the execution times of tests within your solution along with details about the build dependencies between your classes. It will then use this information to optimise future test cycles to give feedback as fast as possible.

Analysing Test Results

Now that you've cleared a full cycle of data from the engine, it's time to make sense of it. The processing queue will show some summary details about the tests that ran in each of the tasks processed by the engine, but this isn't an ideal way to find out what actually happened. So it's now time to close the processing queue and instead open up the Tests window from the top menu.

The Tests Window is a useful window to keep open while you're developing. It provides immediate feedback on all relevant output from the engine (including failed builds and failing tests). It will also give some information about which tests/builds are being run at a particular point in time (watch out for the clock icons).

Depending on the state of your solution and how you've engineered your tests, you may see some failures here. NCrunch does everything it can to run your automated tests in the same way as you usually would, but sometimes tests are programmed with certain environmental considerations that NCrunch isn't aware of. If you have any test failures that don't look right to you, have a look at troubleshooting test issues.

Working With Code Coverage

So... now we know what's going on under the hood, and we can see the results that come from it... Now it's time for the best bit - inline code coverage and performance metrics!

As part of it's build process, NCrunch will automatically instrument your source code and perform code coverage analysis while processing your tests. The analysis is language agnostic and shouldn't interfere with your runtime code.

When you have a look through some of your classes known to be covered by your tests, you should see a line of coloured circles (markers) stretching down the left margin. These markers indicate test code coverage for their adjacent lines of code.

The default colouring of these markers is explained below:

Green marker: Line is covered by passing tests

Red marker: Line is covered, but by at least one failing test

Black marker: No tests are covering this line (feel the fear!!!)

Hot Spot marker: Line is covered by tests but executes slowly

In Visual Studio 2010 and above, the markers will fade out when they are indicating coverage for tests that haven't been run against the latest version of your source code. In Visual Studio 2008, this is shown by the markers shrinking in size.

Note that you can also change these colours at any time. It's also possible to change the marker shapes.

Where exception stack traces are available for your tests, these will be projected over the markers as a series of red Xs showing where the exceptions were raised.

At any time you can hover the mouse over a marker for tooltip information on the number of tests covering the line, exception details, etc. You can also click on a marker to access a popup menu showing all covering tests. Double-clicking a test in the popup will navigate the IDE to the first line of the test in its fixture. You can right-click on a marker for a context menu providing more options.

The constant on-the-fly analysis of your test coverage allows NCrunch to maintain up-to-date mappings between the code coverage and your source code. These mappings are retained across multiple test cycles and always find their way back to the line of code they relate to. This means that it's possible for the engine to kick off a long running test for 20 minutes, then still be able to make constructive use of the code coverage returned by the test when it completes - even if you made several code changes while the test was executing.

If you have regions of code that you don't want NCrunch to analyse (for whatever reason), you can also tell NCrunch not to instrument them by using special inline comments.

Daily Use

As a concurrent test runner, it's recommended that you let NCrunch take responsibility for the day-to-day running of suitable tests. You can use the inline code coverage to help give you immediate feedback on tests you may be working on, along with providing details on which parts of your codebase are safe to touch.

If (for whatever reason) you want to retain control over when the tests are run, you can set the engine to run tests manually through the NCrunch->Set Engine Mode sub-menu. Note that NCrunch will automatically perform background builds while in manual mode, and will continue to prioritise test execution based on your changes, so kicking off the tests manually will still likely give you feedback much faster than a normal test runner.

You can shut the engine off at any time by clicking Disable in the top menu.

If the engine gets into any trouble (and you think the results may not be an accurate reflection of your source code), you can also use the Reset button on the toolbar of the Tests Window - this is effectively the same as disabling and re-enabling the engine, clearing all state and recycling.

Once you're comfortable with how the engine works, know that you've only just scratched the surface of what is possible with NCrunch. You should have a look at optimising its performance and using it for integration testing. The risk/progress bar is also a nice feature that is worth becoming familiar with.

NCrunch does also have a runtime framework, with attributes that can be used to control how your tests are executed. If you are introducing NCrunch to a complex solution, these attributes are worth becoming familiar with.

Happy crunching!