Documentation

Documentation
V4 Documentation

Unexpected error encountered during test execution

This issue is typically reported by NCrunch with the following error message:

NCrunch was unable to retrieve a meaningful result from this test due to an unexpected error

The above error is a catch-all message that NCrunch provides when the test process suddenly disappears without warning or error information. This can be quite a frustrating issue to solve because it can be caused by many things, some of which actually have little to do with the test suite or NCrunch in general.

Under .NET Core/5+, you may be able to find more error information by setting the log verbosity setting to Detailed and the log to output window setting to True. If the problem appears with these settings enabled, NCrunch will often try to pipe the error information out to its VS output window.

Possible reasons for this error are below.

Targeted version of .NET is not installed

Under .NET Core and .NET 5+, targeting a version of the platform that is not installed on the system can cause NCrunch to consistently give the unexpected error whenever any attempt is made to execute any test within the test project.

Note that the Visual Studio test runner will automatically switch to a different version of the framework if the targeted one is not installed. This can give everything the appearance of working when actually installed components are missing from the machine.

Check to make sure that the precise version of .NET being targeted by your test project is installed on your machine. Check the contents of the 'C:\Program Files\dotnet\sdk' directory to ensure the version exists there.

Note that when targeting the x86 platform with .NET Core/5+, you will also need to have the x86 variant of .NET Core/5+ installed on your machine (usually under 'C:\Program Files (x86)\dotnet\sdk'). The x86 version of .NET Core/5+ is not automatically installed with the x64 version and must be installed separately.

A critical .NET package is not installed

Depending on the structure of the test project and its dependencies, there may be a requirement for a specific version of a runtime package to be installed on the system (i.e. Microsoft.AspNetCore.All, Microsoft.NETCore.App, etc).

Normally these should be restored automatically by VS at load or build time, but sometimes this doesn't seem to work as expected. If the package is required in the test environment but cannot be resolved at runtime, this can cause the test process to be torn down and the unexpected error is given.

Ensure you have the correct versions of critical packages installed on your system. Normally, these are stored with the dotnet SDK under 'C:\Program Files\dotnet\shared' and 'C:\Program Files (x86)\dotnet\shared'.

OutOfMemoryException

If the test process runs out of memory while executing tests, the runtime will throw OutOfMemoryException and subsequently tear down the test process.

When OutOfMemoryException is thrown, the state of the process is too constrained for NCrunch to return any error information. Thus an unexpected error is given as the process suddenly disappears.

When excess memory consumption is responsible, usually it is still possible to execute at least some of the tests in the suite, or at least begin a debug session over the tests.

Because of its features, NCrunch has higher memory requirements in the test process than most other runners. This can cause OutOfMemoryExceptions to appear under NCrunch where they may be less prevalent under other runners.

Possible solutions are as follows:

  • Execute the tests involved with a debugger attached. If the exception is thrown, you can determine the state of the process. If the test or code is misbehaving, fixing it is a solution
  • Make sure your tests are being executed in an x64 process with the use CPU architecture configuration setting
  • Where OutOfMemoryException is being thrown due to too many tests being run within the same process, consider using isolated attribute to reduce the number of tests that may be run in a single process under NCrunch. Splitting the tests into different projects can also be a solution to this problem.

Process corruption or unstable runtime

A .NET process can become unstable due to heap corruption, GC malfunction, EngineExecutionExceptions, bugs in unmanaged code or APIs, etc. This can result in sudden termination of the process without error information.

Generally, when this is the cause of the unexpected error, it will still be possible to run the tests with a debugger to learn more about what is happening in the process.

If you are running a pre-release version of the .NET platform, consider targeting an RTM version to see if this solves the problem.

External interference (virus scanners)

Some virus scanners can mistake executing test code for malware and can perform enforcement action.

Generally, this should be reported by the scanner, but it's worth checking the logs to be sure.

Given that the quality level of many anti-malware products is not always where we would like them to be, it is possible for them to interfere with a test run without giving clear indication that they do so. If you suspect a virus scanner is interfering, try temporarily deactivating it to see if the test run yields a different result.

System stability issues

There are a near limitless number of system-wide issues that can cause .NET processes to be suddenly terminated without warning. It is impossible to produce an accurate list of all these issues, but a few examples are:

  • Driver issues, especially for devices that might be utilised by the code under test
  • Out of disk space
  • Out of a critical system resource (i.e. kernel-level handles)
  • Corrupt install state of NCrunch, VS, .NET, or Windows itself
  • Hardware is overheating
  • Hardware is damaged in a way that causes intermittent malfunction/failure

NCrunch compatibility issue

Because of the ever increasing complexity of the .NET platform, it's possible that your test project is exploring a use case that hasn't been considered in the development in NCrunch.

Historically, the most common NCrunch bugs that can cause the unexpected error are related to the resolution of assemblies or Nuget packages at runtime.

If you can produce this issue with a sample project that you can share with us, please submit it to us through the NCrunch support contact form. Before submitting any code, please make sure you have ruled out any of the above causes. If the problem is caused by something in your environment, we won't be able to reproduce it even if your code sample shows the issue for you every time.

Trial NCrunch
Take NCrunch for a spin
Do your fingers a favour and supercharge your testing workflow
Free Download