I'm very excited to announce the release of the much awaited NCrunch V3.  In terms of feature-set, this is the biggest release of NCrunch in 3 years.

Since the release of V2, most improvements to NCrunch have been added incrementally over 27 minor versions.  The combined scope of these changes has probably been wide enough to cover multiple major versions, but the nature of these features made them worth releasing in smaller chunks without the fanfare of a big-bang release.  V3 is a rare departure from this release pattern, mostly because the features targeted for this release included significant structural changes to the software or carried a degree of risk that necessitated more testing.

V3 contains three new major feature areas: VS2017 support, A new configuration system, and new test impact detection.

Visual Studio 2017 RC Support

V3 introduces an installer targeted to the new VS2017 RC.

NCrunch now also has support for the new version of MSTest, including its assembly-level test categories and row tests.

VS2017 saw the introduction of the first publicly available version of Microsoft's tooling for .NET Core, so integration with .NET Core is now possible and is a work in progress.  At the moment I hope to have something available for this in January.

Of course, VS2017 hasn't actually been released yet, so things may still change before it hits RTM.  Any breaking changes made in VS2017 will be addressed in future builds of V3.

New Configuration System

Since the first days of NCrunch, it's been acknowledged that NCrunch's configuration system has been inadequate.  I've been constantly surprised at how many different ways people use the product, and this made it impossible to choose a configuration structure that would work for everyone.  Some people like to share their ignored tests, while others use them as part of their individual workflow.  Some people want granular control over settings at project level, while others just want big broad solution-wide settings.  The list goes on.

V3 adds tremendous flexibility to the configuration system.  Because flexibility also adds complexity, most of the new settings features are hidden behind a 'simple mode' option that is enabled by default.  This is mostly because I'd expect new users of the software to quickly be overwhelmed by the new system and most of its options aren't very relevant when you're using NCrunch for the first time.  Turning off the 'simple mode' option in the Configuration Window will open up the power of the new system.  If you don't like the new system or you don't need it, you can just keep 'simple mode' on forever and NCrunch will behave in the old familiar way.

Settings As Data

Internally in the engine, NCrunch settings are now stored as data rather than static elements in the software.  This makes it possible for settings to be added by modules (an important consideration for the upcoming NCrunch SDK/API), while also giving much more consistent handling of settings regardless of their level in the structure (i.e. global, solution, project, etc).  All settings are now stored in XML readable form.  Binary storage of settings or user preferences is now a thing of the past.  The settings files themselves are designed for easy VCS merging of changes.  The engine is also designed to preserve declared settings it doesn't understand, so the new format should be both forwards and backwards compatible between future versions of NCrunch.

Separation Of User/Shared Settings

One of the biggest problems with the old configuration system was that it forced everyone into storing settings in either user or shared files as determined by the settings themselves.  V3 does away with this.  The new Configuration Window has separate nodes for declared user ("my") and shared settings.  The new 'Share tests ignored by NCrunch' and 'Share metrics excluded by NCrunch' settings control where NCrunch should share the data associated with the ignored tests and metrics, which is also stored in the settings files.

Inheritance

NCrunch now has an inheritance structure that determines which value is applicable for a setting at a given point.  This makes it possible for you to define settings at a higher level than they normally sit (i.e. project build settings can be declared at solution or global level).  This also has consideration for user/shared settings, so you could have a setting that defaults to one value for your team, while overriding this just for yourself.

For more information on how the inheritance works, take a look at the new documentation for this feature area.

Configurable Storage Path For Project Settings Files

Up until now, NCrunch has always stored project settings files adjacent to their project files.  V3 introduces a new setting allowing you to choose a single location for NCrunch to store these files - Project config file storage path.

Engine Mode Upgrades

The new configuration system introduces some big changes to NCrunch's engine modes.  Previously, engine modes have been fixed contexts that could only control the behaviour of the engine in a few select areas.  Now, they have the capability to override almost all global and solution-level settings within NCrunch.  You can now create engine modes that override UI and performance settings, or completely change the workflow of the engine.

Here are a few ideas for engine modes that can now be created under the new system:

• A mode that ramps up the number of processing threads, increases the number of pooled processes and turns on parallel execution to push the engine to its limit.  This could be used to run through lots of tests quickly (AFK processing!).
• A mode that uses the Custom environment variables setting to set an environment variable in every build and test process.  You can then have custom code in your tests that considers this variable, implementing different testing behaviour dependent on the current engine mode.
• You can change the default engine modes so that they don't just change the behaviour of the engine, but also adjust the NCrunch UI to make the mode more obvious.  For example, I'm finding it quite useful to change the coverage marker shape when I'm running tests manually so that I don't forget to switch back to an automatic mode later.

Of course, the existing functionality of NCrunch V2 can still be applied by engine modes.  You can still use them to target specific tests for automatic execution or metrics analysis.  Even more than before, it's now possible to get a very different experience from NCrunch with just the flick of a switch.

Console Tool Upgrades

The ability to inject settings via the NCrunch console tool's command-line has also been integrated with the new configuration system.  The number of settings that can be injected into this tool has been greatly increased; it's now possible to inject project settings and non-scalar values.  This should make it easier to manage the tool from your CI system with less dependency on pre-created configuration files.

Configuration Wizard Improvements

The 'First Time Crunch' configuration wizard has been improved to include a new option on the first page of the wizard.

The option to use the previously chosen settings becomes available after the first time you've run the wizard.  Basically, it remembers what you chose last time and lets you use it again.  Quite simple, but much less frustrating for people that create new solutions often.

Improved Test Impact Detection

NCrunch V3 introduces significant improvements to its detection of impacted tests.

Prior to V3, NCrunch's impact detection has been very rudimentary.  Impacted tests were detected only at text-level, where any test with coverage markers on a changed source file was considered to be impacted.

By default, impact detection is now performed through an extensive IL-level comparison between changed assemblies.  This means that you won't see tests considered as impacted due to comment changes anymore.  The analysis is now also much better targeted, so you shouldn't see tests become impacted if they don't touch the methods you've changed.  The analysis takes into consideration a wide range of edge cases to try and give a highly accurate picture of which tests are likely to be affected by changes.  The goal of the new system is to make it possible to make heavier use of NCrunch's 'Run impacted tests automatically, others manually' engine mode, thereby reducing the resource consumption of NCrunch as a whole.

Impact detection data also carries across sessions, so NCrunch can figure out which tests were impacted by changes made to the solution while Visual Studio was closed.  This makes it possible to use the impact detection to target tests when using the NCrunch console tool.  So the 'Run impacted tests automatically, others manually' can now be used in your CI system, though this should be used with caution as no impact analysis is 100% accurate.

NCrunch will now also track which files are accessed by tests under execution.  Files that are found to correspond with files transferred to the NCrunch workspace will be considered as 'covered' by tests.  When these files are changed, the tests will be accordingly marked as impacted.  This new behaviour is controlled using the Track file dependencies configuration setting.

NCrunch's impact detection behaviour is controlled by the newly introduced Impact detection mode setting.  The old behaviour is still available as this may still have some advantages over the new mode in terms of a marginal engine performance improvement.

Cross-Process And Cross-AppDomain Code Coverage Tracking

NCrunch now has the capability to track code coverage data across application domains and multiple processes.  This functionality was intended to be released as part of V3, but was included in v2.24 by mistake.  So probably you've been enjoying it for a while already.

For NCrunch to track code coverage data in a process other than its own, the process must be spawned your code under test.  It's perfectly OK for the process to be spawned in a fixture setup method or a different test to the one the coverage needs to be collected for, but it can't be created by a mechanism that exists outside of NCrunch's memory space.

The cross-process coverage tracking is very powerful when used in combination with the new impact analysis system.

Other Stuff

V3 also contains another round of fixes and compatibility improvements.  Refer to the release notes for more information.

If you managed to reach the bottom of this page without having clicked the download button yet, you can find it here!