From time to time, we've all had to unit test some multithreaded code. Many of us take shortcuts in understanding the problem instead of taking a deep dive into stronger solutions. We tend to litter our tests with calls to Thread.sleep() and then hope we've waited long enough for asynchronous processes to run before validating the results. But is there a better way? Heck yes! And it's not that complicated or time-consuming—it just requires us to think about multithreaded testing in a different way.

So let's get started.

All These Threads, and Not a Thing to Wear

Threads are awesome. They help us get more done concurrently and let us take advantage of great programming patterns like pub/sub. Unfortunately, they're not always simple to test.

To start, let's consider a common problem that appears when testing multithreaded code.

The Test Completes Before the Thread

The first issue that many of us encounter when testing multithreaded code involves threads that don't complete until after the test completes. To put it another way, the test completes long before the process you're testing completes.

The Example

Let's take a look at this simple example to illustrate the issue. As you can see below, our test does nothing more than execute a thread. A more realistic version would include calling the production code that creates the thread and kicks off some async process. Additionally, our test would want to verify an object's state or validate that a mock was called as part of the assertion. But for now, we'll keep it simple.

        [TestMethod()]
        public void CreateThread()
        {
            Thread thread = new Thread(() => ProcessIt());
            thread.Start();
            Debug.WriteLine("Test is over");
        }

        public void ProcessIt()
        {
            for (int i = 0; i < 5; i++)
            {
                Debug.WriteLine("long async processing for i=" + i);
                Thread.Sleep(100);
            }
        }

If we run the test above, we'll see that the test is over before we've even started executing much code.

Additionally, if your test completes before your thread, your NCrunch code coverage will look rather strange. You see, NCrunch records the code coverage on test boundaries while the test runs. Therefore, if a thread continues to run after the test itself is over, no further code coverage will be recorded.

Furthermore, code coverage from Test 1 could leak into the results for Test 2 if that background thread runs too long.

Sleep Toward a Naive Solution

At this point, when most of us just start learning about threads, we choose the naive solution of adding a Thread.sleep() into our test execution.

        [TestMethod()]
        public void CreateThread()
        {
            Thread thread = new Thread(() => ProcessIt());
            thread.Start();
            Thread.Sleep(600);
            Debug.WriteLine("Test is over");
        }

And for the simplest cases, this seems to work OK.

However, there are a couple of problems with this approach.

First, it only works if we know how long our threads will process—and typically, we have no idea. It can vary based on data, the server that the test runs on, or other processes that run at the same time. This sends us on a one-way trip to Flaky Testville.

Second, our test suite will take more and more time to run. Every time we add a scenario or include tests like this, we're slowing everything down. And soon you'll hate running your unit tests regularly or watching tests spin whenever you make a change.

So what are our options? Here are a number of things that we can do instead.

Thread.Join() to Weave Something Better

A slightly less naive way of testing our multithreaded code adds a call to Thread.Join(). This command will block the calling thread (that is, the thread executing the unit test) until the thread that is joined terminates.

        [TestMethod()]
        public void CreateThread()
        {
            Thread thread = new Thread(() => ProcessIt());
            thread.Start();
            thread.Join();
            Debug.WriteLine("Test is over");
        }

With this approach, we don't have to guess how long our threads will run—our tests will take as long as the underlying threads do.

However, we may end up with awkward tests or Thread.Join() calls in our production code that don't belong there. And we might end up accidentally losing out on the async benefits that threads were going to bring us in the first place.

So let's look at some other solutions.

Async-ing Ship

Now, some of you may be thinking that we're missing a solution here. Why can't we just use async and await? Well, we can.

[TestMethod]
public async Task AwaitForThreadToFinish()
{
  await MyClass.SomeAsyncProcess();
}

I prefer this over Thread.Join() because it offers more flexibility on what to wait for and when. However, this might not work well if we're spinning up multiple threads deeper within the production code. This only saves us if the method we're executing has a single thread.

For some solutions, this might be great. For others, it still doesn't help as much as we'd like.

So, what else could we do?

It's Not Just the Test

Now it's time to start considering our production code. Sometimes, classes are hard to test because they're poorly designed. What do I mean by that?

Well, to make testing simple and to make your code more modular, consider putting your asynchronous code into a test harness that executes everything synchronously.

For this solution, we're going to look at the Humble Object pattern for guidance. Here, we're going to separate out our core logic from anything that relies on asynchronicity. Then you don't have to worry about modifying your test to properly wait for threads. You just write test code to verify the business logic.

The next step involves wrapping the logic in another class that receives the async calls and delegates processing to the business logic.

        public interface MyInterface
        {
            void ProcessIt();
        }

        public class MyClass : MyInterface
        {
            public void ProcessIt()
            {
                //do business logic
            }
        }
        public class MyWrapper
        {
            private MyInterface _myInterface;

            public MyWrapper(MyInterface myInterface)
            {
                _myInterface = myInterface;
            }

            public void ProcessItAsync()
            {
                Thread thread = new Thread(() => _myInterface.ProcessIt());
            }
        }

Ultimately, you'll have two separate tests for this. First, you'll have a test that just validates the business logic of your MyClass.ProcessIt() method. Second, you'll have a test of your wrapper that simply verifies that you're calling the ProcessIt() method on the interface. So there you can just use a simple mock. You'll still need to await the response even if it's not doing anything, but it will all be much faster and better organized overall.

Verify Threads That Play Nicely Together

The other major problem in testing multithreaded code involves testing concurrent logic. Unfortunately, simple solutions don't exist for this problem.

You see, in order to test aspects like correct locking behavior, your test would have to create multiple threads that might execute against your thread-safe objects. However, you can't control when the threads will execute or what order they will execute in. So, although you may prove that your threads can act in a thread-safe manner, you can't verify it with 100% certainty.

So what do you do? Have a couple integration or component tests that spawn threads to give yourself some level of confidence. But don't assume they're foolproof.

Instead, assume they're not foolproof, and take time to digest how all the threads and processes work together. Consider different ways things could break. Then make your objects immutable. It will keep your threads safe.

Additionally, survey your code and think of things that may lead to deadlocks, and then try to program defensively so that you don't run into them. Or, find ways for the program to get out of a bad state.

Overall, concurrency can be difficult. But if you keep basic principles in mind, you'll be able to do it.

Conclusion

Many methods exist for testing multithreaded code, and we've covered a few of them here. They all have different pros and cons. So, take your knowledge of your system and find what solution makes the most sense. And don't fret if you make the wrong decision the first time through. Many of us do. Just continue to experiment with ways to make your tests and your threads simple to maintain and simple to test.

This post was written by Sylvia Fronczak. Sylvia is a software developer that has worked in various industries with various software methodologies. She’s currently focused on design practices that the whole team can own, understand, and evolve over time.