A side project I'm working on at the moment has some rather intense demands for maintaining responsiveness of a UI while performing heavy background processing. One of the biggest problems with loading your CPU up past the max with concurrent execution is that the heavy load affects every process running on the system. Windows has an extensive algorithm designed to avoid starving any threads of processing time, and this means it will force context switches to ensure all threads get a fair go.

Changing the priority of processes and threads has the limited impact of making the kernel favour them slightly more, but this will only go so far. If you have a quad-core processor trying to run several dozen intensive lines of execution concurrently, it's a safe bet that even the highest priority thread will see its fair share of context switching.

This kind of load not only interrupts a user's experience of the software they are running, but it will also cause the system to run much less efficiently. Context switches between threads are relatively expensive operations. More concurrency means more context switches. More context switches means less CPU is available for running user code. Less CPU for user code means the user code takes longer to run, so the load doesn't drop as quickly and you end up with more context switches. You get the idea.

Luckily, the guys who built the CLR took this on board when they designed the .NET thread pool. The thread pool is designed to try and limit concurrency to the maximum possible number of workers without overloading the system - a number equal to the number of available processing cores/CPUs. When the number of workers is higher than the processor count, the thread pool will wait fully half a second before pushing out more workers to pick up the extra load. Provided you're not trying to queue up a huge number of massive tasks that take many minutes to complete, this generally works fairly well.

But what about those situations where you actually ARE queuing up ridiculous amounts of work through code that you can't directly control? How is it possible to maintain a responsive UI for the user?

For my situation, the solution seemed to be setting the processor affinity of the threads under execution. For anyone who isn't familiar with processor affinity, it's basically a Win32 property that can be used to hard-lock a thread so that it will only run on a specific core (or set of cores). You can easily see a working example of this working at a process level (Process Processor Affinity) by going to your windows task manager, right clicking on a process, and selecting 'Set Affinity'.

I figured that by setting the processor affinity of all my worker threads to run on all processors but one, I could ensure there would always be a processor available to pick up any UI work that had to be done. To limit context switching, I set the UI thread's affinity so that it would only run on the processor not used by the other threads. So on a quad core machine, we would have:

Core 1: Does UI work

Core 2, 3, 4: Does everything else!

In order to set the processor affinity for an executing thread in .NET, you need to first identify the thread as it is being executed by the kernel. This is an important point:

.NET Thread != Win32 Thread

The CLR uses an abstraction over the physical O/S threads that actually permits multiple .NET Threads to use the same underlying Win32 thread, or to allow a single .NET Thread to execute over multiple Win32 threads. In practice, this rarely happens, but the abstraction still prevents us from setting the thread's processor affinity directly using the standard System.Threading.Thread class. One option is to use the System.Diagnostics.ProcessThread class, but trying to match instances of this class with the currently executing thread is quite an expensive operation.

So, the best option seemed to be to use good 'ol Win32:

        public static void SetProcessorAffinity(int coreMask)
        {
            int threadId = GetCurrentThreadId();
            SafeThreadHandle handle = null;
            var tempHandle = new object();
            try
            {
                handle = OpenThread(0x60, false, threadId);
                if (SetThreadAffinityMask(handle, new HandleRef(tempHandle, (IntPtr) coreMask)) == IntPtr.Zero)
                {
                    throw new Exception("Failed to set processor affinity for thread");
                }
            }
            finally
            {
                if (handle != null)
                {
                    handle.Close();
                }
            }
        }

        [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        public static extern IntPtr SetThreadAffinityMask(SafeThreadHandle handle, HandleRef mask);

        [SuppressUnmanagedCodeSecurity]
        public class SafeThreadHandle : SafeHandleZeroOrMinusOneIsInvalid
        {
            public SafeThreadHandle(): base(true)
            {
                
            }

            protected override bool ReleaseHandle()
            {
                return CloseHandle(handle);
            }
        }

        [ReliabilityContract(Consistency.WillNotCorruptState, Cer.Success), DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true, ExactSpelling = true)]
        public static extern bool CloseHandle(IntPtr handle);

        [DllImport("kernel32")]
        public static extern int GetCurrentThreadId();

        [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        public static extern SafeThreadHandle OpenThread(int access, bool inherit, int threadId);

Calling SetProcessorAffinity with the correct value allows us to lock the underlying Win32 thread to use only the processors we want it to. It's worth noting that the 'coreMask' parameter to this call is actually a bit mask, NOT an ordinal. This is easy to overlook and can result in some very strange behaviour! Each bit in the mask corresponds to one processor on the machine. So if we wanted to assign a thread's processor affinity to all cores on a machine except for the first one, we just need to do some fancy bit shifting:

        for (int i = 1; i < Environment.ProcessorCount; i++)
        {
            coreMask = coreMask ^ (1 << i);
        }

        SetProcessorAffinity(coreMask)

I hope someone finds this useful. Remember always that trying to set a processor affinity to cores that don't exist will result in a complete failure to make the API call - so watch out for those rusty single core machines!