The MSDN Multiple Windows, Multiple Threads Sample* showed how to launch a new thread with a WPF Window, and will work in most cases.  The sample code (commented and slightly modified) works out to the following:

// Create a thread
Thread newWindowThread = new Thread(new ThreadStart( () =>
{
    // Create and show the Window
    Window1 tempWindow = new Window1();
    tempWindow.Show();
    // Start the Dispatcher Processing
    System.Windows.Threading.Dispatcher.Run();
}));
// Set the apartment state
newWindowThread.SetApartmentState(ApartmentState.STA);
// Make the thread a background thread
newWindowThread.IsBackground = true;
// Start the thread
newWindowThread.Start();

This sample creates a thread, marks it as single threaded apartment state, and starts the Dispatcher on that thread. That is the minimum requirements to get a Window displaying and handling messages correctly, but, unfortunately, has some serious flaws.

*Note: The referenced sample is no longer available as of 2013. A new Multithreaded Web Browser Sample shows a similar technique and code

The first issue – the created thread will run continuously until the application shuts down, given the code in the sample.  The problem is that the ThreadStart delegate used ends with running the Dispatcher.  However, nothing ever stops the Dispatcher processing.  The thread was created as a Background thread, which prevents it from keeping the application alive, but the Dispatcher will continue to pump dispatcher frames until the application shuts down.

In order to fix this, we need to call Dispatcher.InvokeShutdown after the Window is closed.  This would require modifying the above sample to subscribe to the Window’s Closed event, and, at that point, shutdown the Dispatcher:

// Create a thread
Thread newWindowThread = new Thread(new ThreadStart( () =>
{
    Window1 tempWindow = new Window1();
    // When the window closes, shut down the dispatcher
    tempWindow.Closed += (s,e) => 
       Dispatcher.CurrentDispatcher.BeginInvokeShutdown(DispatcherPriority.Background);

    tempWindow.Show();
    // Start the Dispatcher Processing
    System.Windows.Threading.Dispatcher.Run();
}));
// Setup and start thread as before

This eliminates the first issue.  Now, when the Window is closed, the new thread’s Dispatcher will shut itself down, which in turn will cause the thread to complete.

The above code will work correctly for most situations.  However, there is still a potential problem which could arise depending on the content of the Window1 class.  This is particularly nasty, as the code could easily work for most windows, but fail on others.

The problem is, at the point where the Window is constructed, there is no active SynchronizationContext.  This is unlikely to be a problem in most cases, but is an absolute requirement if there is code within the constructor of Window1 which relies on a context being in place.

While this sounds like an edge case, it’s fairly common.  For example, if a BackgroundWorker is started within the constructor, or a TaskScheduler is built using TaskScheduler.FromCurrentSynchronizationContext() with the expectation of synchronizing work to the UI thread, an exception will be raised at some point.  Both of these classes rely on the existence of a proper context being installed to SynchronizationContext.Current, which happens automatically, but not until Dispatcher.Run is called.  In the above case, SynchronizationContext.Current will return null during the Window’s construction, which can cause exceptions to occur or unexpected behavior.

Luckily, this is fairly easy to correct.  We need to do three things, in order, prior to creating our Window:

• Create and initialize the Dispatcher for the new thread manually
• Create a synchronization context for the thread which uses the Dispatcher
• Install the synchronization context

Creating the Dispatcher is quite simple – The Dispatcher.CurrentDispatcher property gets the current thread’s Dispatcher and “creates a new Dispatcher if one is not already associated with the thread.”  Once we have the correct Dispatcher, we can create a SynchronizationContext which uses the dispatcher by creating a DispatcherSynchronizationContext.  Finally, this synchronization context can be installed as the current thread’s context via SynchronizationContext.SetSynchronizationContext.  These three steps can easily be added to the above via a single line of code:

// Create a thread
Thread newWindowThread = new Thread(new ThreadStart( () =>
{
    // Create our context, and install it:
    SynchronizationContext.SetSynchronizationContext(
        new DispatcherSynchronizationContext(
            Dispatcher.CurrentDispatcher));

    Window1 tempWindow = new Window1();
    // When the window closes, shut down the dispatcher
    tempWindow.Closed += (s,e) => 
       Dispatcher.CurrentDispatcher.BeginInvokeShutdown(DispatcherPriority.Background);

    tempWindow.Show();
    // Start the Dispatcher Processing
    System.Windows.Threading.Dispatcher.Run();
}));
// Setup and start thread as before

This now forces the synchronization context to be in place before the Window is created and correctly shuts down the Dispatcher when the window closes.

However, there are quite a few steps.  In my next post, I’ll show how to make this operation more reusable by creating a class with a far simpler API…

• GiveCamp provides real value to organizations that truly need help. 

The Seattle event alone succeeded in helping sixteen non-profit organizations in many different ways.  The projects involved varied dramatically, including website redesigns, SEO, reworking data management workflows, and even game development. 

Many non-profits have a strong need for good, quality technical help.  However, nearly every non-profit organization has an incredibly limited budget.  GiveCamp is a way to really give back, and provide incredibly valuable help to organizations that truly benefit.

My experience has shown many developers to be incredibly generous – this is a chance to dedicate your energy to helping others in a way that really takes advantage of your expertise.  Your time as a developer is incredibly valuable, and this puts something of incredible value directly into the hands of places its needed.

First, and foremost, GiveCamp is about providing technical help to non-profit organizations in need.

• GiveCamp can make you a better developer. 

This is a fantastic opportunity for us, as developers, to work with new people, in a new setting.  The incredibly short time frame (one weekend for a deliverable project) and intense motivation to succeed provides a huge opportunity for learning from peers. 

I’d personally like to thank off the developers with whom I worked – I learned something from each and every one of you.  I hope to see and work with all of you again someday.

• GiveCamp provides an opportunity for you to work outside of your comfort zone.

While it’s always nice to be an expert, it’s also valuable to work on a project where you have little or no direct experience.  My team focused on a complete reworking of our organizations message and a complete new website redesign and deployment using WordPress. 

While I’d used WordPress for my blog, and had some experience, this is completely unrelated to my professional work.  In fact, nobody on our team normally worked directly with the technologies involved – yet together we managed to succeed in delivering our goals. 

As developers, it’s easy to want to stay abreast of new technology surrounding our expertise, but its rare that we get a chance to sit down and work on something practical that is completely outside of our normal realm of work.  I’m a desktop developer by trade, and spent much of the weekend working with CSS and Photoshop.  Many of the projects organizations need don’t match perfectly with the skill set in the room – yet all of the software professionals rose to the occasion and delivered practical, usable applications.

• GiveCamp is a short term, known commitment.

While this seems obvious, I think it’s an important aspect to remember.  This is a huge part of what makes it successful – you can work, completely focused, on a project, then walk away completely when you’re done.  There is no expectation of continued involvement.  While many of the professionals I’ve talked to are willing to contribute some amount of their time beyond the camp, this is not expected.

The freedom this provides is immense.  In addition, the motivation this brings is incredibly valuable.  Every developer in the room was very focused on delivering in time – you have one shot to get it as good as possible, and leave it with the organization in a way that can be maintained by them.  This is a rare experience – and excellent practice at time management for everyone involved.

• GiveCamp provides a great way to meet and network with your peers.

Not only do you get to network with other software professionals in your area – you get to network with amazing people.  Every single person in the room is there to try to help people.  The balance of altruism, intelligence, and expertise in the room is something I’ve never before experienced.

During the presentations of what was accomplished, I felt blessed to participate.  I know many people in the room were incredibly touched by the level of dedication and accomplishment over the weekend.

• GiveCamp is fun.

At the end of the experience, I would have signed up again, even if it was a painful, tedious weekend – merely due to the amazing accomplishments achieved throughout the event.  However, the event is fun. 

Everybody I talked to, the entire weekend, was having a good time.  While there were many faces focused into a near grimace at times (including mine, I’ll admit), this was always in response to a particularly challenging problem or task.  The challenges just added to the overall enjoyment of the weekend – part of why I became a developer in the first place is my love for challenge and puzzles, and a short deadline using unfamiliar technology provided plenty of opportunity for puzzles.  As soon as people would stand up, it was another smile.

If you’re a developer, I’d recommend looking at GiveCamp more closely.  Watch for an event in your area.  If there isn’t one, consider building a team and organizing an event.  The experience is worth the commitment. 

Typically, loading a pre-.NET 4 mixed mode assembly is handled simply by changing your app.Config file, and including the relevant attribute in the startup element:

<?xml version="1.0" encoding="utf-8" ?>
<configuration>
  <startup useLegacyV2RuntimeActivationPolicy="true">
    <supportedRuntime version="v4.0"/>
  </startup>
</configuration>

This causes your application to run correctly, and load the older, mixed-mode assembly without issues. For full details on what’s happening here and why, I recommend reading Mark Miller’s detailed explanation of this attribute and the reasoning behind it.

Before I show any code, let me say:

• I strongly recommend using the official approach of using app.config to set this policy.

That being said, there are (rare) times when, for one reason or another, changing the application configuration file is less than ideal.

While this is the supported approach to handling this issue, the CLR Hosting API includes a means of setting this programmatically via the ICLRRuntimeInfo interface.  Normally, this is used if you’re hosting the CLR in a native application in order to set this, at runtime, prior to loading the assemblies.  However, the F# Samples include a nice trick showing how to load this API and bind this policy, at runtime.  This was required in order to host the Managed DirectX API, which is built against an older version of the CLR.

This is fairly easy to port to C#.  Instead of a direct port, I also added a little addition – by trapping the COM exception received if unable to bind (which will occur if the 2.0 CLR is already bound), I also allow a runtime check of whether this property was setup properly:

public static class RuntimePolicyHelper
{
    public static bool LegacyV2RuntimeEnabledSuccessfully { get; private set; }

    static RuntimePolicyHelper()
    {
        ICLRRuntimeInfo clrRuntimeInfo =
            (ICLRRuntimeInfo)RuntimeEnvironment.GetRuntimeInterfaceAsObject(
                Guid.Empty, 
                typeof(ICLRRuntimeInfo).GUID);
        try
        {
            clrRuntimeInfo.BindAsLegacyV2Runtime();
            LegacyV2RuntimeEnabledSuccessfully = true;
        }
        catch (COMException)
        {
            // This occurs with an HRESULT meaning 
            // "A different runtime was already bound to the legacy CLR version 2 activation policy."
            LegacyV2RuntimeEnabledSuccessfully = false;
        }
    }

    [ComImport]
    [InterfaceType(ComInterfaceType.InterfaceIsIUnknown)]
    [Guid("BD39D1D2-BA2F-486A-89B0-B4B0CB466891")]
    private interface ICLRRuntimeInfo
    {
        void xGetVersionString();
        void xGetRuntimeDirectory();
        void xIsLoaded();
        void xIsLoadable();
        void xLoadErrorString();
        void xLoadLibrary();
        void xGetProcAddress();
        void xGetInterface();
        void xSetDefaultStartupFlags();
        void xGetDefaultStartupFlags();

        [MethodImpl(MethodImplOptions.InternalCall, MethodCodeType = MethodCodeType.Runtime)]
        void BindAsLegacyV2Runtime();
    }
}

Using this, it’s possible to not only set this at runtime, but also verify, prior to loading your mixed mode assembly, whether this will succeed.

In my case, this was quite useful – I am working on a library purely for internal use which uses a numerical package that is supplied with both a completely managed as well as a native solver.  The native solver uses a CLR 2 mixed-mode assembly, but is dramatically faster than the pure managed approach.  By checking RuntimePolicyHelper.LegacyV2RuntimeEnabledSuccessfully at runtime, I can decide whether to enable the native solver, and only do so if I successfully bound this policy.

There are some tricks required here – To enable this sort of fallback behavior, you must make these checks in a type that doesn’t cause the mixed mode assembly to be loaded.  In my case, this forced me to encapsulate the library I was using entirely in a separate class, perform the check, then pass through the required calls to that class.  Otherwise, the library will load before the hosting process gets enabled, which in turn will fail.

This code will also, of course, try to enable the runtime policy before the first time you use this class – which typically means just before the first time you check the boolean value.  As a result, checking this early on in the application is more likely to allow it to work.

Finally, if you’re using a library, this has to be called prior to the 2.0 CLR loading.  This will cause it to fail if you try to use it to enable this policy in a plugin for most third party applications that don’t have their app.config setup properly, as they will likely have already loaded the 2.0 runtime.

As an example, take a simple audio player.  The code below shows how this can be used to properly, at runtime, only use the “native” API if this will succeed, and fallback (or raise a nicer exception) if this will fail:

public class AudioPlayer
{
    private IAudioEngine audioEngine;

    public AudioPlayer()
    {
        if (RuntimePolicyHelper.LegacyV2RuntimeEnabledSuccessfully)
        {
            // This will load a CLR 2 mixed mode assembly
            this.audioEngine = new AudioEngineNative();
        }
        else
        {
            this.audioEngine = new AudioEngineManaged();
        }
    }

    public void Play(string filename)
    {
        this.audioEngine.Play(filename);
    }
}

Now – the warning:

This approach works, but I would be very hesitant to use it in public facing production code, especially for anything other than initializing your own application.  While this should work in a library, using it has a very nasty side effect: you change the runtime policy of the executing application in a way that is very hidden and non-obvious.

I mention that this is only a portion of the entire quote, and, as such, I feel that Knuth is being quoted out of context.  Optimization is important.  It is a critical part of every software development effort, and should never be ignored.  A developer who ignores optimization is not a professional.  Every developer should understand optimization – know what to optimize, when to optimize it, and how to think about code in a way that is intelligent and productive from day one.

I want to start by discussing my own, personal motivation here.  I recently wrote about a performance issue I ran across, and was slammed by multiple comments and emails that effectively boiled down to: “You’re an idiot.  Premature optimization is the root of all evil.  This doesn’t matter.”  It didn’t matter that I discovered this while measuring in a profiler, and that it was a portion of my code base that can take “many hours to complete.”  Even so, multiple people instantly jump to “it’s premature – it doesn’t matter.”

This is a common thread I see.  For example, StackOverflow has many pages of posts with answers that boil down to (mis)quoting Knuth.  In fact, just about any question relating to a performance related issue gets this quote thrown at it immediately – whether it deserves it or not. 

That being said, I did receive some positive comments and emails as well.  Many people want to understand how to optimize their code, approaches to take, tools and techniques they can use, and any other advice they can discover.

First, lets get back to Knuth – I mentioned before that Knuth is being quoted out of context.  Lets start by looking at the entire quote from his 1974 paper Structured Programming with go to Statements:

“We should forget about small efficiencies, say about 97% of the time: premature optimization is the root of all evil. Yet we should not pass up our opportunities in that critical 3%. A good programmer will not be lulled into complacency by such reasoning, he will be wise to look carefully at the critical code; but only after that code has been identified.”

Ironically, if you read Knuth’s original paper, this statement was made in the middle of a discussion of how Knuth himself had changed how he approaches optimization.  It was never a statement saying “don’t optimize”, but rather, “optimizing intelligently provides huge advantages.”  His approach had three benefits: “a) it doesn’t take long” … “b) the payoff is real”, c) you can “be less efficient in the other parts of my programs, which therefore are more readable and more easily written and debugged.”

Looking at Knuth’s premise here, and reading that section of his paper, really leads to a few observations:

• Optimization is important  “he will be wise to look carefully at the critical code”
• Normally, 3% of your code – three lines out of every 100 you write, are “critical code” and will require some optimization: “we should not pass up our opportunities in that critical 3%”
• Optimization, if done well, should not be time consuming: “it doesn’t take long”
• Optimization, if done correctly, provides real benefits: “the payoff is real”

None of this is new information.  People who care about optimization have been discussing this for years – for example, Rico Mariani’s Designing For Performance (a fantastic article) discusses many of the same issues very intelligently.

That being said, many developers seem unable or unwilling to consider optimization.  Many others don’t seem to know where to start.  As such, I’m going to spend some time writing about optimization – what is it, how should we think about it, and what can we do to improve our own code.

I recently ran across a nasty scenario that illustrated to me how dangerous following any fixed rules for optimization can be…

The rules in C# when optimizing code are very different than C or C++.  Often, they’re exactly backwards.  For example, in C and C++, lifting a variable out of loops in order to avoid memory allocations often can have huge advantages.  If some function within a call graph is allocating memory dynamically, and that gets called in a loop, it can dramatically slow down a routine.

This can be a tricky bottleneck to track down, even with a profiler.  Looking at the memory allocation graph is usually the key for spotting this routine, as it’s often “hidden” deep in call graph.  For example, while optimizing some of my scientific routines, I ran into a situation where I had a loop similar to:

for (i=0; i<numberToProcess; ++i)
{
   // Do some work
   ProcessElement(element[i]);
}

This loop was at a fairly high level in the call graph, and often could take many hours to complete, depending on the input data.  As such, any performance optimization we could achieve would be greatly appreciated by our users.

After a fair bit of profiling, I noticed that a couple of function calls down the call graph (inside of ProcessElement), there was some code that effectively was doing:

// Allocate some data required
DataStructure* data = new DataStructure(num);
// Call into a subroutine that passed around and manipulated this data highly
CallSubroutine(data);
// Read and use some values from here
double values = data->Foo;
// Cleanup 
delete data;
// ...
return bar;

Normally, if “DataStructure” was a simple data type, I could just allocate it on the stack.  However, it’s constructor, internally, allocated it’s own memory using new, so this wouldn’t eliminate the problem.  In this case, however, I could change the call signatures to allow the pointer to the data structure to be passed into ProcessElement and through the call graph, allowing the inner routine to reuse the same “data” memory instead of allocating.  At the highest level, my code effectively changed to something like:

DataStructure* data = new DataStructure(numberToProcess);
for (i=0; i<numberToProcess; ++i)
{
   // Do some work
   ProcessElement(element[i], data);
}
delete data;

Granted, this dramatically reduced the maintainability of the code, so it wasn’t something I wanted to do unless there was a significant benefit.  In this case, after profiling the new version, I found that it increased the overall performance dramatically – my main test case went from 35 minutes runtime down to 21 minutes.  This was such a significant improvement, I felt it was worth the reduction in maintainability.

In C and C++, it’s generally a good idea (for performance) to:

• Reduce the number of memory allocations as much as possible,
• Use fewer, larger memory allocations instead of many smaller ones, and
• Allocate as high up the call stack as possible, and reuse memory

I’ve seen many people try to make similar optimizations in C# code.  For good or bad, this is typically not a good idea.  The garbage collector in .NET completely changes the rules here.

In C#, reallocating memory in a loop is not always a bad idea.  In this scenario, for example, I may have been much better off leaving the original code alone.  The reason for this is the garbage collector.  The GC in .NET is incredibly effective, and leaving the allocation deep inside the call stack has some huge advantages.  First and foremost, it tends to make the code more maintainable – passing around object references tends to couple the methods together more than necessary, and overall increase the complexity of the code.  This is something that should be avoided unless there is a significant reason.  Second, (unlike C and C++) memory allocation of a single object in C# is normally cheap and fast.  Finally, and most critically, there is a large advantage to having short lived objects.  If you lift a variable out of the loop and reuse the memory, its much more likely that object will get promoted to Gen1 (or worse, Gen2).  This can cause expensive compaction operations to be required, and also lead to (at least temporary) memory fragmentation as well as more costly collections later.

As such, I’ve found that it’s often (though not always) faster to leave memory allocations where you’d naturally place them – deep inside of the call graph, inside of the loops.  This causes the objects to stay very short lived, which in turn increases the efficiency of the garbage collector, and can dramatically improve the overall performance of the routine as a whole.

In C#, I tend to:

• Keep variable declarations in the tightest scope possible
• Declare and allocate objects at usage

While this tends to cause some of the same goals (reducing unnecessary allocations, etc), the goal here is a bit different – it’s about keeping the objects rooted for as little time as possible in order to (attempt) to keep them completely in Gen0, or worst case, Gen1.  It also has the huge advantage of keeping the code very maintainable – objects are used and “released” as soon as possible, which keeps the code very clean.  It does, however, often have the side effect of causing more allocations to occur, but keeping the objects rooted for a much shorter time.

Now – nowhere here am I suggesting that these rules are hard, fast rules that are always true.  That being said, my time spent optimizing over the years encourages me to naturally write code that follows the above guidelines, then profile and adjust as necessary.  In my current project, however, I ran across one of those nasty little pitfalls that’s something to keep in mind – interop changes the rules.

In this case, I was dealing with an API that, internally, used some COM objects.  In this case, these COM objects were leading to native allocations (most likely C++) occurring in a loop deep in my call graph.  Even though I was writing nice, clean managed code, the normal managed code rules for performance no longer apply. 

After profiling to find the bottleneck in my code, I realized that my inner loop, a innocuous looking block of C# code, was effectively causing a set of native memory allocations in every iteration.  This required going back to a “native programming” mindset for optimization.  Lifting these variables and reusing them took a 1:10 routine down to 0:20 – again, a very worthwhile improvement.

Overall, the lessons here are:

• Always profile if you suspect a performance problem – don’t assume any rule is correct, or any code is efficient just because it looks like it should be
• Remember to check memory allocations when profiling, not just CPU cycles
• Interop scenarios often cause managed code to act very differently than “normal” managed code.
• Native code can be hidden very cleverly inside of managed wrappers

• Compatibility with Visual Studio SP1
• APIs for Windows Phone 7
• Compatibility with non-English installations
• Compatibility with Visual Studio Express Edition
• More efficient Async methods due to a change in the API
• Numerous bug fixes
• New EULA which allows distribution in production environments

Anybody using the Async CTP should consider upgrading to the new version immediately.  For details, visit the Visual Studio Asynchronous Programming page on MSDN.

The ConcurrentDictionary<TKey, TValue> class was added in .NET 4, and provides a thread-safe, lock free collection of key value pairs.  While this is a fantastic replacement for Dictionary<TKey, TValue>, it has a potential flaw when used with values where construction of the value class is expensive.

The typical way this is used is to call a method such as GetOrAdd to fetch or add a value to the dictionary.  It handles all of the thread safety for you, but as a result, if two threads call this simultaneously, two instances of TValue can easily be constructed.

If TValue is very expensive to construct, or worse, has side effects if constructed too often, this is less than desirable.  While you can easily work around this with locking, Stephen Toub provided a very clever alternative – using Lazy<TValue> as the value in the dictionary instead.

This looks like the following.  Instead of calling:

MyValue value = dictionary.GetOrAdd(
                             key, 
                             () => new MyValue(key));

We would instead use a ConcurrentDictionary<TKey, Lazy<TValue>>, and write:

MyValue value = dictionary.GetOrAdd(
                             key, 
                             () => new Lazy<MyValue>(
                                 () => new MyValue(key)))
                          .Value;

This simple change dramatically changes how the operation works.  Now, if two threads call this simultaneously, instead of constructing two MyValue instances, we construct two Lazy<MyValue> instances.

However, the Lazy<T> class is very cheap to construct.  Unlike “MyValue”, we can safely afford to construct this twice and “throw away” one of the instances.

We then call Lazy<T>.Value at the end to fetch our “MyValue” instance.  At this point, GetOrAdd will always return the same instance of Lazy<MyValue>.  Since Lazy<T> doesn’t construct the MyValue instance until requested, the actual MyClass instance returned is only constructed once.

Wrapping existing asynchronous code into a Task or Task<T> is, thankfully, fairly straightforward.  There are two main approaches to this.

Code written using the Asynchronous Programming Model (APM) is very easy to convert to using Task<T>.  The TaskFactory class provides the tools to directly convert APM code into a method returning a Task<T>.  This is done via the FromAsync method.  This method takes the BeginOperation and EndOperation methods, as well as any parameters and state objects as arguments, and returns a Task<T> directly.

For example, we could easily convert the WebRequest BeginGetResponse and EndGetResponse methods into a method which returns a Task<WebResponse> via:

Task<WebResponse> task = Task.Factory
                             .FromAsync<WebResponse>(
                                 request.BeginGetResponse,
                                 request.EndGetResponse,
                                 null);

Event-based Asynchronous Pattern (EAP) code can also be wrapped into a Task<T>, though this requires a bit more effort than the one line of code above.  This is handled via the TaskCompletionSource<T> class.  MSDN provides a detailed example of using this to wrap an EAP operation into a method returning Task<T>.  It demonstrates handling cancellation and exception handling as well as the basic operation of the asynchronous method itself.

The basic form of this operation is typically:

Task<YourResult> GetResultAsync()
{
    var tcs = new TaskCompletionSource<YourResult>();
    // Handle the event, and setup the task results...
    this.GetResultCompleted += (o,e) =>
    {
       if (e.Error != null)
            tcs.TrySetException(e.Error);
       else if (e.Cancelled)
            tcs.TrySetCanceled();
       else
            tcs.TrySetResult(e.Result);
    };

    // Call the EAP-based asynchronous method
    this.GetResult();

    // Return the task from the TaskCompletionSource
    return tcs.Task;
}

We can easily use these methods to wrap our own code into a method that returns a Task<T>.  Existing libraries which cannot be edited can be extended via Extension methods.  The CTP uses this technique to add appropriate methods throughout the framework.

The suggested naming for these methods is to define these methods as “Task<YourResult> YourClass.YourOperationAsync(…)”.  However, this naming often conflicts with the default naming of the EAP.  If this is the case, the CTP has standardized on using “Task<YourResult> YourClass.YourOperationTaskAsync(…)”.

Once we’ve wrapped all of our existing code into operations that return Task<T>, we can begin investigating how the Async CTP can be used with our own code.

The first asynchronous pattern used in the .NET framework was the Asynchronous Programming Model (APM).  This pattern was based around callbacks.  A method is used to start the operation.  It typically is named as BeginSomeOperation.  This method is passed a callback defined as an AsyncCallback, and returns an object that implements IAsyncResult.  Later, the IAsyncResult is used in a call to a method named EndSomeOperation, which blocks until completion and returns the value normally directly returned from the synchronous version of the operation.  Often, the EndSomeOperation call would be called from the callback function passed, which allows you to write code that never blocks.

While this pattern works perfectly to prevent blocking, it can make quite confusing code, and be difficult to implement.  For example, the sample code provided for FileStream’s BeginRead/EndRead methods is not simple to understand.  In addition, implementing your own asynchronous methods requires creating an entire class just to implement the IAsyncResult.

Given the complexity of the APM, other options have been introduced in later versions of the framework.  The next major pattern introduced was the Event-based Asynchronous Pattern (EAP).  This provides a simpler pattern for asynchronous operations.  It works by providing a method typically named SomeOperationAsync, which signals its completion via an event typically named SomeOperationCompleted.

The EAP provides a simpler model for asynchronous programming.  It is much easier to understand and use, and far simpler to implement.  Instead of requiring a custom class and callbacks, the standard event mechanism in C# is used directly.  For example, the WebClient class uses this extensively.  A method is used, such as DownloadDataAsync, and the results are returned via the DownloadDataCompleted event.

While the EAP is far simpler to understand and use than the APM, it is still not ideal.  By separating your code into method calls and event handlers, the logic of your program gets more complex.  It also typically loses the ability to block until the result is received, which is often useful.  Blocking often requires writing the code to block by hand, which is error prone and adds complexity.

As a result, .NET 4 introduced a third major pattern for asynchronous programming.  The Task<T> class introduced a new, simpler concept for asynchrony.  Task and Task<T> effectively represent an operation that will complete at some point in the future.  This is a perfect model for thinking about asynchronous code, and is the preferred model for all new code going forward.  Task and Task<T> provide all of the advantages of both the APM and the EAP models – you have the ability to block on results (via Task.Wait() or Task<T>.Result), and you can stay completely asynchronous via the use of Task Continuations.  In addition, the Task class provides a new model for task composition and error and cancelation handling.  This is a far superior option to the previous asynchronous patterns.

The Visual Studio Async CTP extends the Task based asynchronous model, allowing it to be used in a much simpler manner.  However, it requires the use of Task and Task<T> for all operations.

This core of the new asynchronous functionality is built upon three key features.  First is the Task functionality in .NET 4, and based on Task and Task<TResult>.  While Task was intended to be the primary means of asynchronous programming with .NET 4, the .NET Framework was still based mainly on the Asynchronous Pattern and the Event-based Asynchronous Pattern.

The .NET Framework added functionality and guidance for wrapping existing APIs into a Task based API, but the framework itself didn’t really adopt Task or Task<TResult> in any meaningful way.  The CTP shows that, going forward, this is changing.

One of the three key new features coming in C# is actually a .NET Framework feature.  Nearly every asynchronous API in the .NET Framework has been wrapped into a new, Task-based method calls.  In the CTP, this is done via as external assembly (AsyncCtpLibrary.dll) which uses Extension Methods to wrap the existing APIs.  However, going forward, this will be handled directly within the Framework.  This will have a unifying effect throughout the .NET Framework.  This is the first building block of the new features for asynchronous programming:

Going forward, all asynchronous operations will work via a method that returns Task or Task<TResult>

The second key feature is the new async contextual keyword being added to the language.  The async keyword is used to declare an asynchronous function, which is a method that either returns void, a Task, or a Task<T>.

Inside the asynchronous function, there must be at least one await expression.  This is a new C# keyword (await) that is used to automatically take a series of statements and break it up to potentially use discontinuous evaluation.  This is done by using await on any expression that evaluates to a Task or Task<T>.

For example, suppose we want to download a webpage as a string.  There is a new method added to WebClient: Task<string> WebClient.DownloadStringTaskAsync(Uri).  Since this returns a Task<string> we can use it within an asynchronous function.  Suppose, for example, that we wanted to do something similar to my asynchronous Task example – download a web page asynchronously and check to see if it supports XHTML 1.0, then report this into a TextBox.  This could be done like so:

private async void button1_Click(object sender, RoutedEventArgs e)
{
    string url = "http://reedcopsey.com";
    string content = await new WebClient().DownloadStringTaskAsync(url);
    this.textBox1.Text = string.Format("Page {0} supports XHTML 1.0: {1}",
      url, content.Contains("XHTML 1.0"));
}

Let’s walk through what’s happening here, step by step.  By adding the async contextual keyword to the method definition, we are able to use the await keyword on our WebClient.DownloadStringTaskAsync method call.

When the user clicks this button, the new method (Task<string> WebClient.DownloadStringTaskAsync(string)) is called, which returns a Task<string>.  By adding the await keyword, the runtime will call this method that returns Task<string>, and execution will return to the caller at this point.  This means that our UI is not blocked while the webpage is downloaded.  Instead, the UI thread will “await” at this point, and let the WebClient do it’s thing asynchronously.

When the WebClient finishes downloading the string, the user interface’s synchronization context will automatically be used to “pick up” where it left off, and the Task<string> returned from DownloadStringTaskAsync is automatically unwrapped and set into the content variable.  At this point, we can use that and set our text box content.

There are a couple of key points here:

Asynchronous functions are declared with the async keyword, and contain one or more await expressions

In addition to the obvious benefits of shorter, simpler code – there are some subtle but tremendous benefits in this approach.  When the execution of this asynchronous function continues after the first await statement, the initial synchronization context is used to continue the execution of this function.  That means that we don’t have to explicitly marshal the call that sets textbox1.Text back to the UI thread – it’s handled automatically by the language and framework!  Exception handling around asynchronous method calls also just works.

I’d recommend every C# developer take a look at the documentation on the new Asynchronous Programming for C# and Visual Basic page, download the Visual Studio Async CTP, and try it out.

There is a method in the TaskFactory class which can be used to adapt the existing APIs to the new Task class: TaskFactory.FromAsync.  This method provides a way to convert from the BeginOperation/EndOperation method pair syntax common through .NET Framework directly to a Task<T> containing the results of the operation in the task’s Result parameter.

While this method does exist, it unfortunately comes at a cost – the method overloads are far from simple to decipher, and the resulting code is not always as easily understood as newer code based directly on the Task class.  For example, a single call to handle WebRequest.BeginGetResponse/EndGetReponse, one of the easiest “pairs” of methods to use, looks like the following:

var task = Task.Factory.FromAsync<WebResponse>(
                            request.BeginGetResponse,
                            request.EndGetResponse,
                            null);

The compiler is unfortunately unable to infer the correct type, and, as a result, the WebReponse must be explicitly mentioned in the method call.  As a result, I typically recommend wrapping this into an extension method to ease use.  For example, I would place the above in an extension method like:

public static class WebRequestExtensions
{
    public static Task<WebResponse> GetReponseAsync(this WebRequest request)
    {
        return Task.Factory.FromAsync<WebResponse>(
                        request.BeginGetResponse,
                        request.EndGetResponse,
                        null);
    }
}

This dramatically simplifies usage.  For example, if we wanted to asynchronously check to see if this blog supported XHTML 1.0, and report that in a text box to the user, we could do:

var webRequest = WebRequest.Create("http://www.reedcopsey.com");
webRequest.GetReponseAsync().ContinueWith(t =>
    {
        using (var sr = new StreamReader(t.Result.GetResponseStream()))
        {
            string str = sr.ReadLine();;
            this.textBox1.Text = string.Format("Page at {0} supports XHTML 1.0: {1}", 
                t.Result.ResponseUri, 
                str.Contains("XHTML 1.0"));
        }
    }, TaskScheduler.FromCurrentSynchronizationContext());

By using a continuation with a TaskScheduler based on the current synchronization context, we can keep this request asynchronous, check based on the first line of the response string, and report the results back on our UI directly.

In addition to standard Task continuations , the Task class provides some options for filtering continuations automatically.  This is handled via the TaskContinationOptions enumeration, which provides hints to the TaskScheduler that it should only continue based on the operation of the antecedent task.

This is especially useful when dealing with exceptions.  For example, we can extend the sample from our earlier continuation discussion to include support for handling exceptions thrown by the Factorize method:

// Get a copy of the UI-thread task scheduler up front to use later
var uiScheduler = TaskScheduler.FromCurrentSynchronizationContext();

// Start our task
var factorize = Task.Factory.StartNew(
    () =>
        {
            int primeFactor1 = 0;
            int primeFactor2 = 0;
            bool result = Factorize(10298312, ref primeFactor1, ref primeFactor2);
            return new {
                           Result = result,
                           Factor1 = primeFactor1,
                           Factor2 = primeFactor2
                       };
        });

// When we succeed, report the results to the UI
factorize.ContinueWith(task => textBox1.Text = string.Format("{0}/{1}  [Succeeded {2}]",
                                 task.Result.Factor1,
                                 task.Result.Factor2,
                                 task.Result.Result),
                        CancellationToken.None,
                        TaskContinuationOptions.NotOnFaulted,
                        uiScheduler);

// When we have an exception, report it
factorize.ContinueWith(task => 
                             textBox1.Text = string.Format("Error: {0}", task.Exception.Message),
                        CancellationToken.None,
                        TaskContinuationOptions.OnlyOnFaulted,
                        uiScheduler);

The above code works by using a combination of features.  First, we schedule our task, the same way as in the previous example.  However, in this case, we use a different overload of Task.ContinueWith which allows us to specify both a specific TaskScheduler (in order to have your continuation run on the UI’s synchronization context) as well as a TaskContinuationOption. 

In the first continuation, we tell the continuation that we only want it to run when there was not an exception by specifying TaskContinuationOptions.NotOnFaulted.  When our factorize task completes successfully, this continuation will automatically run on the UI thread, and provide the appropriate feedback.

However, if the factorize task has an exception – for example, if the Factorize method throws an exception due to an improper input value, the second continuation will run.  This occurs due to the specification of TaskContinuationOptions.OnlyOnFaulted in the options.  In this case, we’ll report the error received to the user.

We can use TaskContinuationOptions to filter our continuations by whether or not an exception occurred and whether or not a task was cancelled.  This allows us to handle many situations, and is especially useful when trying to maintain a valid application state without ever blocking the user interface.  The same concepts can be extended even further, and allow you to chain together many tasks based on the success of the previous ones.  Continuations can even be used to create a state machine with full error handling, all without blocking the user interface thread.

Task continuations provide a clean syntax, and a very simple, elegant means of synchronizing asynchronous method results with the user interface.  In addition, continuations provide a very simple, elegant means of working with collections of tasks.

Prior to .NET 4, working with multiple related asynchronous method calls was very tricky.  If, for example, we wanted to run two asynchronous operations, followed by a single method call which we wanted to run when the first two methods completed, we’d have to program all of the handling ourselves.  We would likely need to take some approach such as using a shared callback which synchronized against a common variable, or using a WaitHandle shared within the callbacks to allow one to wait for the second.  Although this could be accomplished easily enough, it requires manually placing this handling into every algorithm which requires this form of blocking.  This is error prone, difficult, and can easily lead to subtle bugs.

Similar to how the Task class static methods providing a way to block until multiple tasks have completed, TaskFactory contains static methods which allow a continuation to be scheduled upon the completion of multiple tasks: TaskFactory.ContinueWhenAll.

This allows you to easily specify a single delegate to run when a collection of tasks has completed.  For example, suppose we have a class which fetches data from the network.  This can be a long running operation, and potentially fail in certain situations, such as a server being down.  As a result, we have three separate servers which we will “query” for our information.  Now, suppose we want to grab data from all three servers, and verify that the results are the same from all three.

With traditional asynchronous programming in .NET, this would require using three separate callbacks, and managing the synchronization between the various operations ourselves.  The Task and TaskFactory classes simplify this for us, allowing us to write:

var server1 = Task.Factory.StartNew( 
                 () => networkClass.GetResults(firstServer) );
var server2 = Task.Factory.StartNew( 
                 () => networkClass.GetResults(secondServer) );
var server3 = Task.Factory.StartNew( 
                 () => networkClass.GetResults(thirdServer) );

var result = Task.Factory.ContinueWhenAll( new[] {server1, server2, server3 },
                 (tasks) => 
                 {
                       // Propogate exceptions (see below)
                       Task.WaitAll(tasks);

                       return this.CompareTaskResults(
                               tasks[0].Result,
                               tasks[1].Result,
                               tasks[2].Result);
                  });

This is clean, simple, and elegant.  The one complication is the Task.WaitAll(tasks); statement.

Although the continuation will not complete until all three tasks (server1, server2, and server3) have completed, there is a potential snag.  If the networkClass.GetResults method fails, and raises an exception, we want to make sure to handle it cleanly.  By using Task.WaitAll, any exceptions raised within any of our original tasks will get wrapped into a single AggregateException by the WaitAll method, providing us a simplified means of handling the exceptions.  If we wait on the continuation, we can trap this AggregateException, and handle it cleanly.  Without this line, it’s possible that an exception could remain uncaught and unhandled by a task, which later might trigger a nasty UnobservedTaskException.  This would happen any time two of our original tasks failed.

Just as we can schedule a continuation to occur when an entire collection of tasks has completed, we can just as easily setup a continuation to run when any single task within a collection completes.  If, for example, we didn’t need to compare the results of all three network locations, but only use one, we could still schedule three tasks.  We could then have our completion logic work on the first task which completed, and ignore the others.  This is done via TaskFactory.ContinueWhenAny:

var server1 = Task.Factory.StartNew( 
                 () => networkClass.GetResults(firstServer) );
var server2 = Task.Factory.StartNew( 
                 () => networkClass.GetResults(secondServer) );
var server3 = Task.Factory.StartNew( 
                 () => networkClass.GetResults(thirdServer) );

var result = Task.Factory.ContinueWhenAny( new[] {server1, server2, server3 },
                 (firstTask) => 
                 {
                       return this.ProcessTaskResult(firstTask.Result);
                  });

Here, instead of working with all three tasks, we’re just using the first task which finishes.  This is very useful, as it allows us to easily work with results of multiple operations, and “throw away” the others.  However, you must take care when using ContinueWhenAny to properly handle exceptions.  At some point, you should always wait on each task (or use the Task.Result property) in order to propogate any exceptions raised from within the task.  Failing to do so can lead to an UnobservedTaskException.

Asynchronous programming methods typically required callback functions.  For example, MSDN’s Asynchronous Delegates Programming Sample shows a class that factorizes a number.  The original method in the example has the following signature:

public static bool Factorize(int number, ref int primefactor1, ref int primefactor2)
{
 //...

However, calling this is quite “tricky”, even if we modernize the sample to use lambda expressions via C# 3.0. 

Normally, we could call this method like so:

int primeFactor1 = 0;
int primeFactor2 = 0;

bool answer = Factorize(10298312, ref primeFactor1, ref primeFactor2);
Console.WriteLine("{0}/{1}  [Succeeded {2}]", primeFactor1, primeFactor2, answer);

If we want to make this operation run in the background, and report to the console via a callback, things get tricker.  First, we need a delegate definition:

public delegate bool AsyncFactorCaller(
     int number,
     ref int primefactor1,
     ref int primefactor2);

Then we need to use BeginInvoke to run this method asynchronously:

int primeFactor1 = 0;
int primeFactor2 = 0;

AsyncFactorCaller caller  = new AsyncFactorCaller(Factorize);
caller.BeginInvoke(10298312, ref primeFactor1, ref primeFactor2,
   result =>
       {
           int factor1 = 0;
           int factor2 = 0;
           bool answer = caller.EndInvoke(ref factor1, ref factor2, result);
           Console.WriteLine("{0}/{1}  [Succeeded {2}]", factor1, factor2, answer);
       }, null);

This works, but is quite difficult to understand from a conceptual standpoint.  To combat this, the framework added the Event-based Asynchronous Pattern, but it isn’t much easier to understand or author.

Using .NET 4’s new Task<T> class and a continuation, we can dramatically simplify the implementation of the above code, as well as make it much more understandable.  We do this via the Task.ContinueWith method.  This method will schedule a new Task upon completion of the original task, and provide the original Task (including its Result if it’s a Task<T>) as an argument.  Using Task, we can eliminate the delegate, and rewrite this code like so:

var background = Task.Factory.StartNew(
    () =>
        {
            int primeFactor1 = 0;
            int primeFactor2 = 0;
            bool result = Factorize(10298312, ref primeFactor1, ref primeFactor2);
            return new {
                           Result = result,
                           Factor1 = primeFactor1,
                           Factor2 = primeFactor2
                       };
        });
background.ContinueWith(task => Console.WriteLine("{0}/{1}  [Succeeded {2}]",
                                 task.Result.Factor1,
                                 task.Result.Factor2,
                                 task.Result.Result));

This is much simpler to understand, in my opinion.  Here, we’re explicitly asking to start a new task, then continue the task with a resulting task.  In our case, our method used ref parameters (this was from the MSDN Sample), so there is a little bit of extra boiler plate involved, but the code is at least easy to understand.

That being said, this isn’t dramatically shorter when compared with our C# 3 port of the MSDN code above.  However, if we were to extend our requirements a bit, we can start to see more advantages to the Task based approach.  For example, supposed we need to report the results in a user interface control instead of reporting it to the Console.  This would be a common operation, but now, we have to think about marshaling our calls back to the user interface.  This is probably going to require calling Control.Invoke or Dispatcher.Invoke within our callback, forcing us to specify a delegate within the delegate.  The maintainability and ease of understanding drops.  However, just as a standard Task can be created with a TaskScheduler that uses the UI synchronization context, so too can we continue a task with a specific context.  There are Task.ContinueWith method overloads which allow you to provide a TaskScheduler.  This means you can schedule the continuation to run on the UI thread, by simply doing:

Task.Factory.StartNew(
    () =>
        {
            int primeFactor1 = 0;
            int primeFactor2 = 0;
            bool result = Factorize(10298312, ref primeFactor1, ref primeFactor2);
            return new {
                           Result = result,
                           Factor1 = primeFactor1,
                           Factor2 = primeFactor2
                       };
        }).ContinueWith(task => textBox1.Text = string.Format("{0}/{1}  [Succeeded {2}]",
                                 task.Result.Factor1,
                                 task.Result.Factor2,
                                 task.Result.Result), 
                        TaskScheduler.FromCurrentSynchronizationContext());

This is far more understandable than the alternative.  By using Task.ContinueWith in conjunction with TaskScheduler.FromCurrentSynchronizationContext(), we get a simple way to push any work onto a background thread, and update the user interface on the proper UI thread.  This technique works with Windows Presentation Foundation as well as Windows Forms, with no change in methodology.

While similar to the WindowCloseBehavior code I posted on the Expression Code Gallery, Nishant Sivakumar’s version works in WPF without taking a dependency on the Expression Blend SDK.

I highly recommend reading this article: Handling a Window’s Closed and Closing Events in the View-Model.  It is a very nice alternative approach to this common problem in MVVM.

I feel honored and proud to receive this award, and hope that I can continue to be a valuable member of the community in the future.  Thank you to everyone who nominated me!

In order to simplify this process, and make Tasks simple to use in simple cases, without sacrificing their power and flexibility, the Task Parallel Library added a new class: TaskFactory.

The TaskFactory class is intended to “Provide support for creating and scheduling Task objects.”  Its entire purpose is to simplify development when working with Task instances.  The Task class provides access to the default TaskFactory via the Task.Factory static property.  By default, TaskFactory uses the default TaskScheduler to schedule tasks on a ThreadPool thread.  By using Task.Factory, we can automatically create and start a task in a single “fire and forget” manner, similar to how we did with ThreadPool.QueueUserWorkItem:

Task.Factory.StartNew(() => this.ExecuteBackgroundWork(myData) );

This provides us with the same level of simplicity we had with ThreadPool.QueueUserWorkItem, but even more power.  For example, we can now easily wait on the task:

// Start our task on a background thread
var task = Task.Factory.StartNew(() => this.ExecuteBackgroundWork(myData) );

// Do other work on the main thread, 
// while the task above executes in the background
this.ExecuteWorkSynchronously();

// Wait for the background task to finish
task.Wait();

TaskFactory simplifies creation and startup of simple background tasks dramatically.

In addition to using the default TaskFactory, it’s often useful to construct a custom TaskFactory.  The TaskFactory class includes an entire set of constructors which allow you to specify the default configuration for every Task instance created by that factory. 

This is particularly useful when using a custom TaskScheduler.  For example, look at the sample code for starting a task on the UI thread in Part 15:

// Given the following, constructed on the UI thread
// TaskScheduler uiScheduler = TaskScheduler.FromCurrentSynchronizationContext();

// When inside a background task, we can do
string status = GetUpdatedStatus();

(new Task(() =>
    {
        statusLabel.Text = status;
    }))
.Start(uiScheduler);

This is actually quite a bit more complicated than necessary.  When we create the uiScheduler instance, we can use that to construct a TaskFactory that will automatically schedule tasks on the UI thread.  To do that, we’d create the following on our main thread, prior to constructing our background tasks:

// Construct a task scheduler from the current SynchronizationContext (UI thread)
var uiScheduler = TaskScheduler.FromCurrentSynchronizationContext();
// Construct a new TaskFactory using our UI scheduler
var uiTaskFactory = new TaskFactory(uiScheduler);

If we do this, when we’re on a background thread, we can use this new TaskFactory to marshal a Task back onto the UI thread.  Our previous code simplifies to:

// When inside a background task, we can do
string status = GetUpdatedStatus();

// Update our UI
uiTaskFactory.StartNew( () => statusLabel.Text = status);

Notice how much simpler this becomes!  By taking advantage of the convenience provided by a custom TaskFactory, we can now marshal to set data on the UI thread in a single, clear line of code!

Unfortunately, one perfect scenario for MEF tends to fall short of it’s full potential is in Windows Presentation Foundation development.  In particular, there are many times when the XAML parser constructs objects in WPF development, which makes composition of those parts difficult.  The current release of MEF (Preview Release 9) addresses this for Silverlight developers via System.ComponentModel.Composition.CompositionInitializer.  However, there is no equivalent class for WPF developers.

The CompositionInitializer class provides the means for an object to compose itself.  This is very useful with WPF and Silverlight development, since it allows a View, such as a UserControl, to be generated via the standard XAML parser, and still automatically pull in the appropriate ViewModel in an extensible manner.  Glenn Block has demonstrated the usage for Silverlight in detail, but the same issues apply in WPF.

As an example, let’s take a look at a very simple case.  Take the following XAML for a Window:

<Window x:Class="WpfApplication1.MainView"
        xmlns="http://schemas.microsoft.com/winfx/2006/xaml/presentation"
        xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
        Title="MainWindow" Height="220" Width="300">
    <Grid>
        <TextBlock Text="{Binding TheText}" />
    </Grid>
</Window>

This does nothing but create a Window, add a simple TextBlock control, and use it to display the value of our “TheText” property in our DataContext class.  Since this is our main window, WPF will automatically construct and display this Window, so we need to handle constructing the DataContext and setting it ourselves.

We could do this in code or in XAML, but in order to do it directly, we would need to hard code the ViewModel type directly into our XAML code, or we would need to construct the ViewModel class and set it in the code behind.  Both have disadvantages, and the disadvantages grow if we’re using MEF to compose our ViewModel.

Ideally, we’d like to be able to have MEF construct our ViewModel for us.  This way, it can provide any construction requirements for our ViewModel via [ImportingConstructor], and it can handle fully composing the imported properties on our ViewModel.  CompositionInitializer allows this to occur.

We use CompositionInitializer within our View’s constructor, and use it for self-composition of our View.  Using CompositionInitializer, we can modify our code behind to:

public partial class MainView : Window
{
    public MainView()
    {
        InitializeComponent();
        CompositionInitializer.SatisfyImports(this);
    }

    [Import("MainViewModel")]
    public object ViewModel
    {
        get { return this.DataContext; }
        set { this.DataContext = value; }
    }
}

We then can add an Export on our ViewModel class like so:

[Export("MainViewModel")]
public class MainViewModel
{
    public string TheText
    {
        get
        {
            return "Hello World!";
        }
    }
}

MEF will automatically compose our application, decoupling our ViewModel injection to the DataContext of our View until runtime.  When we run this, we’ll see:

There are many other approaches for using MEF to wire up the extensible parts within your application, of course.  However, any time an object is going to be constructed by code outside of your control, CompositionInitializer allows us to continue to use MEF to satisfy the import requirements of that object.

In order to use this from WPF, I’ve ported the code from MEF Preview 9 and Glenn Block’s (now obsolete) PartInitializer port to Windows Presentation Foundation.  There are some subtle changes from the Silverlight port, mainly to handle running in a desktop application context.  The default behavior of my port is to construct an AggregateCatalog containing a DirectoryCatalog set to the location of the entry assembly of the application.  In addition, if an “Extensions” folder exists under the entry assembly’s directory, a second DirectoryCatalog for that folder will be included.  This behavior can be overridden by specifying a CompositionContainer or one or more ComposablePartCatalogs to the System.ComponentModel.Composition.Hosting.CompositionHost static class prior to the first use of CompositionInitializer.

Please download CompositionInitializer and CompositionHost for VS 2010 RC, and contact me with any feedback.

• Composition.Initialization.Desktop.zip

Edit on 3/29:

Glenn Block has since updated his version of CompositionInitializer (and ExportFactory<T>!), and made it available here:

• http://cid-f8b2fd72406fb218.skydrive.live.com/self.aspx/blog/Composition.Initialization.Desktop.zip

This is a .NET 3.5 solution, and should soon be pushed to CodePlex, and made available on the main MEF site.

Once we’ve implemented our tasks, we need to execute them.  In the Task Parallel Library, the execution of Tasks is handled via an instance of the TaskScheduler class.

The TaskScheduler class is an abstract class which provides a single function: it schedules the tasks and executes them within an appropriate context.  This class is the class which actually runs individual Task instances.  The .NET Framework provides two (internal) implementations of the TaskScheduler class.

Since a Task, based on our decomposition, should be a self-contained piece of code, parallel execution makes sense when executing tasks.  The default implementation of the TaskScheduler class, and the one most often used, is based on the ThreadPool.  This can be retrieved via the TaskScheduler.Default property, and is, by default, what is used when we just start a Task instance with Task.Start().

Normally, when a Task is started by the default TaskScheduler, the task will be treated as a single work item, and run on a ThreadPool thread.  This pools tasks, and provides Task instances all of the advantages of the ThreadPool, including thread pooling for reduced resource usage, and an upper cap on the number of work items.  In addition, .NET 4 brings us a much improved thread pool, providing work stealing and reduced locking within the thread pool queues.  By using the default TaskScheduler, our Tasks are run asynchronously on the ThreadPool.

There is one notable exception to my above statements when using the default TaskScheduler.  If a Task is created with the TaskCreationOptions set to TaskCreationOptions.LongRunning, the default TaskScheduler will generate a new thread for that Task, at least in the current implementation.  This is useful for Tasks which will persist for most of the lifetime of your application, since it prevents your Task from starving the ThreadPool of one of it’s work threads.

The Task Parallel Library provides one other implementation of the TaskScheduler class.  In addition to providing a way to schedule tasks on the ThreadPool, the framework allows you to create a TaskScheduler which works within a specified SynchronizationContext.  This scheduler can be retrieved within a thread that provides a valid SynchronizationContext by calling the TaskScheduler.FromCurrentSynchronizationContext() method.

This implementation of TaskScheduler is intended for use with user interface development.  Windows Forms and Windows Presentation Foundation both require any access to user interface controls to occur on the same thread that created the control.  For example, if you want to set the text within a Windows Forms TextBox, and you’re working on a background thread, that UI call must be marshaled back onto the UI thread.  The most common way this is handled depends on the framework being used.  In Windows Forms, Control.Invoke or Control.BeginInvoke is most often used.  In WPF, the equivelent calls are Dispatcher.Invoke or Dispatcher.BeginInvoke.

As an example, say we’re working on a background thread, and we want to update a TextBlock in our user interface with a status label.  The code would typically look something like:

// Within background thread work...
string status = GetUpdatedStatus();

Dispatcher.BeginInvoke(DispatcherPriority.Normal,
    new Action( () => 
        {
            statusLabel.Text = status;
        }));

// Continue on in background method

This works fine, but forces your method to take a dependency on WPF or Windows Forms.  There is an alternative option, however.  Both Windows Forms and WPF, when initialized, setup a SynchronizationContext in their thread, which is available on the UI thread via the SynchronizationContext.Current property.  This context is used by classes such as BackgroundWorker to marshal calls back onto the UI thread in a framework-agnostic manner.

The Task Parallel Library provides the same functionality via the TaskScheduler.FromCurrentSynchronizationContext() method.  When setting up our Tasks, as long as we’re working on the UI thread, we can construct a TaskScheduler via:

TaskScheduler uiScheduler = TaskScheduler.FromCurrentSynchronizationContext();

We then can use this scheduler on any thread to marshal data back onto the UI thread.  For example, our code above can then be rewritten as:

string status = GetUpdatedStatus();

(new Task(() => 
    {
        statusLabel.Text = status;
    }))
.Start(uiScheduler);

// Continue on in background method

This is nice since it allows us to write code that isn’t tied to Windows Forms or WPF, but is still fully functional with those technologies.  I’ll discuss even more uses for the SynchronizationContext based TaskScheduler when I demonstrate task continuations, but even without continuations, this is a very useful construct.

In addition to the two implementations provided by the Task Parallel Library, it is possible to implement your own TaskScheduler.  The ParallelExtensionsExtras project within the Samples for Parallel Programming provides nine sample TaskScheduler implementations.  These include schedulers which restrict the maximum number of concurrent tasks, run tasks on a single threaded apartment thread, use a new thread per task, and more.

In my introduction to the Task class, I focused on the most basic version of Task.  This version of Task, the standard Task class, is most often used with an Action delegate.  This allows you to implement for each task within the task decomposition as a single delegate.

Typically, when using the new threading constructs in .NET 4 and the Task Parallel Library, we use lambda expressions to define anonymous methods.  The advantage of using a lambda expression is that it allows the Action delegate to directly use variables in the calling scope.  This eliminates the need to make separate Task classes for Action<T>, Action<T1,T2>, and all of the other Action<…> delegate types.  As an example, suppose we wanted to make a Task to handle the ”Show Splash” task from our earlier decomposition.  Even if this task required parameters, such as a message to display, we could still use an Action delegate specified via a lambda:

// Store this as a local variable
string messageForSplashScreen = GetSplashScreenMessage();
// Create our task
Task showSplashTask = new Task(
    () =>
        {
            // We can use variables in our outer scope, 
            // as well as methods scoped to our class!
            this.DisplaySplashScreen(messageForSplashScreen);
        });

This provides a huge amount of flexibility.  We can use this single form of task for any task which performs an operation, provided the only information we need to track is whether the task has completed successfully or not.  This leads to my first observation:

Use a Task with a System.Action delegate for any task for which no result is generated.

This observation leads to an obvious corollary: we also need a way to define a task which generates a result.  The Task Parallel Library provides this via the Task<TResult> class.

Task<TResult> subclasses the standard Task class, providing one additional feature – the ability to return a value back to the user of the task.  This is done by switching from providing an Action delegate to providing a Func<TResult> delegate.  If we decompose our problem, and we realize we have one task where its result is required by a future operation, this can be handled via Task<TResult>.  For example, suppose we want to make a task for our “Check for Update” task, we could do:

Task<bool> checkForUpdateTask = new Task<bool>(
    () =>
        {
            return this.CheckWebsiteForUpdate();
        });

Later, we would start this task, and perform some other work.  At any point in the future, we could get the value from the Task<TResult>.Result property, which will cause our thread to block until the task has finished processing:

// This uses Task<bool> checkForUpdateTask generated above...
// Start the task, typically on a background thread
checkForUpdateTask.Start();

// Do some other work on our current thread
this.DoSomeWork();

// Discover, from our background task, whether an update is available
// This will block until our task completes
bool updateAvailable = checkForUpdateTask.Result;

This leads me to my second observation:

Use a Task<TResult> with a System.Func<TResult> delegate for any task which generates a result.

Task and Task<TResult> provide a much cleaner alternative to the previous Asynchronous Programming design patterns in the .NET framework.  Instead of trying to implement IAsyncResult, and providing BeginXXX() and EndXXX() methods, implementing an asynchronous programming API can be as simple as creating a method that returns a Task or Task<TResult>.  The client side of the pattern also is dramatically simplified – the client can call a method, then either choose to call task.Wait() or use task.Result when it needs to wait for the operation’s completion.

While this provides a much cleaner model for future APIs, there is quite a bit of infrastructure built around the current Asynchronous Programming design patterns.  In order to provide a model to work with existing APIs, two other forms of Task exist.  There is a constructor for Task which takes an Action<Object> and a state parameter.  In addition, there is a constructor for creating a Task<TResult> which takes a Func<Object, TResult> as well as a state parameter.  When using these constructors, the state parameter is stored in the Task.AsyncState property.

While these two overloads exist, and are usable directly, I strongly recommend avoiding this for new development.  The two forms of Task which take an object state parameter exist primarily for interoperability with traditional .NET Asynchronous Programming methodologies.  Using lambda expressions to capture variables from the scope of the creator is a much cleaner approach than using the untyped state parameters, since lambda expressions provide full type safety without introducing new variables.

The Task class is a wrapper for a delegate representing a single, discrete task within your decomposition.  We will go into various methods of construction for tasks later, but, when reduced to its fundamentals, an instance of a Task is nothing more than a wrapper around a delegate with some utility functionality added. 

In order to fully understand the Task class within the new Task Parallel Library, it is important to realize that a task really is just a delegate – nothing more.  In particular, note that I never mentioned threading or parallelism in my description of a Task.  Although the Task class exists in the new System.Threading.Tasks namespace:

Tasks are not directly related to threads or multithreading.

Of course, Task instances will typically be used in our implementation of concurrency within an application, but the Task class itself does not provide the concurrency used.  The Task API supports using Tasks in an entirely single threaded, synchronous manner.

Tasks are very much like standard delegates.  You can execute a task synchronously via Task.RunSynchronously(), or you can use Task.Start() to schedule a task to run, typically asynchronously.  This is very similar to using delegate.Invoke to execute a delegate synchronously, or using delegate.BeginInvoke to execute it asynchronously.

The Task class adds some nice functionality on top of a standard delegate which improves usability in both synchronous and multithreaded environments.

The first addition provided by Task is a means of handling cancellation via the new unified cancellation mechanism of .NET 4.  If the wrapped delegate within a Task raises an OperationCanceledException during it’s operation, which is typically generated via calling ThrowIfCancellationRequested on a CancellationToken, or if the CancellationToken used to construct a Task instance is flagged as canceled, the Task’s IsCanceled property will be set to true automatically.  This provides a clean way to determine whether a Task has been canceled, often without requiring specific exception handling.

Tasks also provide a clean API which can be used for waiting on a task.  Although the Task class explicitly implements IAsyncResult, Tasks provide a nicer usage model than the traditional .NET Asynchronous Programming Model.  Instead of needing to track an IAsyncResult handle, you can just directly call Task.Wait() to block until a Task has completed.  Overloads exist for providing a timeout, a CancellationToken, or both to prevent waiting indefinitely.  In addition, the Task class provides static methods for waiting on multiple tasks – Task.WaitAll and Task.WaitAny, again with overloads providing time out options.  This provides a very simple, clean API for waiting on single or multiple tasks.

Finally, Tasks provide a much nicer model for Exception handling.  If the delegate wrapped within a Task raises an exception, the exception will automatically get wrapped into an AggregateException and exposed via the Task.Exception property.  This exception is stored with the Task directly, and does not tear down the application.  Later, when Task.Wait() (or Task.WaitAll or Task.WaitAny) is called on this task, an AggregateException will be raised at that point if any of the tasks raised an exception. 

For example, suppose we have the following code:

Task taskOne = new Task(
                       () =>
                       {
                           throw new ApplicationException("Random Exception!");
                       });
Task taskTwo = new Task(
                       () =>
                       {
                           throw new ArgumentException("Different exception here");
                       });

// Start the tasks
taskOne.Start();
taskTwo.Start();

try
{
    Task.WaitAll(new[] { taskOne, taskTwo });
}
catch (AggregateException e)
{
    Console.WriteLine(e.InnerExceptions.Count);
    foreach (var inner in e.InnerExceptions)
        Console.WriteLine(inner.Message);
}

Here, our routine will print:

2
Different exception here
Random Exception!

Note that we had two separate tasks, each of which raised two distinctly different types of exceptions.  We can handle this cleanly, with very little code, in a much nicer manner than the Asynchronous Programming API.  We no longer need to handle TargetInvocationException or worry about implementing the Event-based Asynchronous Pattern properly by setting the AsyncCompletedEventArgs.Error property.  Instead, we just raise our exception as normal, and handle AggregateException in a single location in our calling code.

However, as I mentioned in Part 1, Task Decomposition tends to be a bit more difficult than data decomposition, and can require a bit more effort.  Before we being parallelizing our algorithm based on the tasks being performed, we need to decompose our problem, and take special care of certain considerations such as ordering and grouping of tasks.

Up to this point in this series, I’ve focused on parallelization techniques which are most appropriate when a problem space can be decomposed by data.  Using PLINQ and the Parallel class, I’ve shown how problem spaces where there is a collection of data, and each element needs to be processed, can potentially be parallelized.

However, there are many other routines where this is not appropriate.  Often, instead of working on a collection of data, there is a single piece of data which must be processed using an algorithm or series of algorithms.  Here, there is no collection of data, but there may still be opportunities for parallelism.

As I mentioned before, in cases like this, the approach is to look at your overall routine, and decompose your problem space based on tasks.  The idea here is to look for discrete “tasks,” individual pieces of work which can be conceptually thought of as a single operation.

Let’s revisit the example I used in Part 1, an application startup path.  Say we want our program, at startup, to do a bunch of individual actions, or “tasks”.  The following is our list of duties we must perform right at startup:

• Display a splash screen
• Request a license from our license manager
• Check for an update to the software from our web server
• If an update is available, download it
• Setup our menu structure based on our current license
• Open and display our main, welcome Window
• Hide the splash screen

The first step in Task Decomposition is breaking up the problem space into discrete tasks.

This, naturally, can be abstracted as seven discrete tasks.  In the serial version of our program, if we were to diagram this, the general process would appear as:

These tasks, obviously, provide some opportunities for parallelism.  Before we can parallelize this routine, we need to analyze these tasks, and find any dependencies between tasks.  In this case, our dependencies include:

• The splash screen must be displayed first, and as quickly as possible.
• We can’t download an update before we see whether one exists.
• Our menu structure depends on our license, so we must check for the license before setting up the menus.
• Since our welcome screen will notify the user of an update, we can’t show it until we’ve downloaded the update.
• Since our welcome screen includes menus that are customized based off the licensing, we can’t display it until we’ve received a license.
• We can’t hide the splash until our welcome screen is displayed.

By listing our dependencies, we start to see the natural ordering that must occur for the tasks to be processed correctly.

The second step in Task Decomposition is determining the dependencies between tasks, and ordering tasks based on their dependencies.

Looking at these tasks, and looking at all the dependencies, we quickly see that even a simple decomposition such as this one can get quite complicated.  In order to simplify the problem of defining the dependencies, it’s often a useful practice to group our tasks into larger, discrete tasks.  The goal when grouping tasks is that you want to make each task “group” have as few dependencies as possible to other tasks or groups, and then work out the dependencies within that group.  Typically, this works best when any external dependency is based on the “last” task within the group when it’s ordered, although that is not a firm requirement.  This process is often called Grouping Tasks.  In our case, we can easily group together tasks, effectively turning this into four discrete task groups:

1. Show our splash screen –

This needs to be left as its own task.  First, multiple things depend on this task, mainly because we want this to start before any other action, and start as quickly as possible.

2. Check for Update and Download the Update if it Exists -

These two tasks logically group together.  We know we only download an update if the update exists, so that naturally follows.  This task has one dependency as an input, and other tasks only rely on the final task within this group.

3. Request a License, and then Setup the Menus –

Here, we can group these two tasks together.  Although we mentioned that our welcome screen depends on the license returned, it also depends on setting up the menu, which is the final task here.  Setting up our menus cannot happen until after our license is requested.  By grouping these together, we further reduce our problem space.

4. Display welcome and hide splash -

Finally, we can display our welcome window and hide our splash screen.  This task group depends on all three previous task groups – it cannot happen until all three of the previous groups have completed.

By grouping the tasks together, we reduce our problem space, and can naturally see a pattern for how this process can be parallelized.  The diagram below shows one approach:

The orange boxes show each task group, with each task represented within.  We can, now, effectively take these tasks, and run a large portion of this process in parallel, including the portions which may be the most time consuming.  We’ve now created two parallel paths which our process execution can follow, hopefully speeding up the application startup time dramatically.

The main point to remember here is that, when decomposing your problem space by tasks, you need to:

• Define each discrete action as an individual Task
• Discover dependencies between your tasks
• Group tasks based on their dependencies
• Order the tasks and groups of tasks

Recursive algorithms, especially ones based on a form of divide and conquer, are often a very good candidate for parallelization.

This is apparent from a common sense standpoint.  Since we’re dividing up the total work in the algorithm, we have an obvious, built-in partitioning scheme.  Once partitioned, the data can be worked upon independently, so there is good, clean isolation of data.

Implementing this type of algorithm is fairly simple.  The Parallel class in .NET 4 includes a method suited for this type of operation: Parallel.Invoke.  This method works by taking any number of delegates defined as an Action, and operating them all in parallel.  The method returns when every delegate has completed:

Parallel.Invoke(
    () => 
        {
            Console.WriteLine("Action 1 executing in thread {0}",
                                                 Thread.CurrentThread.ManagedThreadId);
        },
    () => 
        {
            Console.WriteLine("Action 2 executing in thread {0}",
                                                 Thread.CurrentThread.ManagedThreadId);
        },
    () => 
        {
            Console.WriteLine("Action 3 executing in thread {0}",
                                                 Thread.CurrentThread.ManagedThreadId);
        }
    );

Running this simple example demonstrates the ease of using this method.  For example, on my system, I get three separate thread IDs when running the above code.  By allowing any number of delegates to be executed directly, concurrently, the Parallel.Invoke method provides us an easy way to parallelize any algorithm based on divide and conquer.  We can divide our work in each step, and execute each task in parallel, recursively.

For example, suppose we wanted to implement our own quicksort routine.  The quicksort algorithm can be designed based on divide and conquer.  In each iteration, we pick a pivot point, and use that to partition the total array.  We swap the elements around the pivot, then recursively sort the lists on each side of the pivot. 

For example, let’s look at this simple, sequential implementation of quicksort:

public static void QuickSort<T>(T[] array) where T : IComparable<T>
{
    QuickSortInternal(array, 0, array.Length - 1);
}

private static void QuickSortInternal<T>(T[] array, int left, int right)
    where T : IComparable<T>
{
    if (left >= right)
    {
        return;
    }

    SwapElements(array, left, (left + right) / 2);
    int last = left;
    for (int current = left + 1; current <= right; ++current)
    {
        if (array[current].CompareTo(array[left]) < 0)
        {
            ++last;
            SwapElements(array, last, current);
        }
    }

    SwapElements(array, left, last);

    QuickSortInternal(array, left, last - 1);
    QuickSortInternal(array, last + 1, right);
}

static void SwapElements<T>(T[] array, int i, int j)
{
    T temp = array[i];
    array[i] = array[j];
    array[j] = temp;
}

Here, we implement the quicksort algorithm in a very common, divide and conquer approach.  Running this against the built-in Array.Sort routine shows that we get the exact same answers (although the framework’s sort routine is slightly faster).  On my system, for example, I can use framework’s sort to sort ten million random doubles in about 7.3s, and this implementation takes about 9.3s on average.

Looking at this routine, though, there is a clear opportunity to parallelize.  At the end of QuickSortInternal, we recursively call into QuickSortInternal with each partition of the array after the pivot is chosen.  This can be rewritten to use Parallel.Invoke by simply changing it to:

    // Code above is unchanged...
    SwapElements(array, left, last);

    Parallel.Invoke(
        () => QuickSortInternal(array, left, last - 1),
        () => QuickSortInternal(array, last + 1, right)
    );
}

This routine will now run in parallel.  When executing, we now see the CPU usage across all cores spike while it executes. 

However, there is a significant problem here – by parallelizing this routine, we took it from an execution time of 9.3s to an execution time of approximately 14 seconds!  We’re using more resources as seen in the CPU usage, but the overall result is a dramatic slowdown in overall processing time.

This occurs because parallelization adds overhead.  Each time we split this array, we spawn two new tasks to parallelize this algorithm!  This is far, far too many tasks for our cores to operate upon at a single time.  In effect, we’re “over-parallelizing” this routine.  This is a common problem when working with divide and conquer algorithms, and leads to an important observation:

When parallelizing a recursive routine, take special care not to add more tasks than necessary to fully utilize your system.

This can be done with a few different approaches, in this case.  Typically, the way to handle this is to stop parallelizing the routine at a certain point, and revert back to the serial approach.  Since the first few recursions will all still be parallelized, our “deeper” recursive tasks will be running in parallel, and can take full advantage of the machine.  This also dramatically reduces the overhead added by parallelizing, since we’re only adding overhead for the first few recursive calls. 

There are two basic approaches we can take here.  The first approach would be to look at the total work size, and if it’s smaller than a specific threshold, revert to our serial implementation.  In this case, we could just check right-left, and if it’s under a threshold, call the methods directly instead of using Parallel.Invoke.

The second approach is to track how “deep” in the “tree” we are currently at, and if we are below some number of levels, stop parallelizing.  This approach is a more general-purpose approach, since it works on routines which parse trees as well as routines working off of a single array, but may not work as well if a poor partitioning strategy is chosen or the tree is not balanced evenly.

This can be written very easily.  If we pass a maxDepth parameter into our internal routine, we can restrict the amount of times we parallelize by changing the recursive call to:

// Code above is unchanged...
SwapElements(array, left, last);

if (maxDepth < 1)
{
    QuickSortInternal(array, left, last - 1, maxDepth);
    QuickSortInternal(array, last + 1, right, maxDepth);
}
else
{
    --maxDepth;
    Parallel.Invoke(
        () => QuickSortInternal(array, left, last - 1, maxDepth),
        () => QuickSortInternal(array, last + 1, right, maxDepth));
}

We no longer allow this to parallelize indefinitely – only to a specific depth, at which time we revert to a serial implementation.  By starting the routine with a maxDepth equal to Environment.ProcessorCount, we can restrict the total amount of parallel operations significantly, but still provide adequate work for each processing core.

With this final change, my timings are much better.  On average, I get the following timings:

• Framework via Array.Sort: 7.3 seconds
• Serial Quicksort Implementation: 9.3 seconds
• Naive Parallel Implementation: 14 seconds
• Parallel Implementation Restricting Depth: 4.7 seconds

Finally, we are now faster than the framework’s Array.Sort implementation.

Cancellation in .NET 4 is based around a new, lightweight struct called CancellationToken.  A CancellationToken is a small, thread-safe value type which is generated via a CancellationTokenSource.  There are many goals which led to this design.  For our purposes, we will focus on a couple of specific design decisions:

• Cancellation is cooperative.  A calling method can request a cancellation, but it’s up to the processing routine to terminate – it is not forced.
• Cancellation is consistent.  A single method call requests a cancellation on every copied CancellationToken in the routine.

Let’s begin by looking at how we can cancel a PLINQ query.  Supposed we wanted to provide the option to cancel our query from Part 6:

double min = collection
                .AsParallel()
                .Min(item => item.PerformComputation());

We would rewrite this to allow for cancellation by adding a call to ParallelEnumerable.WithCancellation as follows:

var cts = new CancellationTokenSource();

// Pass cts here to a routine that could, 
// in parallel, request a cancellation

try
{
    double min = collection
                    .AsParallel()
                    .WithCancellation(cts.Token)
                    .Min(item => item.PerformComputation());
}
catch (OperationCanceledException e)
{
    // Query was cancelled before it finished
}

Here, if the user calls cts.Cancel() before the PLINQ query completes, the query will stop processing, and an OperationCanceledException will be raised. 

Be aware, however, that cancellation will not be instantaneous.  When cts.Cancel() is called, the query will only stop after the current item.PerformComputation() elements all finish processing.  cts.Cancel() will prevent PLINQ from scheduling a new task for a new element, but will not stop items which are currently being processed.  This goes back to the first goal I mentioned – Cancellation is cooperative.  Here, we’re requesting the cancellation, but it’s up to PLINQ to terminate.

If we wanted to allow cancellation to occur within our routine, we would need to change our routine to accept a CancellationToken, and modify it to handle this specific case:

public void PerformComputation(CancellationToken token)
{
    for (int i=0; i<this.iterations; ++i)
    {
        // Add a check to see if we've been canceled
        // If a cancel was requested, we'll throw here
        token.ThrowIfCancellationRequested();

        // Do our processing now
        this.RunIteration(i);
    }
}

With this overload of PerformComputation, each internal iteration checks to see if a cancellation request was made, and will throw an OperationCanceledException at that point, instead of waiting until the method returns.  This is good, since it allows us, as developers, to plan for cancellation, and terminate our routine in a clean, safe state.

This is handled by changing our PLINQ query to:

try
{
    double min = collection
                    .AsParallel()
                    .WithCancellation(cts.Token)
                    .Min(item => item.PerformComputation(cts.Token));
}
catch (OperationCanceledException e)
{
    // Query was cancelled before it finished
}

PLINQ is very good about handling this exception, as well.  There is a very good chance that multiple items will raise this exception, since the entire purpose of PLINQ is to have multiple items be processed concurrently.  PLINQ will take all of the OperationCanceledException instances raised within these methods, and merge them into a single OperationCanceledException in the call stack.  This is done internally because we added the call to ParallelEnumerable.WithCancellation.

If, however, a different exception is raised by any of the elements, the OperationCanceledException as well as the other Exception will be merged into a single AggregateException.

The Task Parallel Library uses the same cancellation model, as well.  Here, we supply our CancellationToken as part of the configuration.  The ParallelOptions class contains a property for the CancellationToken.  This allows us to cancel a Parallel.For or Parallel.ForEach routine in a very similar manner to our PLINQ query.  As an example, we could rewrite our Parallel.ForEach loop from Part 2 to support cancellation by changing it to:

try
{
    var cts = new CancellationTokenSource();
    var options = new ParallelOptions()
                      {
                          CancellationToken = cts.Token
                      };
    Parallel.ForEach(customers, options, customer =>
    {
        // Run some process that takes some time...
        DateTime lastContact = theStore.GetLastContact(customer);
        TimeSpan timeSinceContact = DateTime.Now - lastContact;

        // Check for cancellation here
        options.CancellationToken.ThrowIfCancellationRequested();

        // If it's been more than two weeks, send an email, and update...
        if (timeSinceContact.Days > 14)
        {
           theStore.EmailCustomer(customer);
           customer.LastEmailContact = DateTime.Now;
        }
    });
}
catch (OperationCanceledException e)
{
    // The loop was cancelled
}

Notice that here we use the same approach taken in PLINQ.  The Task Parallel Library will automatically handle our cancellation in the same manner as PLINQ, providing a clean, unified model for cancellation of any parallel routine.  The TPL performs the same aggregation of the cancellation exceptions as PLINQ, as well, which is why a single exception handler for OperationCanceledException will cleanly handle this scenario.  This works because we’re using the same CancellationToken provided in the ParallelOptions.  If a different exception was thrown by one thread, or a CancellationToken from a different CancellationTokenSource was used to raise our exception, we would instead receive all of our individual exceptions merged into one AggregateException.

When working with Data Parallelism, there is one primary configuration option we often need to control – the number of threads we want the system to use when parallelizing our routine.  By default, PLINQ and the TPL both use the ThreadPool to schedule tasks.  Given the major improvements in the ThreadPool in CLR 4, this default behavior is often ideal. 

However, there are times that the default behavior is not appropriate.  For example, if you are working on multiple threads simultaneously, and want to schedule parallel operations from within both threads, you might want to consider restricting each parallel operation to using a subset of the processing cores of the system.  Not doing this might over-parallelize your routine, which leads to inefficiencies from having too many context switches.

In the Task Parallel Library, configuration is handled via the ParallelOptions class.  All of the methods of the Parallel class have an overload which accepts a ParallelOptions argument.

We configure the Parallel class by setting the ParallelOptions.MaxDegreeOfParallelism property.  For example, let’s revisit one of the simple data parallel examples from Part 2:

Parallel.For(0, pixelData.GetUpperBound(0), row =>
{
    for (int col=0; col < pixelData.GetUpperBound(1); ++col)
    {
        pixelData[row, col] = AdjustContrast(pixelData[row, col], minPixel, maxPixel);
    }
});

Here, we’re looping through an image, and calling a method on each pixel in the image.  If this was being done on a separate thread, and we knew another thread within our system was going to be doing a similar operation, we likely would want to restrict this to using half of the cores on the system.  This could be accomplished easily by doing:

var options = new ParallelOptions();
options.MaxDegreeOfParallelism = Math.Max(Environment.ProcessorCount / 2, 1);

Parallel.For(0, pixelData.GetUpperBound(0), options, row =>
{
    for (int col=0; col < pixelData.GetUpperBound(1); ++col)
    {
        pixelData[row, col] = AdjustContrast(pixelData[row, col], minPixel, maxPixel);
    }
});

Now, we’re restricting this routine to using no more than half the cores in our system.  Note that I included a check to prevent a single core system from supplying zero; without this check, we’d potentially cause an exception.  I also did not hard code a specific value for the MaxDegreeOfParallelism property.  One of our goals when parallelizing a routine is allowing it to scale on better hardware.  Specifying a hard-coded value would contradict that goal.

Parallel LINQ also supports configuration, and in fact, has quite a few more options for configuring the system.  The main configuration option we most often need is the same as our TPL option: we need to supply the maximum number of processing threads.  In PLINQ, this is done via a new extension method on ParallelQuery<T>: ParallelEnumerable.WithDegreeOfParallelism.

Let’s revisit our declarative data parallelism sample from Part 6:

double min = collection.AsParallel().Min(item => item.PerformComputation());

Here, we’re performing a computation on each element in the collection, and saving the minimum value of this operation.  If we wanted to restrict this to a limited number of threads, we would add our new extension method:

int maxThreads = Math.Max(Environment.ProcessorCount / 2, 1);
double min = collection
                 .AsParallel()
                 .WithDegreeOfParallelism(maxThreads)
                 .Min(item => item.PerformComputation());

This automatically restricts the PLINQ query to half of the threads on the system.

PLINQ provides some additional configuration options.  By default, PLINQ will occasionally revert to processing a query in parallel.  This occurs because many queries, if parallelized, typically actually cause an overall slowdown compared to a serial processing equivalent.  By analyzing the “shape” of the query, PLINQ often decides to run a query serially instead of in parallel.  This can occur for (taken from MSDN):

• Queries that contain a Select, indexed Where, indexed SelectMany, or ElementAt clause after an ordering or filtering operator that has removed or rearranged original indices.
• Queries that contain a Take, TakeWhile, Skip, SkipWhile operator and where indices in the source sequence are not in the original order.
• Queries that contain Zip or SequenceEquals, unless one of the data sources has an originally ordered index and the other data source is indexable (i.e. an array or IList(T)).
• Queries that contain Concat, unless it is applied to indexable data sources.
• Queries that contain Reverse, unless applied to an indexable data source.

If the specific query follows these rules, PLINQ will run the query on a single thread.  However, none of these rules look at the specific work being done in the delegates, only at the “shape” of the query.  There are cases where running in parallel may still be beneficial, even if the shape is one where it typically parallelizes poorly.  In these cases, you can override the default behavior by using the WithExecutionMode extension method.  This would be done like so:

var reversed = collection
                  .AsParallel()
                  .WithExecutionMode(ParallelExecutionMode.ForceParallelism)
                  .Select(i => i.PerformComputation())
                  .Reverse();

Here, the default behavior would be to not parallelize the query unless collection implemented IList<T>.  We can force this to run in parallel by adding the WithExecutionMode extension method in the method chain.

Finally, PLINQ has the ability to configure how results are returned.  When a query is filtering or selecting an input collection, the results will need to be streamed back into a single IEnumerable<T> result.  For example, the method above returns a new, reversed collection.  In this case, the processing of the collection will be done in parallel, but the results need to be streamed back to the caller serially, so they can be enumerated on a single thread.

This streaming introduces overhead.  IEnumerable<T> isn’t designed with thread safety in mind, so the system needs to handle merging the parallel processes back into a single stream, which introduces synchronization issues.  There are two extremes of how this could be accomplished, but both extremes have disadvantages.

The system could watch each thread, and whenever a thread produces a result, take that result and send it back to the caller.  This would mean that the calling thread would have access to the data as soon as data is available, which is the benefit of this approach.  However, it also means that every item is introducing synchronization overhead, since each item needs to be merged individually.

On the other extreme, the system could wait until all of the results from all of the threads were ready, then push all of the results back to the calling thread in one shot.  The advantage here is that the least amount of synchronization is added to the system, which means the query will, on a whole, run the fastest.  However, the calling thread will have to wait for all elements to be processed, so this could introduce a long delay between when a parallel query begins and when results are returned.

The default behavior in PLINQ is actually between these two extremes.  By default, PLINQ maintains an internal buffer, and chooses an optimal buffer size to maintain.  Query results are accumulated into the buffer, then returned in the IEnumerable<T> result in chunks.  This provides reasonably fast access to the results, as well as good overall throughput, in most scenarios.

However, if we know the nature of our algorithm, we may decide we would prefer one of the other extremes.  This can be done by using the WithMergeOptions extension method.  For example, if we know that our PerformComputation() routine is very slow, but also variable in runtime, we may want to retrieve results as they are available, with no bufferring.  This can be done by changing our above routine to:

var reversed = collection
                  .AsParallel()
                  .WithExecutionMode(ParallelExecutionMode.ForceParallelism)
                  .WithMergeOptions(ParallelMergeOptions.NotBuffered)
                  .Select(i => i.PerformComputation())
                  .Reverse();

On the other hand, if are already on a background thread, and we want to allow the system to maximize its speed, we might want to allow the system to fully buffer the results:

var reversed = collection
                  .AsParallel()
                  .WithExecutionMode(ParallelExecutionMode.ForceParallelism)
                  .WithMergeOptions(ParallelMergeOptions.FullyBuffered)
                  .Select(i => i.PerformComputation())
                  .Reverse();

Notice, also, that you can specify multiple configuration options in a parallel query.  By chaining these extension methods together, we generate a query that will always run in parallel, and will always complete before making the results available in our IEnumerable<T>.