Asynchronous Programming

This topic looks at the challenges and solutions in building concurrent systems. Concurrency means two or more things happening at the same time, and programming in the face of concurrency is hard. It is also important, however – as we reach the limits in how fast a computer can process data it becomes increasingly important to parallelise execution, so we can do multiple things at once.

Your goal is to understand:

• Threads – the basic unit of parallelised work
• The basic problems with parallel execution of code – shared state, locking and deadlocks
• The C# tools available to support concurrent programming

Threads

You will have plenty of experience of computers performing multiple tasks at once. Running the web browser in which you’re reading this article doesn’t stop your email client checking for new mail. The Windows operating system handles this via multiple independent processes, roughly one per application. The physical CPU inside your computer doesn’t necessarily execute multiple processes at the same time, but short slices of time are given to each process in turn and this happens fast enough to give the impression of simultaneous execution. Different processes are largely isolated from each other, so your web browser and email client can’t interact with each other except via a few well-defined pathways.

A similar concept exists within each process, which can have multiple threads. Just like processes, threads share time on the CPU and hence many threads can make progress in parallel. Threads are the means by which you enable your code to do multiple things at the same time.

For example, consider the behaviour of your web browser. If you click a link, the browser starts loading the new page. But while it’s doing this you still have access to the Home button, can create new tabs, etc. This is all handled via threads. A typical desktop application has a dedicated UI thread which handles all user input, while separate threads handle any long-running processing (such as loading a web page). In this way the UI remains responsive to user input at all times, however much background activity is taking place.

Creating a new thread in C# can be done by creating a new instance of the Thread class.

new Thread(timer.Run).Start();

We give the thread a function to execute, and tell it to start. There are plenty of more advanced options you can specify, but the above will do for a simple background thread, and for anything more complicated you should instead use one of the other techniques discussed later in this topic.

Concurrent access to data

The example above shows how to start a thread. Having done so, you have two parallel streams of execution within your program. In the majority of cases these parallel streams will need to communicate in some manner – otherwise it is hard to realise the benefits of having the work done in parallel. (The main exception is where the thread is outputting some data to another system, and you have no interest in the result of that operation or even whether it succeeds).

The benefit of using multiple threads, rather than separate processes, is that the threads share the same memory and can happily access the same variables. Thus, the most common communication mechanism is accessing shared data structures. For example, you might have a queue of records to process and several threads popping records off the queue and processing them in parallel – that queue provides a communication channel between your threads.

However, having multiple threads access the same data simultaneously can cause problems. Consider something as simple as executing the statement x++. Suppose two threads have access to the same x and both execute this statement at the same time – what happens?

What you probably want to happen is for the value of x to increment by 2, and this might be the outcome. However, it might not be. Internally x++ is implemented as two operations, one that fetches the value of x and adds one to it, and another that updates x with a new value. It’s a bit like this:

var temp = x + 1;
x = temp;

Consider what happens if our two threads are running at exactly the same time, if x starts off equalling 1. Note that each thread has its own temp, because this is local to a particular method rather than being shared state.

x has only been incrementing by 1 overall – the two threads did the same thing at the same time, rather than being cumulative. This is probably a bug in our program.

In practice the threads probably won’t execute at literally the same time, even if they try to; they will be executed in turn by the CPU. That means there are two ways this can play out:

Option 1:

Option 2:

Option 2 is probably what we wanted to happen. But whether we hit option 1 or 2 depends entirely on when the CPU decides to stop giving time to one thread and start giving time to the other. Which scenario occurs is not under our control. This adds a further complication to the issue of building concurrent systems – the behaviour is not consistent, and bugs will often appear only intermittently. It’s even perfectly possible to have bugs that will regularly occur in the field, but will rarely or even never occur when running the code in the debugger!

Locking

The straightforward resolution to the challenge above is to synchronise access to the relevant code – prevent two threads from accessing the data at the same time. Let’s build a slightly more realistic example that you can execute:

private class DataStore { public int Value { get; set; } }
 
private DataStore store = new DataStore();
 
public void ConcurrencyTest()
{
  var thread1 = new Thread(IncrementTheValue);
  var thread2 = new Thread(IncrementTheValue);
 
  thread1.Start();
  thread2.Start();
 
  thread1.Join(); // Wait for the thread to finish executing
  thread2.Join();
 
  Console.WriteLine($"Final value: {store.Value}");
}
 
private void IncrementTheValue()
{
  store.Value++;
}

If you run this enough times, you should see answers of both 1 and 2.

We can fix this with a lock:

private void IncrementTheValue()
{
  lock (store)
  {
    store.Value++;
  }
}

However, many times you run this you should see a value of 2 outputted. The code now obtains a lock on the store object (that is, on this specific instance in memory, not on the variable name store). Only one thread can obtain such a lock at a time. Whichever thread gets to the lock statement first will get the lock; the other thread cannot pass this point until the first thread has released the lock, which happens automatically at the end of the lock block. Hence, option 1 can never occur.

You can create a lock on any object. It doesn’t actually have to be related to the data you’re modifying, although it’s a good idea if it is because that helps ensure the code makes sense to future readers.

Problems with locking

The goal of concurrency is to allow the computer to achieve several things at once. Locking therefore has one obvious problem, which is that it prevents that from happening. In the example above, the reason why the concurrency problem went away is that we forced the system into executing only one thing at a time.

Now if we assume that in a real system there’s a lot more going on, and it’s only this one line of code that needs a lock around it, that’s not too bad. Most of the work is being parallelised, and it’s just occasionally that we need to prevent that. And indeed there will be no pause unless the threads happen to hit this line of code at exactly the same time. The only performance penalty suffered by the code is the small cost of managing the locks.

However, if there needs to be a lot of code that’s synchronised via a lock, or if the blocking behaviour (one thread waiting for a lock) happens frequently, the performance impact can become significant and undermine the benefits of having parallelised execution in the first place.

There are also other problems with locks. For example, if there’s more than one independent lock then a deadlock can occur. This is where two threads are each waiting for each other. There’s a classic canned example of this called the Dining Philosophers problem:

The obvious solution is this. Each philosopher should:

• Keep philosophising until the left fork is available, and then pick it up
• Continue philosophising until the right fork is available, and then pick that up
• Eat their fill
• Put down the right fork, then the left fork
• Repeat

However, there is a problem with this solution. What happens if all five philosophers pick up the left fork (first bullet)? All the forks are now in use, so none can pick up a second fork. All five philosophers are stuck on the second bullet, and will eventually starve.

It is very easy to create a similar scenario in your code, for example by adding a second shared variable to the sample code above. If some threads take out locks in one order, and other threads take out locks in a different order, deadlock will – occasionally – occur. Your code will hang indefinitely.

Concurrent collections

The risk of locking problems makes it very hard to write safe multithreaded code. .NET provides some help in the form of concurrent collections, in the System.Collections.Concurrent namespace. These provide a variety of standard data structures that are inherently thread-safe, that is, you can call them from multiple threads without needing to manage your own locking. For example:

• ConcurrentDictionary<TKey,TValue> provides a thread-safe dictionary. In multithreaded code the standard paradigm of test whether a value is in the dictionary; if not, then add it isn’t safe (why not?), so ConcurrentDictionary contains an AddOrUpdate method that will in a single operation either add the item or replace the existing value at that key.
• ConcurrentBag<T> provides an unordered collection of objects

And so on.

Note that these don’t remove all the underlying challenges of multithreaded code, but they do help reduce the amount of manual effort required.

Immutable data structures

You may recall the claim in an earlier topic that immutable data structures – i.e. those whose contents are unchanging – are useful in multithreaded code. Hopefully the above discussion helps to illustrate why. Locking, with its associated problems, is only required if you are going to modify some data. Merely reading unchanging data is fine – many pieces of code can look at the same location in memory at the same time, without problem.

Consider an example: You have a fixed list of 100 tasks, and you want to divide these between two threads for execution. Here are two solutions:

• Solution A. Put the tasks onto a Queue. Start two threads, and have each repeatedly pull a task off the queue until the queue is empty.
• Solution B. Put the tasks in an array. Start two threads, one of which is responsible for even numbered items in the array and one for odd numbered items. Each thread maintains a variable to track progress through the array, until both have reached the end.

Which is better? There are potentially some big advantages to A – if the tasks are of uneven complexity then you will evenly balance the effort between the threads, while with B one thread may end up taking a lot longer if it gets unlucky with its tasks. However, B is much simpler in terms of concurrency – the only shared state is immutable (the variables to track progress are local to each thread), so, unlike solution A no locking is required.

In this simple example the locking required to support A would be fairly straightforward, and in any case .NET provides a ConcurrentQueue<T> for precisely this sort of scenario. But keeping data immutable where possible eliminates a whole tricky class of problems, and is therefore a trick well worth bearing in mind.

Library support for simpler threading

C# and .NET have evolved a range of different options for helping to manage multithreaded programming without having to worry too much about managing threads and shared state. We will look very briefly at some common options – you will need to do further research before using any of these approaches extensively in your own coding, but it’s important to be familiar with them.

Thread pools

One challenge with creating threads is managing them all. In particular, if you use large numbers of threads then creating them all can be expensive, and you run the risk of trying to create too many and hitting operating system limits. In general, except small numbers of long-running background threads, you should avoid creating threads by hand and instead use the thread pool. This is a set of threads managed by .NET which can be reused as needed throughout the lifetime of your application. Rather than creating a thread, you ask the thread pool to execute a method; it will do so as soon as possible (normally straight away, but if there are insufficient threads in the pool it will queue up the new activity until there’s space). Once your code is complete, the thread is available for the thread pool to allocate another activity to.

foreach (var keeper in keepers)
{
    foreach (var animal in keeper.GetResponsibleAnimals<Animal>())
    {
          if (animal.IsHungry())
          {
                ThreadPool.QueueUserWorkItem(_ => keeper.FeedAnimal(animal));
          }
    }
}

Tasks

Threads in C# map directly onto the underlying operating system concept of a thread. However, working at this low level of detail is not necessarily the easiest approach. Typically, what you really want to do is get some task to be executed, and once that task is complete perhaps access its results or perform some further task. The .NET Task object provides a more convenient abstraction for thinking at this level.

foreach (var keeper in keepers)
{
    foreach (var animal in keeper.GetResponsibleAnimals<ICanBeGroomed>())
    {
        Task.Run(() => keeper.GroomAnimal(animal));
    }
}

In this case all we do is let the Task run and then forget about it. We don’t do anything particularly interesting with the results. But Tasks have a lot more power. Task.Run returns a Task (in this case) or Task<T> (if the method passed in returns a value of type T). This is a representation of the task – you can:

• Find out whether it’s finished yet, or wait for it to finish
• Obtain the value return (if it has finished)
• Tell this task to ContinueWith another task, i.e. create a new task by stringing together a load of smaller tasks

This can be a very useful abstraction for parallel programming. All the detail of scheduling tasks to be run on appropriate threads is taken away from you, and you don’t even need to use shared state to pass data between your various activities because the Task object will wrap up the return values for you.

Parallel library

When you’re using threads to improve performance, a common pattern is to divide work up between the threads. This is what’s happening in the examples above. The Parallel library simplifies this one step further – it provides variants of For and ForEach that automatically execute the loop body in parallel across multiple threads.

Parallel.ForEach(keepers, keeper =>
{
    foreach (var animal in keeper.GetResponsibleAnimals<Animal>())
    {
          if (animal.IsHungry())
          {
                keeper.FeedAnimal(animal);
          }
    }
});

In fact there won’t necessarily be one thread per keeper – it all depends on how many keepers there are. The Parallel library will make a sensible decision about how to divide up the workload between threads (and there are overloads of the ForEach method that give you some control over this decision-making process). This cuts down significantly on the amount of work you need to do to parallelise the activities within the loop. The previous version that used the ThreadPool would probably have run into problems (or at least inefficiencies) with very large numbers of keepers, but Parallel.ForEach will do the worrying about that for you.

PLINQ

Parallelising foreach statements is all very well, but if you’re thinking functionally then you will probably be using LINQ instead. Fortunately, there is a parallel version of this – Parallel LINQ (PLINQ). This is based on the AsParallel() method that you can call on any IEnumerable<T>.

Asynchronous programming

Asynchronous, or non-blocking, programming means writing code that doesn’t get delayed, or blocked, waiting for external activity. Asynchronous programming is different from multithreaded programming – the external activity might be something that happens on a separate thread, but it might be something completely outside our application like a remote web server we are downloading some content from.

Consider this sample code, which (somewhat naively – don’t try this at home) checks how many words in an MSDN article aren’t real words according to a downloaded word list:

public int CountNonExistentWords()
{
  string article = new WebClient().DownloadString(
    @"https://msdn.microsoft.com/en-gb/library/mt674882.aspx");
  string words = new WebClient().DownloadString(
    @"https://github.com/dwyl/english-words");
 
  HashSet<string> wordList = new HashSet<string>(words.Split('\n'));
 
  var nonExistentWords = 0;
 
  foreach (string word in article.Split('\n', ' '))
  {
    if (!wordList.Contains(word)) nonExistentWords++;
  }
 
  return nonExistentWords;
}

This code runs and produces a result. However, it is slower than it needs to be – it blocks on both the DownloadString calls. We can try to optimise our own code as much as we like, but ultimately the slow bit is probably this download from a remote site.

Fortunately WebClient has a DownloadStringTaskAsync method which performs the download asynchronously and returns a Task<string> which will eventually allow us to see the result. This allows us to carry out the two downloads in parallel:

public int CountNonExistentWords()
{
  Task<string> articleTask = new WebClient().DownloadStringTaskAsync(
    @"https://msdn.microsoft.com/en-gb/library/mt674882.aspx");
  Task<string> wordsTask = new WebClient().DownloadStringTaskAsync(
    @"https://github.com/dwyl/english-words");
 
  string article = articleTask.Result;
  string words = wordsTask.Result;
 
  HashSet<string> wordList = new HashSet<string>(words.Split('\n'));
 
  var nonExistentWords = 0;
 
  foreach (string word in article.Split('\n', ' '))
  {
    if (!wordList.Contains(word)) nonExistentWords++;
  }
 
  return nonExistentWords;
}

Now our code is only as slow as the slowest download – so potentially as much as twice as quick as it was before. We did this simply by avoiding our code having to sit around waiting for so long.

This is all very well, but our CountNonExistentWords method is still pretty slow. Suppose we want to perform a couple of similar operations at the same time – we need CountNonExistentWords itself to be called asynchronously and return a Task<int>, so that the calling code can get on and do something else while it’s waiting. C# provides a convenient language feature for helping to write fully asynchronous code, in the async and await keywords. Here’s how to modify the above example to take advantage:

public async Task<int> CountNonExistentWordsAsync()
{
  Task<string> articleTask = new WebClient().DownloadStringTaskAsync(
    @"https://msdn.microsoft.com/en-gb/library/mt674882.aspx");
  Task<string> wordsTask = new WebClient().DownloadStringTaskAsync(
    @"https://github.com/dwyl/english-words");
 
  string article = await articleTask;
  string words = await wordsTask;
 
  HashSet<string> wordList = new HashSet<string>(words.Split('\n'));
 
  var nonExistentWords = 0;
 
  foreach (string word in article.Split('\n', ' '))
  {
    if (!wordList.Contains(word)) nonExistentWords++;
  }
 
  return nonExistentWords;
}

Note that when we want to get the value out of the Task<string>s, we await these tasks. This is a cue to the C# compiler that it should rearrange our code and ensure that, if the results are not yet ready, this method will stop executing and the calling code will continue. When the task is done, then at a convenient moment our code here will resume.

In order to enable this behaviour, the method must be declared async and return some Task<T> (unless it returns no value, in which case void is fine). By convention, asynchronous methods should have Async at the end of their name.

The calling code can invoke our method, and then get on with other work. When it needs the result it can await our result in turn. Hence, your entire application can become asynchronous with minimal effort compared to writing regular synchronous code.

Further reading

Learning C# 3.0 does not cover this topic. However, Head First C# does include some material on asynchronous programming and Tasks in Chapter 11 on Async, await and data contract serialization.

MSDN has good material on Asynchronous Programming and PLINQ, and you can of course find MSDN resources on the other classes we have discussed (Thread, ThreadPool, Task, Parallel) via its search feature.