Asynchronous Programming

Concurrency is an enormous topic, and entire books have been written about it. This section will be just a short primer. If you want to read more, then the Oracle documentation on concurrency is an excellent reference.

Threads

You will have plenty of experience of computers performing multiple tasks at once. Running the web browser in which you’re reading this article doesn’t stop your email client checking for new mail. The Windows operating system handles this via multiple independent processes, roughly one per application. The physical CPU inside your computer doesn’t necessarily execute multiple processes at the same time, but short slices of time are given to each process in turn and this happens fast enough to give the impression of simultaneous execution. Different processes are largely isolated from each other, so your web browser and email client can’t interact with each other except via a few well-defined pathways.

A similar concept exists within each process, which can have multiple threads. Just like processes, threads share time on the CPU and hence many threads can make progress in parallel. Threads are the means by which you enable your code to do multiple things at the same time.

For example, consider the behaviour of your web browser. If you click a link, the browser starts loading the new page. But while it’s doing this you still have access to the Home button, can create new tabs, etc. This is all handled via threads. A typical desktop application has a dedicated UI thread which handles all user input, while separate threads handle any long-running processing (such as loading a web page). In this way the UI remains responsive to user input at all times, however much background activity is taking place.

In Java, you create new threads via the Thread class. You can either subclass Thread directly or create an instance of the functional interface Runnable and pass that as an argument to the Thread constructor as in the following example. The thread is started via the start method.

Runnable longRunningCalculation = () -> {
    for(Input input : getInputs()) {
        doCalculation(input);
    }
};
 
Thread backgroundCalculationThread = new Thread(longRunningCalculation);
backgroundCalculationThread.start();

Runnable myRunnable = () -> {
    System.out.println(Thread.currentThread().getName());
};
Thread myThread = new Thread(myRunnable);
myThread.run();
myThread.start();

Termination

When you have a single thread in your application, it is fairly obvious when the program ends.

Threads end when their run method completes, and you can simply leave them to it. However, if you wish to wait for a thread to finish, you can call join(), which will pause the current thread until the other is finished.

If you want to signal a thread to stop prematurely, a common idiom is to have the thread periodically check a flag and stop when the flag is set to true.

class Worker implements Runnable {
    public void run() {
        try {
            // Do some work
        } catch (InterruptedException e) {
            LOG.warn("Thread was interrupted", e);
        }
    }
}

for (Input input : inputs) () {
    doSomeLengthyCalculations(input)
    if (Thread.interrupted()) {
        throw new InterruptedException();
    }
}

Concurrent Data Access

volatile and synchronized

In order to improve performance, the JVM will by default cache variables in per-thread caches. But, this means two threads may not see updates made by each other to the same variable! Declaring a field volatile tells the JVM that the field is shared between threads, and guarantees that any read of a volatile variable will return the value most recently written to that variable by any other thread.

Here’s an example from an article by Brian Goetz that was published on https://developer.ibm.com/. If a new instance of BackgroundFloobleLoader is called from the main thread and BackgroundFloobleLoader#initInBackground is called from a separate background thread, then if theFlooble isn’t marked volatile then it may not be visible on the main thread after it’s been updated.

public class BackgroundFloobleLoader {
    public volatile Flooble theFlooble;
    public void initInBackground() {
        // do lots of slow initialisation
        theFlooble = new Flooble();
    }
}

The guarantee provided by volatile, that writes from other threads will be visible, is very weak. Here’s a counter implemented using volatile variables:

class DodgyCounter {
    private volatile int count;
 
    public void increment() {
        count = count + 1;
    }
 
    public int get() {
        return count;
    }
}

The line count = count + 1 is implemented internally as two operations:

int temp = count + 1;
count = temp;

If the value of count is 1 and two threads call increment at the same time then what we probably want to happen is:

But every so often, the threads will interleave:

The solution is to synchronise access to the relevant part of the code. In Java, the synchronized keyword is used to prevent more than one thread from entering the same block of code at once.

class Counter {
    private volatile int count;
     
    public synchronized void increment() {
        count = count + 1;
    }
     
    public synchronized int get() {
        return count;
    }
}

Adding synchronized means that no two threads are allowed to enter either the increment or get methods of the same instance of Counter while any other thread is in either the increment or get methods of that instance.

As well as synchronising access to methods, it is also possible to synchronise access to small blocks of code using synchronized statements, which you can read more about here.

When a thread encounters synchronized it will pause and wait for the lock, potentially causing the enclosing method to take a long time.

Atomic Variables and Concurrent Collections

The risk of locking problems makes it very hard to write safe multithreaded code using volatile and synchronized. You need to think carefully about how different threads can interact with each other, and there are always a lot of corner cases to think about. Java provides a number of higher-level concurrency objects for common use-cases. These are divided into:

• Atomic variables, such as AtomicInteger, AtomicBoolean, and AtomicReference<V>, which wrap single variables.
• Concurrent collections, for example BlockingQueue and ConcurrentMap, which provide concurrent versions of classes from the Collections API.

For example, rather than writing your own counter as in the previous example, you could use AtomicInteger which you can call from multiple threads without needing to manage your own locking.

Executors

One challenge with creating threads is managing them all. In particular, creating threads is quite expensive, and creating a large number of short-lived threads can quickly blow up.

Executors are typically implemented as a managed set of threads (commonly called a thread pool), which will be reused as needed throughout the lifetime of your application. Rather than creating a thread, you ask the executor to execute a method; it will do so as soon as possible (normally straight away, but if there are insufficient threads in the pool it will queue up the new activity until there’s space). Once your code is complete, the thread is available for the thread pool to allocate another activity to.

As you can see it’s pretty easy to spin up a thread pool and send it some work:

ExecutorService pool = Executors.newFixedThreadPool(poolSize);
pool.execute(() -> {
    // Do some work
});

For a full example see the Executor Service documentation. In particular, note that a thread pool should always be shut down properly.

Scheduled Tasks

For regular or one-off scheduled tasks, use a ScheduledExecutorService. For example, this provides an easy way to perform monitoring tasks, without interfering with your main application logic.

Returning Values

None of the examples you’ve done so far handle a fairly common case with asynchronous programming, and that is handling the result. A common example is when a UI wants to display a collection of images but not freeze the screen while it retrieves the images.

The problem is that the background thread may not have access to the UI components, so the background thread needs to return the image to the UI thread. However, the background thread can’t directly call methods on the UI thread. In pseudocode the problem can be written as follows:

UI Thread
---------
function loadPage()
    Create an image getter thread for http://myimage.com/img123
    Start the thread
end function
 
 
Image getter thread
-------------------
function run(imageUrl)
    Go to the internet, get the image
    Wait for a response
    Send the image to the UI thread <-- This line of code is impossible
end function

Other languages, for example C# and JavaScript, get around this problem using a structure that’s becoming more common called async/await. However, Java 8 doesn’t have that. Instead, it uses a callback which still works but is a slightly older way of doing things. In this way, when you start the thread you wouldn’t just pass in the url of the image you want to get but would also pass in the name of a function that you want to be called when the result is available. So the pseudocode would change as follows.

UI Thread
---------
function loadPage()
    Create an image getter thread for http://myimage.com/img123
    Tell the image getter thread to use handleResult(image) when a result is available
    Start the thread
end function
 
function handleResult(image)
    Display image
end function
 
 
Image getter thread
-------------------
function run(imageUrl)
    Go to the internet, get the image
    Wait for a response
    Send the image to the callback function
end function

This can be done using a CompletableFuture.

Further Reading

Chapter 11 of Effective Java covers concurrency. The first three items in this chapter are probably the most relevant for this topic.

From Concurrent to Parallel

While Java doesn’t have the async/await structure that other languages do, it does have streams which other languages doesn’t. There’s a feature called Parallel Streams which can potentially make a massive difference to the performance of a piece of code, but it uses multithreading to do it and consequently has similar problems of volatile variables and deadlock that we’ve seen already. Have a look at this TED Talk video to learn more. It’s quite long but very interesting.