Explaining how virtual threads work – Concurrency – Virtual Threads, Structured Concurrency

213. Explaining how virtual threads work

Now that we know how to create and start a virtual thread, let’s see how it actually works.Let’s start with a meaningful diagram:

Figure 10.7 – How virtual threads works

As you can see, figure 10.7 is similar to 10.6 only that we have added a few more elements.First of all, notice that the platform threads run under a ForkJoinPool umbrella. This is a First-In-First-Out (FIFO) dedicated fork/join pool dedicated to scheduling and orchestrating the relationships between virtual threads and platform threads (a detailed coverage of Java fork/join framework is available in Java Coding Problems, First Edition, Chapter 11).

This dedicated ForkJoinPool is controlled by the JVM and it acts as the virtual thread scheduler based on a FIFO queue. Its initial capacity (number of threads) is equal to the number of available cores and it can be increased up to 256. The default virtual thread scheduler is implemented in the java.lang.VirtualThread class:

private static ForkJoinPool createDefaultScheduler() {…}

Do not confuse this ForkJoinPool with the one used for parallel streams (Common Fork Join Pool – ForkJoinPool.commonPool()).

Between the virtual threads and the platform threads, there is a one-to-many association. Nevertheless, the JVM schedules virtual threads to run on platform threads in such a way that only one virtual thread run on a platform thread at a time. When the JVM assigns a virtual thread to a platform thread, the so-called stack chunk object of the virtual thread is copied from the heap memory on the platform thread. If the code running on a virtual thread encounters a blocking (I/O) operation that should be handled by the JVM then the virtual thread is released by copying its stack chunk object back into the heap (this operation of copying the stack chunk between the heap memory and platform thread is the cost of blocking a virtual thread – this is much cheaper than blocking a platform thread). Meanwhile, the platform thread can run other virtual threads. When the blocking (I/O) of the released virtual thread is done, JVM rescheduled the virtual thread for execution on a platform thread. This can be the same platform thread or another one.

The operation of assigning a virtual thread to a platform thread is called mounting. The operation of unassigning a virtual thread from the platform thread is called unmounting. The platform thread running the assigned virtual thread is called a carrier thread.

Let’s have an example that reveals how the virtual threads are mounted:

private static final int NUMBER_OF_TASKS
  = Runtime.getRuntime().availableProcessors();
Runnable taskr = () ->
  logger.info(Thread.currentThread().toString());      
try (ExecutorService executor
     = Executors.newVirtualThreadPerTaskExecutor()) {
  for (int i = 0; i < NUMBER_OF_TASKS + 1; i++) {
    executor.submit(taskr);
  }
}

In this snippet of code, we create a number of virtual threads equal to the number of available cores + 1. On my machine, I have 8 cores (so, 8 carriers) and each of them carries a virtual thread. Since we have + 1, a carrier will work twice. The output reveals this scenario (check out the workers, here worker-8 run virtual threads #30 and #31):

VirtualThread[#25]/runnable@ForkJoinPool-1-worker-3
VirtualThread[#30]/runnable@ForkJoinPool-1-worker-8
VirtualThread[#28]/runnable@ForkJoinPool-1-worker-6
VirtualThread[#22]/runnable@ForkJoinPool-1-worker-1
VirtualThread[#24]/runnable@ForkJoinPool-1-worker-2
VirtualThread[#29]/runnable@ForkJoinPool-1-worker-7
VirtualThread[#26]/runnable@ForkJoinPool-1-worker-4
VirtualThread[#27]/runnable@ForkJoinPool-1-worker-5
VirtualThread[#31]/runnable@ForkJoinPool-1-worker-8

But, we can configure the ForkJoinPool via three system properties as follows:

jdk.virtualThreadScheduler.parallelism – number of CPU cores

jdk.virtualThreadScheduler.maxPoolSize – maximum pool size (256)

jdk.virtualThreadScheduler.minRunnable – minimum number of running threads (half the pool size)

In a subsequent problem, we will use these properties to better shape virtual thread context switching (mounting/unmounting) details.