• Certification Exams of Java Explaining how virtual threads work Java Certification Exams The benefits of cloud architecture
• Posted By

Explaining how virtual threads work 2 – Concurrency – Virtual Threads, Structured Concurrency

Capturing virtual threads

So far we learned that a virtual thread is mounted by JVM to a platform thread which becomes its carrier thread. Moreover, the carrier thread runs the virtual thread until it hit a blocking (I/O) operation. At that point, the virtual thread is unmounted from the carrier thread and it will be rescheduled after the blocking (I/O) operation is done.While this scenario is true for most of the blocking operations resulting in unmounting the virtual threads and freeing the platform thread (and the underlying OS thread), there are a few exceptional cases when the virtual threads are not unmounted. There are two main causes for this behavior:

Limitations on OS (for instance, a significant number of filesystem operations)

Limitations on JDK (for instance, Object#wait())

When the virtual thread cannot be unmounted from its carrier thread it means that the carrier thread and the underlying OS thread are blocked. This may affect the scalability of the application, so, if the platform threads pool allows it, JVM can take the decision of adding one more platform thread. So, for a period of time, the number of platform threads may exceed the number of available cores.

Pinning virtual threads

There are also two other use cases when a virtual thread cannot be unmounted:

When the virtual thread runs code inside a synchronized method/block

When the virtual thread invokes a foreign function or native method (topic covered in Chapter 7)

In this scenario, we say that the virtual thread is pinned to the carrier thread. This may affect the scalability of the application, but JVM will not increase the number of platform threads. Instead of this, we should take action and refactor the synchronized blocks to ensure that the locking code is simple, clear, and short. Whenever possible prefer java.util.concurrent locks instead of synchronized blocks. If we manage to avoid long and frequent locking periods then we will not face any significant scalability issues. In future releases, the JDK team aims to eliminate the pinning inside synchronized blocks.

• Certification Exams of Java Explaining how virtual threads work Java Certification Exams Understanding cloud architecture
• Posted By

Explaining how virtual threads work – Concurrency – Virtual Threads, Structured Concurrency

213. Explaining how virtual threads work

Now that we know how to create and start a virtual thread, let’s see how it actually works.Let’s start with a meaningful diagram:

Figure 10.7 – How virtual threads works

As you can see, figure 10.7 is similar to 10.6 only that we have added a few more elements.First of all, notice that the platform threads run under a ForkJoinPool umbrella. This is a First-In-First-Out (FIFO) dedicated fork/join pool dedicated to scheduling and orchestrating the relationships between virtual threads and platform threads (a detailed coverage of Java fork/join framework is available in Java Coding Problems, First Edition, Chapter 11).

This dedicated ForkJoinPool is controlled by the JVM and it acts as the virtual thread scheduler based on a FIFO queue. Its initial capacity (number of threads) is equal to the number of available cores and it can be increased up to 256. The default virtual thread scheduler is implemented in the java.lang.VirtualThread class:

private static ForkJoinPool createDefaultScheduler() {…}

Do not confuse this ForkJoinPool with the one used for parallel streams (Common Fork Join Pool – ForkJoinPool.commonPool()).

Between the virtual threads and the platform threads, there is a one-to-many association. Nevertheless, the JVM schedules virtual threads to run on platform threads in such a way that only one virtual thread run on a platform thread at a time. When the JVM assigns a virtual thread to a platform thread, the so-called stack chunk object of the virtual thread is copied from the heap memory on the platform thread. If the code running on a virtual thread encounters a blocking (I/O) operation that should be handled by the JVM then the virtual thread is released by copying its stack chunk object back into the heap (this operation of copying the stack chunk between the heap memory and platform thread is the cost of blocking a virtual thread – this is much cheaper than blocking a platform thread). Meanwhile, the platform thread can run other virtual threads. When the blocking (I/O) of the released virtual thread is done, JVM rescheduled the virtual thread for execution on a platform thread. This can be the same platform thread or another one.

The operation of assigning a virtual thread to a platform thread is called mounting. The operation of unassigning a virtual thread from the platform thread is called unmounting. The platform thread running the assigned virtual thread is called a carrier thread.

Let’s have an example that reveals how the virtual threads are mounted:

private static final int NUMBER_OF_TASKS
  = Runtime.getRuntime().availableProcessors();
Runnable taskr = () ->
  logger.info(Thread.currentThread().toString());      
try (ExecutorService executor
     = Executors.newVirtualThreadPerTaskExecutor()) {
  for (int i = 0; i < NUMBER_OF_TASKS + 1; i++) {
    executor.submit(taskr);
  }
}

In this snippet of code, we create a number of virtual threads equal to the number of available cores + 1. On my machine, I have 8 cores (so, 8 carriers) and each of them carries a virtual thread. Since we have + 1, a carrier will work twice. The output reveals this scenario (check out the workers, here worker-8 run virtual threads #30 and #31):

VirtualThread[#25]/runnable@ForkJoinPool-1-worker-3
VirtualThread[#30]/runnable@ForkJoinPool-1-worker-8
VirtualThread[#28]/runnable@ForkJoinPool-1-worker-6
VirtualThread[#22]/runnable@ForkJoinPool-1-worker-1
VirtualThread[#24]/runnable@ForkJoinPool-1-worker-2
VirtualThread[#29]/runnable@ForkJoinPool-1-worker-7
VirtualThread[#26]/runnable@ForkJoinPool-1-worker-4
VirtualThread[#27]/runnable@ForkJoinPool-1-worker-5
VirtualThread[#31]/runnable@ForkJoinPool-1-worker-8

But, we can configure the ForkJoinPool via three system properties as follows:

jdk.virtualThreadScheduler.parallelism – number of CPU cores

jdk.virtualThreadScheduler.maxPoolSize – maximum pool size (256)

jdk.virtualThreadScheduler.minRunnable – minimum number of running threads (half the pool size)

In a subsequent problem, we will use these properties to better shape virtual thread context switching (mounting/unmounting) details.