Introducing virtual threads – Concurrency – Virtual Threads, Structured Concurrency

211. Introducing virtual threads

Java allows us to write multithreaded applications via the java.lang.Thread class. These are classical Java threads that are basically just thin wrappers of OS (kernel) threads. As you’ll see, these classical Java threads are referred to as platform threads and they are available for quite a lot of time (from JDK 1.1, as the following diagram reveals):

Figure 10.4 – JDK multithreading evolution

Next, let’s hit the road to JDK 19 virtual threads.

What’s the problem with platform (OS) threads?

OS threads are expensive in every single way, or more clearly, they are costly in time and space. Creating OS threads is a costly operation that requires a lot of stack space (around 20 megabytes) for storing their context, Java call stacks, and additional resources. Moreover, the OS thread scheduler is responsible to schedule Java threads and this is another costly operation that requires moving around a significant amount of data. This is referred to as thread context switching and it requires a lot of resources to take place.In the following figure, you can see the one-to-one relationship between a Java thread and an OS thread:

Figure 10.5 – JVM to OS threads

For decades, our multithreaded application runs in this context. This long time and experience taught us that we can create a limited number of Java threads (because of low throughput) and that we should reuse them wisely. The number of Java threads is a limiting factor that usually is exhausted before other resources such as network connections, CPU, and so on. Java doesn’t make any difference between threads that perform intensive-computational tasks (so, threads that are really exploiting the CPU) or they just wait for data (they just hang on the CPU).Let’s have a quick exercise. Let’s assume that our machine has 8 GB of memory and a single Java thread needs 20 MB. This means that we can have room for around 400 Java threads (8 GB = 8000 MB / 20 MB = 400 threads). Next, let’s assume that these threads perform I/O operations over a network. Each such I/O operation needs around 100 ms to complete, while the request preparation and response processing needs around 500 ns. So, a thread work for 1000 ns (0.001 ms) and just waits for 100 ms (100,000,000 ns) for the I/O operation to complete. This means that at 8 GB of memory, the 400 threads will use 0.4% of CPU, under 1% which is very low. We can conclude that a thread is idle for 99.99% of the time.Based on this exercise, it is quite obvious that Java threads become a bottleneck in throughput that doesn’t allow us to solicit hardware at full capacity. Of course, we can sweeten the situation a little bit by using thread pools for minimizing the costs but it still does not solve the major issues of dealing with resources. You have to go for CompletableFuture, reactive programming (for instance, Spring Mono and Flux) and so on.But, how many classical Java threads we can create? We can easily find out by running a simple snippet of code as follows:

AtomicLong counterOSThreads = new AtomicLong();
      
while (true) {
  new Thread(() -> {
    long currentOSThreadNr
      = counterOSThreads.incrementAndGet();
    System.out.println(“Thread: ” + currentOSThreadNr);              
    LockSupport.park();              
  }).start();
}

Or, if we want to taste from the new concurrent API, we can call the new Thread.ofPlatform() method as follows (OfPlatform is a sealed interface introduced in JDK 19):

AtomicLong counterOSThreads = new AtomicLong();
while (true) {
  Thread.ofPlatform().start(() -> {
    long currentOSThreadNr
      = counterOSThreads.incrementAndGet();
    System.out.println(“Thread: ” + currentOSThreadNr);              
    LockSupport.park();              
  });
}

On my machine, I got an OutOfMemoryError after around 40,000 Java threads. Depending on your OS and hardware this number may vary.The Thread.ofPlatform() method was added in JDK 19 to easily distinguish between Java threads (classical Java threads as we know them for decades – thin wrappers of OS threads) and the new kids in town, the virtual threads.