Introducing structured concurrency 2 – Concurrency – Virtual Threads, Structured Concurrency

We skip the rest of the code since you can find it in the bundled code.Of course, we can implement code-answers to each of these questions via error handling, tasks abandon and abortion, ExecutorService, and so on, but this means a lot of work for the developer. Writing failsafe solutions that carefully cover all possible scenarios across multiple tasks/subtasks while tracking their progress in a concurrent environment is not an easy job. Not to mention how hard is to understand and maintain the resulting code by another developer or even the same developer after 1-2 years or even months.It is time to add some structure to this code, so let’s introduce structured concurrency (or, Project Loom).Structured concurrency relies on several pillars meant to bring lightweight concurrency in Java. The fundamental pillar or principle of structured concurrency is highlighted next.

The fundamental principle of structured concurrency: When a task has to be solved concurrently then all the threads needed to solve it are spun and rejoined in the same block of code. In other words, all these threads’ lifetime is bound to the block’s scope, so we have clear and explicit entry-exit points for each concurrent code block.

Based on this principle, the thread that initiates a concurrent context is the parent-thread or the owner-thread. All threads started by the parent-thread are children-threads or forks, so between them, these threads are siblings. Together, the parent-thread and the children-threads, define a parent-child hierarchy.Putting the structured concurrency principle in a diagram can be seen as follows:

Figure 10.2 – Parent-child hierarchy in structured concurrency

In the context of parent-child hierarchy, we have support for error/exception handling with short-circuiting, cancellation propagation, and monitoring/observability.Error/exception handling with short-circuiting: If a child-thread fails then all child-threads are canceled unless they are complete. For instance, if futureTester(1) fails, then futureTester(2) and futureTester(3) are automatically canceled.Cancellation propagation: If the parent-thread is interrupted until joining the child-threads is over then these forks (the child-threads/subtasks) are canceled automatically. For instance, if the thread executing buildTestingTeam() gets interrupted then its three forks are automatically canceled.Monitoring/observability: A thread dump reveals crystal-clear the entire parent-child hierarchy no matter how many levels have been spawned. Moreover, in structured concurrency, we take advantage of scheduling and memory management of threads.While these are purely concepts, writing code that respects and follows these concepts requires the proper API and the following awesome callout:

Figure 10.3 – Don’t reuse virtual threads

Cut this out and stick it somewhere to see it every day! So, in structured concurrency, don’t reuse virtual threads. I know what you are thinking: hey dude, threads are expensive and limited, so we have to reuse them. A quick hint: we are talking about virtual threads (massive throughput), not classical threads, but this topic is covered in the next problem.

Technical requirements – Fundamentals of Cloud Architecture

Technical requirements

To fully engage with the content in this chapter on cloud computing architecture, you should have a basic understanding of computer systems, networking concepts, and information technology.

Additionally, the following technical requirements are recommended:

  • Internet access: You should have a reliable internet connection to access online resources, references, and examples related to cloud computing.
  • A computing device: A desktop computer, laptop, tablet, or smartphone with a modern web browser is necessary to read this chapter’s content and access any online materials.
  • A web browser: The latest version of a modern web browser such as Google Chrome, Mozilla Firefox, Microsoft Edge, or Safari is recommended. This ensures compatibility and optimal viewing experience of web-based resources and interactive content.
  • Familiarity with cloud services: Some familiarity with cloud services and their basic functionalities will enhance your understanding of this chapter. This includes knowledge of cloud computing models such as Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Software-as-a-Service (SaaS).

The history of cloud computing

Cloud computing has a rich history that has evolved over several decades. The concept of cloud computing dates back to the 1960s when computer scientists at MIT and Dartmouth College proposed the idea of a “utility computing” system that would allow users to access computing resources on demand.

In the 1970s, IBM introduced virtualization technology, which allowed multiple operating systems to run on a single mainframe computer. This technology enabled companies to consolidate their IT resources and reduce costs.

In the 1990s, the development of the World Wide Web and the rise of e-commerce led to the creation of web-based applications and services. This led to the development of early cloud computing platforms such as Salesforce, which provided customer relationship management (CRM) services over the internet.

In 2002, Amazon launched its web services division, offering cloud-based infrastructure services such as storage and computing power. This was followed by the launch of Amazon Elastic Compute Cloud (EC2) in 2006, which allowed users to rent computing capacity on demand.

In 2008, Google launched its cloud computing platform, Google App Engine, which allowed developers to build and run web applications on Google’s infrastructure.

Microsoft followed suit in 2010 with the launch of Windows Azure, which provided cloud-based services for building and deploying applications.

The growth of cloud computing has been fueled by advances in virtualization technology, which allows computing resources to be shared and used more efficiently. The development of cloud-based services and infrastructure has also made it easier for businesses to scale their IT resources up or down based on demand.

Today, cloud computing has become an integral part of many businesses, offering a range of benefits such as cost savings, scalability, flexibility, and improved collaboration. Cloud computing has also enabled the development of new technologies such as serverless computing, which allows developers to build and run applications without managing servers or infrastructure.

The main idea behind cloud computing was to provide a flexible and cost-effective way for users to access computing resources on demand. In the early days of computing, businesses and organizations had to invest in their IT infrastructure, including hardware, software, and networking equipment. This was expensive and often required a large upfront investment, which made it difficult for small and medium-sized businesses to compete with larger organizations.

Cloud computing was envisioned as a way to address this challenge by providing a shared pool of computing resources that could be accessed over the internet. This allowed businesses to pay only for the resources they needed, and to scale up or down as needed to meet changing demand.

In addition to cost savings, cloud computing was also seen as a way to improve the flexibility and agility of IT operations. By providing access to a shared pool of resources, cloud computing could enable businesses to quickly deploy new applications, scale up or down as needed, and respond to changing business needs more quickly than traditional IT infrastructure.

The thought behind cloud computing was to provide a more efficient, flexible, and cost-effective way for businesses to access the computing resources they need to operate and compete in today’s fast-paced digital economy.

Understanding cloud architecture 2 – Fundamentals of Cloud Architecture
  • Cloud components: Cloud computing involves several components, such as VMs, containers, storage, networking, security, databases, and middleware. A cloud architect must have a clear understanding of each component’s capabilities and limitations to design and implement efficient and secure cloud solutions. Cloud computing encompasses various components that contribute to its functionality and infrastructure. Examples of these components include VMs, which allow you to run multiple operating systems on a single physical server, enabling efficient resource utilization. Containers, such as Docker and Kubernetes, offer lightweight, isolated environments for deploying and managing applications across different cloud environments. Storage services, such as Amazon S3 and Google Cloud Storage, provide scalable and reliable storage for data and files. Networking services, such as Amazon Virtual Private Cloud (VPC) and Azure Virtual Network, enable the creation of virtual networks to connect resources securely. Security services such as encryption, access control, and firewalls help protect data and applications. Cloud databases, such as Amazon RDS and Microsoft Azure SQL Database, provide scalable and managed database solutions. Middleware tools facilitate communication and integration between different software components and services in the cloud. These components collectively form the infrastructure and services that power cloud computing, offering organizations the flexibility, scalability, and convenience of cloud-based solutions.
  • Cloud providers: Many cloud providers offer various cloud services and tools to build and deploy cloud solutions such as AWS, Microsoft Azure, and Google Cloud Platform (GCP). A cloud architect must have a deep understanding of these providers and their services to choose the right provider and services for their project. There are several prominent cloud providers in the market, each offering a wide range of services. AWS is a leading cloud provider, offering services such as Amazon EC2 for virtual servers, Amazon S3 for scalable storage, and Amazon RDS for managed databases. Microsoft Azure provides services such as Azure Virtual Machines, Azure Blob Storage, and Azure SQL Database. GCP offers services such as Google Compute Engine, Google Cloud Storage, and Google Cloud Spanner for distributed databases. Other notable cloud providers include IBM Cloud, with services such as IBM Cloud Virtual Servers and IBM Cloud Object Storage, and Oracle Cloud, offering services such as Oracle Compute and Oracle Database Cloud. These cloud providers offer a comprehensive suite of services, including compute, storage, databases, machine learning (ML), networking, and security, enabling organizations to build, deploy, and scale applications and infrastructure in the cloud. Figure 1.5 depicts the basic cloud architecture in AWS with key services such as VPC, EC2 (Compute), DynamoDB, and others:

Figure 1.5 – Basic cloud architecture in AWS

  • Cloud security: Cloud security is a critical component of cloud architecture. A cloud architect must design and implement security measures to protect the cloud infrastructure, data, and applications from unauthorized access, data breaches, and other security threats. Cloud security is a critical aspect of cloud computing, and several providers offer robust security services and solutions. One prominent cloud security provider is Cloudflare, which offers a range of security services such as DDoS protection, web application firewalls (WAFs), and content delivery networks (CDNs) to protect against malicious attacks. Another notable provider is Palo Alto Networks, which offers cloud security solutions such as Prisma Cloud, providing visibility, compliance, and threat protection across multi-cloud environments. Microsoft Azure also provides a comprehensive set of security services, including Azure Security Center, Azure Active Directory, and Azure Sentinel, offering identity management, threat detection, and security monitoring capabilities. AWS offers services such as AWS Identity and Access Management (IAM), AWS WAF, and AWS GuardDuty to help secure cloud environments. These cloud security providers and services play a crucial role in safeguarding data, applications, and infrastructure in the cloud, ensuring confidentiality, integrity, and availability of resources.

Overall, cloud architecture involves designing and managing cloud solutions that are scalable, reliable, secure, and cost-effective. A successful cloud architect must have a strong understanding of cloud technologies, architecture principles, and business needs to design and implement efficient and effective cloud solutions. In the upcoming section, we’ll explore the significant advantages and benefits that cloud architecture offers to organizations and individuals. Cloud computing has revolutionized the way we store, access, and process data, providing numerous advantages over traditional on-premises infrastructure.

Combining StructuredTaskScope and streams 2 – Concurrency – Virtual Threads, Structured Concurrency

And, in the next figure, you can see the event recorded for ending this virtual thread:

Figure 10.12 – JFR event for ending a virtual thread

Besides the JFR CLI, you can use more powerful tools for consuming the virtual thread events such as JDK Mission Control (https://www.oracle.com/java/technologies/jdk-mission-control.html) and the well-known Advanced Management Console (https://www.oracle.com/java/technologies/advancedmanagementconsole.html).For getting a stack trace for threads that block while pinned we can set the system property jdk.tracePinnedThreads. A complete (verbose) stack trace is available via -Djdk.tracePinnedThreads=full. Or if all you need is a brief/short stack trace then rely on -Djdk.tracePinnedThreads=short.In our example, we can easily get a pinned virtual thread by marking the fetchTester() method as synchronized (remember that a virtual thread cannot be unmounted if it runs code inside a synchronized method/block):

public static synchronized String fetchTester(int id)
    throws IOException, InterruptedException {
  …
}

In this context, JFR will record a pinned virtual thread as in the following figure:

Figure 10.13 – JFR event for a pinned virtual thread

If we run the application with -Djdk.tracePinnedThreads=full then your IDE will print a detailed stack trace that starts as follows:

Thread[#26,ForkJoinPool-1-worker-1,5,CarrierThreads]    java.base/java.lang.VirtualThread$VThreadContinuation.onPinned(VirtualThread.java:183)
…

You can see the complete output by executing the bundled code. Of course, you can get a thread dump and analyze it via several other tools. You may prefer any of jstack, Java Mission Control (JMC), jvisualvm, or jcmd. For instance, we can obtain a thread dump in plain text or JSON format via jcmd as follows:

jcmd <PID> Thread.dump_to_file -format=text <file>
jcmd <PID> Thread.dump_to_file -format=json <file>

Next, let’s play with jconsole (JMX) to quickly analyze the performance of virtual threads.

Using Java Management Extensions (JMX)

Until JDK 20 (inclusive), JMX provide support for monitoring only the platform and threads. But, we can still use JMX to observe the performance brought by virtual threads in comparison with platform threads.For instance, we can use JMX to monitor platform threads at each 500 ms via the following snippet of code:

ScheduledExecutorService scheduledExecutor
      = Executors.newScheduledThreadPool(1);
scheduledExecutor.scheduleAtFixedRate(() -> {
  ThreadMXBean threadBean
    = ManagementFactory.getThreadMXBean();
  ThreadInfo[] threadInfo
    = threadBean.dumpAllThreads(false, false);
  logger.info(() -> “Platform threads: ” + threadInfo.length);
}, 500, 500, TimeUnit.MILLISECONDS);

We rely on this code in the following three scenarios.

Running 10000 tasks via cached thread pool executor

Next, let’s add a snippet of code that run 10000 tasks via newCachedThreadPool() and platform threads. We also measure the time elapsed to execute these tasks:

long start = System.currentTimeMillis();
      
try (ExecutorService executorCached
    = Executors.newCachedThreadPool()) {
  IntStream.range(0, 10_000).forEach(i -> {
    executorCached.submit(() -> {
      Thread.sleep(Duration.ofSeconds(1));
      logger.info(() -> “Task: ” + i);
      return i;
    });
  });
}
      
logger.info(() -> “Time (ms): “
  + (System.currentTimeMillis() – start));

On my machine, it took 8147 ms (8 seconds) to run these 10000 tasks using at peak 7729 platform threads. The following screenshot from jconsole (JMX) reveals this information:

Figure 10.14 – Running 10000 tasks via cached thred pool executor

Next, let’s repeat this test via a fixed thread pool.

The benefits of cloud architecture 2 – Fundamentals of Cloud Architecture

Cloud services provide a range of collaboration tools that enable teams to work together more efficiently and productively. Some of the key collaboration features provided by cloud services are as follows:

  • Real-time collaboration: Cloud services provide real-time collaboration features such as co-authoring, commenting, and chat, allowing teams to work on the same document or project simultaneously and communicate with each other in real time
  • Shared storage: Cloud services provide shared storage, making it easier for teams to access and share files and documents, regardless of their location or device
  • Version control: Cloud services offer version control features that allow teams to track changes made to documents and restore previous versions if necessary
  • Integration with other tools: Cloud services integrate with a wide range of other collaboration tools such as project management tools, instant messaging, and video conferencing, providing a seamless collaboration experience
  • Access control: Cloud services provide access control features that enable teams to control who has access to their files and documents, ensuring that sensitive data is protected
  • Mobile access: Cloud services are accessible from anywhere, on any device, making it easy for teams to collaborate even when they are not in the same location

Cloud-based collaboration tools provided by cloud architecture can help organizations improve their productivity, streamline their workflows, and foster better collaboration among their teams. In today’s fast-paced business environment, the increasing prevalence of remote work and distributed teams has elevated the significance of cloud-based collaboration. By embracing cloud services, organizations can effectively adapt to these changes and gain a competitive edge.

Integration with other tools in cloud architecture allows cloud services to seamlessly integrate with other collaboration and productivity tools used by an organization. This integration helps create a more efficient and streamlined workflow by allowing users to access all their tools and data from a single location.

Some examples of tools that can be integrated with cloud services include project management software, communication and collaboration tools, CRM systems, and email clients. Here are some benefits of integrating cloud services with other tools:

  • Improved productivity: Integration with other tools enables users to access all their data and tools in one place, reducing the need to switch between different applications and improving productivity
  • Better collaboration: Integration with collaboration tools such as instant messaging and video conferencing can improve communication and collaboration among team members.
  • Automation: Integration with other tools can enable automation of repetitive tasks, such as data entry and reporting, saving time and reducing the risk of errors
  • Data consistency: Integration with other tools can help ensure data consistency across different systems, reducing the risk of errors and improving data quality
  • Real-time updates: Integration with other tools can enable real-time updates, ensuring that all team members have access to the latest data and information

Cloud computing systems are designed to seamlessly connect and collaborate with a wide range of existing tools and technologies. This integration enables organizations to leverage their existing infrastructure, applications, and data seamlessly within the cloud environment. By integrating with other tools, cloud architecture allows for smooth data transfer, streamlined workflows, and improved interoperability between different systems. This integration capability enhances productivity, efficiency, and the overall effectiveness of cloud-based solutions by providing a unified and cohesive ecosystem for organizations to leverage their existing tools and resources alongside cloud services. Integration with other tools is an important aspect of cloud architecture because it helps organizations create a more efficient and streamlined workflow, improving productivity, collaboration, and data quality. By integrating cloud services with other tools, organizations can create a more cohesive and effective technology ecosystem that supports their business objectives.

The following section provides a concise overview of the essential guidelines for designing and implementing effective cloud architectures. It emphasizes key practices such as scalability, high availability, performance optimization, security implementation, cost optimization, automation, and monitoring.

Introducing virtual threads – Concurrency – Virtual Threads, Structured Concurrency

211. Introducing virtual threads

Java allows us to write multithreaded applications via the java.lang.Thread class. These are classical Java threads that are basically just thin wrappers of OS (kernel) threads. As you’ll see, these classical Java threads are referred to as platform threads and they are available for quite a lot of time (from JDK 1.1, as the following diagram reveals):

Figure 10.4 – JDK multithreading evolution

Next, let’s hit the road to JDK 19 virtual threads.

What’s the problem with platform (OS) threads?

OS threads are expensive in every single way, or more clearly, they are costly in time and space. Creating OS threads is a costly operation that requires a lot of stack space (around 20 megabytes) for storing their context, Java call stacks, and additional resources. Moreover, the OS thread scheduler is responsible to schedule Java threads and this is another costly operation that requires moving around a significant amount of data. This is referred to as thread context switching and it requires a lot of resources to take place.In the following figure, you can see the one-to-one relationship between a Java thread and an OS thread:

Figure 10.5 – JVM to OS threads

For decades, our multithreaded application runs in this context. This long time and experience taught us that we can create a limited number of Java threads (because of low throughput) and that we should reuse them wisely. The number of Java threads is a limiting factor that usually is exhausted before other resources such as network connections, CPU, and so on. Java doesn’t make any difference between threads that perform intensive-computational tasks (so, threads that are really exploiting the CPU) or they just wait for data (they just hang on the CPU).Let’s have a quick exercise. Let’s assume that our machine has 8 GB of memory and a single Java thread needs 20 MB. This means that we can have room for around 400 Java threads (8 GB = 8000 MB / 20 MB = 400 threads). Next, let’s assume that these threads perform I/O operations over a network. Each such I/O operation needs around 100 ms to complete, while the request preparation and response processing needs around 500 ns. So, a thread work for 1000 ns (0.001 ms) and just waits for 100 ms (100,000,000 ns) for the I/O operation to complete. This means that at 8 GB of memory, the 400 threads will use 0.4% of CPU, under 1% which is very low. We can conclude that a thread is idle for 99.99% of the time.Based on this exercise, it is quite obvious that Java threads become a bottleneck in throughput that doesn’t allow us to solicit hardware at full capacity. Of course, we can sweeten the situation a little bit by using thread pools for minimizing the costs but it still does not solve the major issues of dealing with resources. You have to go for CompletableFuture, reactive programming (for instance, Spring Mono and Flux) and so on.But, how many classical Java threads we can create? We can easily find out by running a simple snippet of code as follows:

AtomicLong counterOSThreads = new AtomicLong();
      
while (true) {
  new Thread(() -> {
    long currentOSThreadNr
      = counterOSThreads.incrementAndGet();
    System.out.println(“Thread: ” + currentOSThreadNr);              
    LockSupport.park();              
  }).start();
}

Or, if we want to taste from the new concurrent API, we can call the new Thread.ofPlatform() method as follows (OfPlatform is a sealed interface introduced in JDK 19):

AtomicLong counterOSThreads = new AtomicLong();
while (true) {
  Thread.ofPlatform().start(() -> {
    long currentOSThreadNr
      = counterOSThreads.incrementAndGet();
    System.out.println(“Thread: ” + currentOSThreadNr);              
    LockSupport.park();              
  });
}

On my machine, I got an OutOfMemoryError after around 40,000 Java threads. Depending on your OS and hardware this number may vary.The Thread.ofPlatform() method was added in JDK 19 to easily distinguish between Java threads (classical Java threads as we know them for decades – thin wrappers of OS threads) and the new kids in town, the virtual threads.

Hooking virtual threads and sync code – Concurrency – Virtual Threads, Structured Concurrency

214. Hooking virtual threads and sync code

The goal of this problem is to highlight how virtual threads interact with synchronous code. For this, we use the built-in java.util.concurrent.SynchronousQueue. This is a built-in blocking queue that allows only one thread to operate at a time. More precisely, a thread that wants to insert an element in this queue is blocked until another thread attempts to remove an element from it and vice versa. Basically, a thread cannot insert an element unless another thread attempts to remove an element.Let’s assume that a virtual thread attempts to insert in a SynchronousQueue, while a platform thread attempts to remove from this queue. In code lines, we have:

SynchronousQueue<Integer> queue = new SynchronousQueue<>();
Runnable task = () -> {
  logger.info(() -> Thread.currentThread().toString()
    + ” sleeps for 5 seconds”);
  try { Thread.sleep(Duration.ofSeconds(5)); }
    catch (InterruptedException ex) {}
  logger.info(() -> “Running “
    + Thread.currentThread().toString());
  queue.add(Integer.MAX_VALUE);
};
logger.info(“Before running the task …”);
Thread vThread = Thread.ofVirtual().start(task);
logger.info(vThread.toString());

So, the virtual thread (vThread) waits for 5 seconds before attempting to insert an element in the queue. However, it will not successfully insert an element until another thread attempts to remove an element from this queue:

logger.info(() -> Thread.currentThread().toString()
  + ” can’t take from the queue yet”);
int maxint = queue.take();                      
logger.info(() -> Thread.currentThread().toString()
  + “took from queue: ” + maxint);              
      
logger.info(vThread.toString());
logger.info(“After running the task …”);

Here, the Thread.currentThread() refers to the main thread of the application which is a platform thread not blocked by vThread. This thread successfully removes from the queue only if another thread attempts to insert (here, vThread):The output of this code looks as follows:

[09:41:59] Before running the task …
[09:42:00] VirtualThread[#22]/runnable
[09:42:00] Thread[#1,main,5,main]
           can’t take from the queue yet
[09:42:00] VirtualThread[#22]/runnable@ForkJoinPool-1-worker-1
           sleeps for 5 seconds
[09:42:05] VirtualThread[#22]/runnable@ForkJoinPool-1-worker-1
           inserts in the queue
[09:42:05] Thread[#1,main,5,main]took from queue: 2147483647
[09:42:05] VirtualThread[#22]/terminated
[09:42:05] After running the task …

The virtual thread started its execution (is in runnable state) but the main thread cannot remove from the queue until the virtual thread will insert an element, so it is blocked by the queue.take() operation:

[09:42:00] VirtualThread[#22]/runnable
[09:42:00] Thread[#1,main,5,main]
           can’t take from the queue yet

Meanwhile, the virtual thread sleeps for 5 seconds (at this time the main thread has nothing to do) and afterward, it inserts an element:

[09:42:00] VirtualThread[#22]/runnable@ForkJoinPool-1-worker-1
           sleeps for 5 seconds
[09:42:05] VirtualThread[#22]/runnable@ForkJoinPool-1-worker-1
           inserts in the queue

The virtual thread has inserted an element into the queue, so the main thread can remove this element from it:

[09:42:05] Thread[#1,main,5,main]took from queue: 2147483647

The virtual thread is also terminated:

[09:42:05] VirtualThread[#22]/terminated

So, virtual threads, platform threads, and synchronous code work as expected. In the bundled code, you can find an example where the virtual and platform threads switch their places. So, the platform thread attempts to insert and the virtual thread attempts to remove.

Exemplifying thread context switching 2 – Concurrency – Virtual Threads, Structured Concurrency

Example 2

In this example, let’s start by limiting the parallelism to 1 (is like having a single core and a single virtual thread):

System.setProperty(
  “jdk.virtualThreadScheduler.maxPoolSize”, “1”);
System.setProperty(
  “jdk.virtualThreadScheduler.maxPoolSize”, “1”);
System.setProperty(
  “jdk.virtualThreadScheduler.maxPoolSize”, “1”);

Next, let’s consider that we have a slow task (we call it slow because it sleeps for 5 seconds):

Runnable slowTask = () -> {
  logger.info(() -> Thread.currentThread().toString()
    + ” | working on something”);          
  logger.info(() -> Thread.currentThread().toString()
    + ” | break time (blocking)”);
  try { Thread.sleep(Duration.ofSeconds(5)); }
    catch (InterruptedException ex) {} // blocking          
  logger.info(() -> Thread.currentThread().toString()
    + ” | work done”);
};

And, a fast task (similar to the slow task but sleeps only 1 second):

Runnable fastTask = () -> {
  logger.info(() -> Thread.currentThread().toString()
    + ” | working on something”);          
  logger.info(() -> Thread.currentThread().toString()
    + ” | break time (blocking)”);
  try { Thread.sleep(Duration.ofSeconds(1)); }
    catch (InterruptedException ex) {} // blocking          
  logger.info(() -> Thread.currentThread().toString()
    + ” | work done”);
};

Next, we define two virtual threads to execute these two tasks as follows:

Thread st = Thread.ofVirtual()
  .name(“slow-“, 0).start(slowTask);
Thread ft = Thread.ofVirtual()
  .name(“fast-“, 0).start(fastTask);
      
st.join();
ft.join();

If we run this code then the output will be as follows:

[08:38:46] VirtualThread[#22,slow-0]/runnable
           @ForkJoinPool-1-worker-1 | working on something
[08:38:46] VirtualThread[#22,slow-0]/runnable
           @ForkJoinPool-1-worker-1 | break time (blocking)
[08:38:46] VirtualThread[#24,fast-0]/runnable
           @ForkJoinPool-1-worker-1 | working on something
[08:38:46] VirtualThread[#24,fast-0]/runnable
           @ForkJoinPool-1-worker-1 | break time (blocking)
[08:38:47] VirtualThread[#24,fast-0]/runnable
           @ForkJoinPool-1-worker-1 | work done
[08:38:51] VirtualThread[#22,slow-0]/runnable
           @ForkJoinPool-1-worker-1 | work done

If we analyze this output, we notice that the execution starts the slow task. The fast task cannot be executed since worker-1 (the only available worker) is busy executing the slow task.

[08:38:46] VirtualThread[#22,slow-0]/runnable
           @ForkJoinPool-1-worker-1 | working on something

Worker-1 executes the slow task until this task hits the sleeping operation. Since this is a blocking operation, the corresponding virtual thread (#22) is unmounted from worker-1.

[08:38:46] VirtualThread[#22,slow-0]/runnable
           @ForkJoinPool-1-worker-1 | break time (blocking)

JVM takes advantage of the fact that worker-1 is available and pushes for the execution of the fast task.

[08:38:46] VirtualThread[#24,fast-0]/runnable
           @ForkJoinPool-1-worker-1 | working on something

The fast task also hits a sleeping operation and its virtual thread (#24) is unmounted.

[08:38:46] VirtualThread[#24,fast-0]/runnable
           @ForkJoinPool-1-worker-1 | break time (blocking)

But, the fast task sleeps only for 1 second, so its blocking operation is over before the slow task blocking operation which is still sleeping. So, the JVM can schedule the fast task for execution again, and worker-1 is ready to accept it.

[08:38:47] VirtualThread[#24,fast-0]/runnable
           @ForkJoinPool-1-worker-1 | work done

At this moment, the fast task is done and worker-1 is free. But, the slow task is still sleeping. After these long 5 seconds, the JVM schedules the slow task for execution and worker-1 is there to take it.

[08:38:51] VirtualThread[#22,slow-0]/runnable
           @ForkJoinPool-1-worker-1 | work done

Done!

Example 3

This example is just a slight modification of Example 2. This time, let’s consider that the slow task contains a non-blocking operation that runs forever. In this case, this operation is simulated via an infinite loop:

Runnable slowTask = () -> {
  logger.info(() -> Thread.currentThread().toString()
    + ” | working on something”);          
  logger.info(() -> Thread.currentThread().toString()
    + ” | break time (non-blocking)”);
  while(dummyTrue()) {} // non-blocking          
  logger.info(() -> Thread.currentThread().toString()
    + ” | work done”);
};
static boolean dummyTrue() { return true; }

We have a single worker (worker-1) and the fast task is the same as in Example 2. If we run this code, the execution hangs on as follows:

[09:02:45] VirtualThread[#22,slow-0]/runnable
           @ForkJoinPool-1-worker-1 | working on something
[09:02:45] VirtualThread[#22,slow-0]/runnable
           @ForkJoinPool-1-worker-1 | break time(non-blocking)
// hang on

The execution hangs on because the infinite loop is not seen as a blocking operation. In other words, the virtual thread of the slow task (#22) is never unmounted. Since there is a single worker, the JVM cannot push for the execution of the fast task.If we increase the parallelism from 1 to 2 then the fast task will be successfully executed by worker-2, while worker-1 (executing the slow task) will simply hang on to a partial execution. We can avoid such situations by relying on a timeout join such as join(Duration duration). This way, after the given timeout, the slow task will be automatically interrupted. So, pay attention to such scenarios.

Exemplifying thread context switching – Concurrency – Virtual Threads, Structured Concurrency

215. Exemplifying thread context switching

Remember that a virtual thread is mounted on a platform thread and it is executed by that platform thread until a blocking operation occurs. At that point, the virtual thread is unmounted from the platform thread and it will be rescheduled for execution by the JVM later on after the blocking operation is done. This means that, during its lifetime, a virtual thread can be mounted multiple times on different or the same platform thread.In this problem, let’s write several snippets of code meant to capture and exemplify this behavior.

Example 1

In the first example, let’s consider the following thread factory that we can use to easily switch between the platform and virtual threads:

static class SimpleThreadFactory implements ThreadFactory {
  @Override
  public Thread newThread(Runnable r) {
  return new Thread(r);                      // classic thread
  // return Thread.ofVirtual().unstarted(r); // virtual thread
  }
}

Next, we try to execute the following task via 10 platform threads:

public static void doSomething(int index) {
  logger.info(() -> index + ” “
    + Thread.currentThread().toString());
  try { Thread.sleep(Duration.ofSeconds(3)); }
    catch (InterruptedException ex) {}
  logger.info(() -> index + ” “
    + Thread.currentThread().toString());
}

Between the two logging lines, we have a blocking operation (sleep()). Next, we rely on newThreadPerTaskExecutor() to submit 10 tasks that should log their details, sleep for 3 seconds, and log again:

try (ExecutorService executor = 
    Executors.newThreadPerTaskExecutor(
      new SimpleThreadFactory())) {
  for (int i = 0; i < MAX_THREADS; i++) {
    int index = i;
    executor.submit(() -> doSomething(index));
  }
}

Running this code with platform threads reveals the following side-to-side output:

Figure 10.8 – Using platform threads

By carefully inspecting this figure, we notice that there is a fixed association between these numbers. For instance, the task with id 5 is executed by Thread-5, task 3 by Thread-3, and so on. After sleeping (blocking operation), these numbers are unchanged. This means that while the tasks are sleeping the threads are just hanging and waiting there. They have no work to do.Let’s switch from platform threads to virtual threads and let’s run again:

@Override
public Thread newThread(Runnable r) {
  // return new Thread(r);                // classic thread
  return Thread.ofVirtual().unstarted(r); // virtual thread
}

Now, the output is resumed in the following figure:

Figure 10.9 – Using virtual threads

This time, we see that things are more dynamic. For instance, the task with id 5 is started by a virtual thread executed by worker-6 but is finished by worker-4. The task with id 3 is started by a virtual thread executed by worker-4 but is finished by worker-6. This means that, while a task is sleeping (blocking operation), the corresponding virtual thread is unmounted and its worker can serve other virtual threads. When the sleeping is over, the JVM schedules the virtual thread for execution and is mounted on another (it could be the same as well) worker. This is also referred to as thread context switching.

Introducing scope object (StructuredTaskScope) 2 – Concurrency – Virtual Threads, Structured Concurrency

First, we create a StructuredTaskScope in a try-with-resources pattern. StructuredTaskScope implements AutoCloseable:

try (StructuredTaskScope scope
      = new StructuredTaskScope<String>()) {
    …
}

The scope is a wrapper for the virtual threads’ lifetime. We use the scope to fork as many virtual threads (subtasks) as needed via the fork(Callable task) method. Here, we fork only one virtual thread and get back a Subtask (forking is a non-blocking operation):

Subtask<String> subtask = scope.fork(() -> fetchTester(1));

Next, we have to call the join() method (or joinUntil(Instant deadline)). This method waits for all threads (all Subtask instances) forked from this scope (or, all threads that have been submitted to this scope) to complete, so it is a blocking operation. A scope should block only for waiting its subtasks to complete, and this is happening via join() or joinUntil().

scope.join();          

When the execution passes this line, we know that all threads (all forked Subtask) forked from this scope are complete with a result or an exception (each subtask run indenpendently, therefore each of them can complete with a result or an exception). Here, we call the non-blocking get() mehod to get the result but pay attention that calling get() for a task that did not complete will rise an exception as IllegalStateException(“Owner did not join after forking subtask”):

String result = subtask.get();

On the other hand, we can obtain the exception of a failed task via exception(). However, if we call exception() for a subtask (Subtask) that is completed with a result then we will get back an exception as IllegalStateException(“Subtask not completed or did not complete with exception”). So, if you are not sure that your task(s) is always complete with a result or an exception, it is better to call get() or exception() only after you test the state of the corresponding Subtask. A state of SUCCESS will safely allow you to call get(), while a state of FAILED will safely allow you to call exception(). So, in our case, we may prefer it this way:

String result = “”;
if (subtask.state().equals(Subtask.State.SUCCESS)) {
  result = subtask.get();
}

Beside Subtask.State.SUCCESS and Subtask.State.FAILED, we also have Subtask.State.UNAVAILABLE which means that the subtask is not available (for instance, if the subtask is still running then its state is UNAVAILABLE, but it could be other cause as well).That’s all!

ExecutorService vs. StructuredTaskScope

The previous code looks like the code that we write via a classical ExecutorService but, there are two big differences between these solutions. First of all, an ExecutorService holds the precious platform threads and allows us to pool them. On the other hand, a StructuredTaskScope is just a thin launcher for virtual threads that are cheap and shouldn’t be pooled. So, once we’ve done our job, a StructuredTaskScope can be destroyed and garbage collected. Second, an ExecutorService holds a single queue for all the tasks and the threads take from this queue whenever they have the chance to do it. A StructuredTaskScope relies on a fork/join pool and each virtual thread has it own wait queue. However, a virtual thread can steal a task from another queue as well. This is known as the work-stealing pattern and it was covered in Java Coding Problem, First Edition, Chapter 11.

Copyright © 2025 Concurrency - Virtual Threads, Structured Concurrency | Powered by washingtonred.org