Our application is based on Spring Boot 3.2 and runs on JDK 21. We’ve gone all-in on virtual threads, and can’t sing their praises enough. Tomcat handles all requests with a virtual thread pool, our usage of @Async uses a virtual thread pool, etc.

My trouble is that thread contention issues can still arise if one of our endpoints calls code that subsequently runs on a non-virtual thread pool. One example is a request handler that calls code surrounded by a CircuitBreaker.run() method. At least in terms of Reliance4J, .run() runs the Supplier on its internal thread pool, which is by default limited to the number of CPU cores. This means that I could have 100 instances of my request handler running on virtual threads all competing for the four platform threads available in Reliance4J’s pool.

The obvious answer would be to increase the thread pool used by Reliance4J. This would help, but may be the best/only solution available. My concern is that it still does not scale in the same way that the system would if it were virtual threads all the way down. This is not ideal and it really cancels out the benefit that virtual threads were giving us.

Another option would be to explore replacing Reliance4J’s thread pool, but it appears that there are other Loom-friendliness issues (use of synchronized keyword, etc) that would cause trouble (i.e. thread pinning) in the wrapper code around their thread pool. It’s also apparent that the Reliance4J team has not yet embraced virtual threads and this feels a little bit like an abuse of their tool. It also creates some maintainability concerns as Reliance4J updates its code in the future.

Let me whittle this down to two questions:

Does a circuit breaker implementation that plays nice with virtual threads exist? Are any of the existing implementations working towards Loom-friendliness?

What solution am I not considering? I imagine that I could create my own simple circuit breaker which could launch a runnable on a separate virtual thread. This would work, provided it was possible to preempt a virtual thread. I’m not sure that’s possible. Is that even reasonable?

FWIW: I’m sure someone will be quick to point out that because my request handler threads are so lengthy virtual threads don’t provide much of an advantage over a more traditional thread pool. In my situation, the request handler usually can find the information it needs in a cache and only launches the circuit breaker-protected code if that lookup fails. In this way, I’ve got a mix of short and long-lived threads (single-digit milliseconds to tens of seconds) and it does provide a substantial advantage.