Recently I noticed that in our server logs there were quite a few of Caused by: io.vertx.core.http.HttpClosedException: Connection was closed errors, the scenario was service A made an http API call to service B and finally A threw this HttpClosedException. This API call was actually going through AWS internal ALB (A -> AWS ALB -> B), I also searched ALB log, but could not even find any traffic going into service B.

We were using vert.x to implement our serviced. So I then reviewed our code implementation of HttpClient, it was implemented as a Singleton which was encapsulated into a shared library, every service project would make reference to this shared library and use the Singleton HttpClient instance for API calls. The implementation was something similar to the following:

public class MyHttpClient implements HttpClient {
  private HttpClient httpClient;
  private Context context;
  private static volatile MyHttpClient vertxHttpClient;

  private MyHttpClient(Vertx vertx) {
    WorkerVerticle workerVerticle = new WorkerVerticle();
    vertx.deployVerticle(workerVerticle,
      new DeploymentOptions().setWorker(true).setWorkerPoolSize(1));
    while (workerVerticle.getContext() == null) {
      try {
        Thread.sleep(1000);
      } catch (InterruptedException e) {
        // Ignore the exception and continue wait for another 1s
      }
    }
    workerVerticle.getContext().runOnContext(e -> {
      final HttpClientOptions options = new HttpClientOptions()
        .setSsl(true)
        .setIdleTimeout(30)
        .setIdleTimeoutUnit(TimeUnit.SECONDS)
        .setKeepAliveTimeout(30)
        .setHttp2KeepAliveTimeout(30)
        .setConnectTimeout(30000)
        .setMaxPoolSize(32)
        .setHttp2MaxPoolSize(32);

      httpClient = vertx.createHttpClient(options);
    });
  }

  public static MyHttpClient getInstance() {
    MyHttpClient localRef = vertxHttpClient;
    if (localRef == null) {
      synchronized (MyHttpClient.class) {
        localRef = vertxHttpClient;
        if (localRef == null) {
          vertxHttpClient = localRef = new MyHttpClient(Vertx.currentContext().owner());
        }
      }
    }
    return localRef;
  }
}

And for each of the AbstractVerticle in each service, we used HttpClient like below:

public class MainVerticle extends AbstractVerticle {
  @Override
  public void start(Promise<Void> startPromise) throws Exception {
    // ...
    this.httpClient = MyHttpClient.getInstance();
    // ...
  }
}

So, questions:

• Is it safe or good practice to use Singleton instance of HttpClient in vert.x?
• According to the official documentation, HttpClient can be shared between verticles, but the creation of the HttpClient should be outside of any of the verticles. So, in our implementation, we intended to reuse the HttpClient instance across multiple verticles to save the performance, but I am not sure if the above implementation of the Singleton pattern is correct
• Any other suggestion on how we could implement and use singleton of HttpClient in vert.x in a right way?

Thanks.