I’ve been trying to run a Hazelcast Jet pipeline across multiple nodes to fully utilize my machine resources. However, it seems like the job is only running on one node.

Here is my pipeline code:

Pipeline pipeline = Pipeline.create();

BatchSource<String> source = SourceBuilder.batch("test-source", context -> new DataReader())
        .<String>fillBufferFn(DataReader::fillBuffer)
        .destroyFn(DataReader::destroy)
        .build();

pipeline.readFrom(source)
        .mapUsingServiceAsync(ServiceFactories.sharedService(ctx -> new AESDecryptionService("xxxxxx")), AESDecryptionService::decryptAsync).setName("AESDecryptionService")
        .writeTo(Sinks.logger());

hazelcastInstance.getJet().newJob(pipeline);

Here is the digraph DAG:

digraph DAG {
    "test-source" [localParallelism=1];
    "AESDecryptionService" [localParallelism=4];
    "loggerSink" [localParallelism=1];
    "test-source" -> "AESDecryptionService" [queueSize=4];
    "AESDecryptionService" -> "loggerSink" [queueSize=4];
}

I wonder if I might have missed some configuration or option that causes the sub-tasks in the DAG to not be distributed across multiple nodes (if I understand the official documentation correctly)?

Any insights or suggestions on how to ensure my Hazelcast Jet pipeline runs efficiently across all nodes in the cluster would be greatly appreciated!