I know that using a for loop in Spark is not a good practice as it runs sequentially, not in parallel. However, as I’m new to working with Spark, I haven’t found a better solution yet.