I have an API naming , getVisualier , when I am hitting it multiple times in milliseconds , I am not getting any response, but when I am hitting the same api , singly ,using Replay XHR ,it is working fine every time.
The error I am geting is, Column name does not exist . Suppose if it is hit 3 times without delay with 3 different payloads, 3 or 2 will get reponse , while the other will not get response. This is random and uneven , when done again an and again . The dataset which it is handing is a csv or xlsx file , having rows more than 2000
Please see if the issue is regarding spark session or so –

this is how I have made my spark session –

from pyspark.sql import SparkSession

—————————————————————————-

class NewSparkSession:

# ------------------------------------------------------------------------

def __init__(self):
    self.base_session = (
        SparkSession.builder.master("local[*]")
        .appName("dataHudi")
        .config("spark.driver.bindAddress", "0.0.0.0")
        .config("spark.driver.host", "localhost")
        .config(
            "spark.jars.packages",
            "org.apache.hudi:hudi-spark3.3-bundle_2.12:0.13.1",
        )
        .config(
            "spark.serializer",
            "org.apache.spark.serializer.KryoSerializer",
        )
        .config(
            "spark.sql.catalog.spark_catalog",
            "org.apache.spark.sql.hudi.catalog.HoodieCatalog",
        )
        .config(
            "spark.sql.extensions",
            "org.apache.spark.sql.hudi.HoodieSparkSessionExtension",
        )
        .getOrCreate()
    )

# ------------------------------------------------------------------------

def Create_Session(self):
    # Create a new session with isolated SQL configurations
    spark_session = self.base_session
    return spark_session

I have written spark.sql() , under that I have written query . Three different queries are executed ,every time when the API is hit . From this the front end is making graphs. But I am getting the exception column name does not exist . But the same query is excuted when done Replay XHR, i.e.,
the API is hit only onece , and only one query is made .