I’m trying to run some functions for my ETL pipelines and log them in the process, the issue is that when I call my function my log message is instantly shown, I want to display it during the execution.

This is an example of my func:

def ReadDataframe(path=None, format="parquet") -> DataFrame:
  

  if not isinstance(path, str) or not path:
    raise ValueError(logging.error("** LOAD ** : The 'path' parameter must be a non-empty string."))

  try:
    df = spark.read.format(format).load(path)
    logging.info("** LOAD ** : Data from path %s was successfully loaded into a DataFrame", path)
    return df
    
  except AnalysisException as e:
    logging.error("** LOAD ** : Error loading data into DataFrame. AnalysisException: %s", e)
    raise
  
  except Exception as e:
    logging.error("** LOAD ** : An unexpected error occurred while loading data into DataFrame: %s", e)
    raise

But when I call the function:

df = ReadDataframe(path=path)

The log message is instantly shown:

“** LOAD ** : Data from path %s was successfully loaded into a DataFrame”

I need to display the log when I execute the lazy code:
df.show()

Is there any way I can achieve this? Or any other approach to my problem?