i have a python script which writes api resopnse data to csv file and it handles bulk of data (around 50-70k).

It skips some lines (roughly 5-10 lines) in csv file.
csv file is new which means it will be created when script is running. (sciprt will be ran once)

Can anyone help me on this?

Here’s my code snip

def writeCSV(param1, api_code, api_payload):

    file_name = config_function("csv_file","config.yaml")

    logger.info(f"Attempting to write data: {param1}")

    if not os.path.exists(file_name):
        logger.warning(f"The csv file {file_name} dose not exist. It will be created.")
    elif not os.access(file_name, os.W_OK):
        logger.error(f"The csv file {file_name} is not accessible. Check file permissions.")
        return # exit if the file cannot be written to

    try:
        with open (file_name, mode="a", newline="") as file:
            logger.info(f"opened file {file_name} successfully in append mode.")

            writer = csv.writer(file)

        # write data
            writer.writerow(
                [
                    param1,
                    api_code if api_code is not None else "",
                    api_payload if api_payload is not None else "",
                ]
            )
        logger.info(f"row inserted successfully: {param1}")
    except Exception as e:
        logger.error(f"failed to insert row for: {param1}. Error: {str(e)}")

    # confirm completion of the function
    logger.info(f"finished processing data for: {param1}")