I know how to use Spark in Databricks to create a CSV, but it always has lots of side effects.

For example, here is my code:

file_path = “dbfs:/mnt/target_folder/file.csv”
df.write.mode("overwrite").csv(file_path, header=True)

Then what I got is

• A folder with name file.csv
• In the folder there are addtional files called _committed_xxxx, “_started_xxxx”, “_SUCCESS”
• Multiple files with part-xxxx

What I want is only a SINGLE CSV file with the name file.csv, how can I achieve this?

I tried to use pandas.to_csv function, but it’s not working on Databricks notebook, the error is

OSError: Cannot save file into a non-existent directory