We are trying to partition by dataset dynamically on a column value
dataset
.repartition(1)
.write().format("delta")
.partitionBy(("sysPartitionKey"))
.save(clientId + "/" + folderName + "/FileName");
If in case dataset is empty, it does not produce any empty delta parquet files in folder. If we remove partition by, it works fine.
Can you let us know how we can produce empty delta parquet files if dataset is empty.
2