Sometimes this code works, sometimes it doesn’t, I have this code:

new_table_name = "table_x923"

attempts = 3
for attempt in range(attempts):
    try:
        table_data_inc.write 
            .format("jdbc") 
            .option("url", jdbc_url) 
            .option("dbtable", "schema_name.random_table_name_abc123") 
            .option("user", user_sql) 
            .option("password", password_sql) 
            .option("driver", "org.postgresql.Driver") 
            .option("batchsize", 250) 
            .option("socketTimeout", "120") 
            .option("connectTimeout", "120") 
            .mode("overwrite") 
            .save()
        break
    except Exception as e:
        if attempt < attempts - 1:
            time.sleep(2 ** attempt)
            continue
        else:
            raise e

And get this error:

Py4JJavaError: An error occurred while calling o577.save.
: org.postgresql.util.PSQLException: The connection attempt failed.
    at org.postgresql.core.v3.ConnectionFactoryImpl.openConnectionImpl(ConnectionFactoryImpl.java:354)

Any recommendations assuring the persistance of the data resiliently? Thanks

I added the try/except, and it works sometimes and this is for an important data ingestion using pyspark to PostgreSQL.