I’m attempting to insert some data into Google Cloud SQL (running postgres) and it takes forever.
It takes roughly 1 minute to insert 10 rows.

I am not doing anything fancy, just initializing the engine:

def engine():
    instance_connection_name = env(
        "INSTANCE_CONNECTION_NAME"
    )  
    db_user = env("DB_USER")  # e.g. 'my-db-user'
    db_pass = env("DB_PASS")  # e.g. 'my-db-password'
    db_name = env("DB_NAME")  # e.g. 'my-database'

    ip_type = IPTypes.PRIVATE if os.environ.get("PRIVATE_IP") else IPTypes.PUBLIC

    GCP = env("GCP", False)
    if GCP:
        connector = Connector()

        def getconn() -> pg8000.dbapi.Connection:
            conn: pg8000.dbapi.Connection = connector.connect(
                instance_connection_name,
                "pg8000",
                user=db_user,
                password=db_pass,
                db=db_name,
                ip_type=ip_type,
            )
            return conn

        return sqlalchemy.create_engine(
            "postgresql+pg8000://",
            creator=getconn,
            # ...
        )

Then all I am doing is:

 e = engine()
 df.to_sql('consumption', e, if_exists='append', index=False)

With my 10 rows, 10 columns dataframe. And it takes 1 minute.

I am using a shared Cloud SQL instance so I though maybe there are some availability problems but it actually always takes that amount of time. Though if I insert the same data manually through a CSV file and psql , the insert is done instantly.

Does anyone have any insights ?