I’m encountering a ComputeError when trying to create a Polars DataFrame from the result of a TimescaleDB query. The error message is:
ComputeError: could not append value: 1.41431 of type: f64 to the builder; make sure that all rows have the same schema or consider increasing infer_schema_length it might also be that a value overflows the data-type's capacity
Here’s the relevant part of my code:
# Query result from TimescaleDB
conn = psycopg2.connect(database="the_db", user="the_user", password="the_password", host="the_host", port="the_port")
query = "SELECT * FROM the_table"
cur = conn.cursor()
cur.execute(query)
res = cur.fetchall()
# Creating DataFrame using Pandas (works fine)
df_pd = pd.DataFrame(res)
# Creating DataFrame using Polars (raises the error)
df = pl.DataFrame(res)
I expected the pl.DataFrame(res) line to work similarly to the pd.DataFrame(res) line, but it raises the mentioned error. I’ve also noticed that a similar error was discussed in a closed GitHub issue for Polars, but I am still encountering this error despite using the latest version of Polars (0.20.31).
Has anyone faced a similar issue or have any insights on how to resolve this?
I checked the types of the values in res and they seem to be consistent. I also tried other things like rounding floating-point numbers to 5 decimal places or increasing infer_schema_length
, but they didn’t work and I got the same error again.