Polars <–> Arrow <–> Databases_(x,y,z)
- Polars DataTypes (Rust version)
- Arrow DataTypes
- MySQL DataTypes
- PostgreSQL DataTypes
Is there a maintained reference on mappings between these and any possible gotchas?
e.g. if moving between Float types may drop some one of the more exotic values (like +/- Inf) or cause ‘NaN’ to map to ‘Null’. Or if some mappings strip timezone from some date types, or round.
I’m particularly interested in using Polars (via Rust & Python) as the main center of work for a lot of analysis, where it would pull in data from various stores. But I’m not sure how to tackle the type mappings that are occurring and when I need to add manual care to ensure gain or loss of the representation space is preserved.
(Also, I’d love to just feed Polars a schema from, say, MySQL and have it map to a Polars schema — I’m not sure if we’re at that stage yet.)