I have a table in BigQuery with multiple columns, one of them being JSON. Say for example, col1 is STRING, col2 is INT and col3 is JSON. I have the data in a Polars Dataframe which I am trying to write into BigQuery. Inside the Dataframe, the col3 is stored as STRING.

The write attempt is done through in two steps:

1. Create a stream variable and use df.write_parquet(stream) to have a stream pointing to the data.
2. client.load_table_from_file(stream, destination, job_config) method from the bigquery python sdk. The LoadJobConfig parameter also specifies the schema of the destination table (which has the JSON type specified).

When executing this, the error: google.api_core.exceptions.BadRequest: 400 Unsupported field type: JSON; reason: invalid, message: Unsupported field type: JSON is thrown.

How to resolve this? If I change the data type to STRING everything works, but I’d really like to keep the data type as JSON as there are existing scripts reading from the table.