Reading and loading data from a cosmos db data in a dataframe but there is a field which is empty like {} which i want to replace or remove because while writing the same data in parquet file,
it is giving error like datasource dont support empty schema
quoterequest_response field is having below data
result:
policy: {}
setid:null
result is StructType, policy is also structType but setid is stringType
I tried changing the schema of policy from StructType[] to StringType() but while creating the dataframe using
after that tried creating dataframe with the new_schema but I am unable to read that new_df
Mohit Bhakat is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.