I have a dataframe df1 with all the columns are in string(100+ Columns), now i want to cast it to appropriate type with inferschema
Like, for example what we do if we have a csv file and we want the inferschame
val df = spark.read.format(“csv”).option(“header”, “true”).option(“inferSchema”, “true”).load(csvFilePath)
Expecting something like,
val df = df1.read.option(“inferSchema”, “true”)
A new df with appropriate datatype,
Note that we can’t change the df1
Need the solution that will work on databricks
Sam is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.