I am trying to read a nested JSON downloaded from: text
When running the code:
<code># Read the JSON file with specified options
df = spark.read.format("json")
.option("inferschema", "true")
.option("mode", "permissive")
.load("/FileStore/tables/nested-2.json")
# Show the DataFrame
df.show()
</code>
<code># Read the JSON file with specified options
df = spark.read.format("json")
.option("inferschema", "true")
.option("mode", "permissive")
.load("/FileStore/tables/nested-2.json")
# Show the DataFrame
df.show()
</code>
# Read the JSON file with specified options
df = spark.read.format("json")
.option("inferschema", "true")
.option("mode", "permissive")
.load("/FileStore/tables/nested-2.json")
# Show the DataFrame
df.show()
I am getting the error : AnalysisException: Since Spark 2.3, the queries from raw JSON/CSV files are disallowed when the
referenced columns only include the internal corrupt record column.
While this is happening for file5.JSON, this did not happen for file4.JSON on the link provided. Can anyone point out the issue.