I am new to apache spark and I am trying to run my first example in scala:
object DataFramesBasics extends App {
// creating a SparkSession
val spark = SparkSession.builder()
.appName("DataFrames Basics")
.config("spark.master", "local")
.getOrCreate()
// reading a DF
val firstDF: DataFrame = spark.read
.format("json")
.option("inferSchema", "true")
.load("src/main/resources/data/cars2.json")
firstDF.show()
}
The file has 406 rows, nothing too big I would say.
but when I run that code, I got:
scala: Error: org.jetbrains.jps.incremental.scala.remote.ServerException
java.lang.StackOverflowError
at scala.runtime.Statics.anyHash(Statics.java:127)
at scala.util.hashing.MurmurHash3.listHash(MurmurHash3.scala:285)
at scala.util.hashing.MurmurHash3$.seqHash(MurmurHash3.scala:355)
at scala.collection.Seq.hashCode(Seq.scala:41)
at scala.collection.Seq.hashCode$(Seq.scala:41)
at scala.collection.AbstractSeq.hashCode(Seq.scala:1190)
at scala.runtime.Statics.anyHash(Statics.java:127)
at scala.util.hashing.MurmurHash3.productHash(MurmurHash3.scala:76)
at scala.util.hashing.MurmurHash3$.productHash(MurmurHash3.scala:343)
at scala.runtime.ScalaRunTime$._hashCode(ScalaRunTime.scala:158)
at org.jetbrains.plugins.scala.compiler.data.CompilerJars.hashCode(CompilerJars.scala:16)
at scala.runtime.Statics.anyHash(Statics.java:127)
at scala.util.hashing.MurmurHash3.productHash(MurmurHash3.scala:76)
at scala.util.hashing.MurmurHash3$.productHash(MurmurHash3.scala:343)
at scala.Some.hashCode(Option.scala:618)
at scala.runtime.Statics.anyHash(Statics.java:127)
at scala.util.hashing.MurmurHash3.productHash(MurmurHash3.scala:76)
at scala.util.hashing.MurmurHash3$.productHash(MurmurHash3.scala:343)
at scala.runtime.ScalaRunTime$._hashCode(ScalaRunTime.scala:158)
at org.jetbrains.plugins.scala.compiler.data.CompilerData.hashCode(CompilerData.scala:5)
at java.base/java.util.HashMap.hash(HashMap.java:338)
at java.base/java.util.HashMap.getNode(HashMap.java:576)
at java.base/java.util.LinkedHashMap.get(LinkedHashMap.java:536)
at org.jetbrains.jps.incremental.scala.local.Cache.$anonfun$createPromise$1(Cache.scala:112)
at org.jetbrains.jps.incremental.scala.local.Cache.withLock(Cache.scala:144)
at org.jetbrains.jps.incremental.scala.local.Cache.createPromise(Cache.scala:106)
at org.jetbrains.jps.incremental.scala.local.Cache.compute(Cache.scala:69)
I am running it in Intellij IDEA.
Any ideas?