Kryo serialization issue with Spark push-based shuffle
I’m running a spark job on AWS EMR 6.14 (based on Spark 3.4.1). I wanted to test the new push-based shuffle introduced in Spark 3.2.0 but it’s failing with a kryo exception when I’m enabling it.