I have a spark sql run with Spark-3.2. The sql has SMJ and join type is LeftSemi, but the right side is skewed.
The rule OptimizeSkewedJoin is not supported LeftSemi on the right side.
spark plan
I attempted to modify the method org.apache.spark.sql.execution.adaptive.OptimizeSkewedJoin#canSplitRightSide
to support right side for skewed joins.
While, the result data of sql is not correct. I want to know why not support and how to deal with it?
New contributor
SeanWang is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.