I am running same queries in Spark on YARN and Spark on K8s. Both K8s & YARN refer to the same hive metastore and hdfs path. When I run the job in YRAN certain table is getting broadcasted (in join), while same is not happening in K8s. In both the environment broadcast threshold is same. Table is also same. But there is difference in plan when run on YARN vs K8s. And both places broadcast is enabled.

Why this difference in behaviour?