Relative Content

Tag Archive for apache-sparkamazon-s3pysparkaws-glue

SPARK GLUE; EMRFS vs HDFS committers for 100.000+ partitions

I have data that i heavily partition. when running historical loads for several years, it can go up to 100.000+ partitions.