I have a multi level partitioned data as follows.
/country=aaa/province=bbb/city=ccc/part-…..
/country=aaa/province=bbb/city=ddd/part-…..
Is it possible to retrieve the partition value within the foreachPartiton method in spark. I am using Java to call the foreachPartiton method.
I could get Partition ID using this function TaskContext.getPartitionId() within the foreachPartiton method. Is it possible to get the partition value as well like country=aaa, province = bbb & city = ccc within the foreachPartiton method?
Also TaskContext.getPartitionId() method returns the integer value corresponding to the partition Id but not the partition value.