How to sed List of into to item reader on spring batch remote partitioning
The main issue in the partitioning approach in the spring batch is integrating or sharing data between the manager(master) instance and the workers (slaves) instance.
It means, the manager instance takes partitions among eligible data and sends some info such as partition ID and metadata about partitioned data to workers. On the opposite side, workers start to split steps among themselves. On the other hand, the one step (read, process, write) that can be done by one instance, will be separated and done by some workers. Now, the metadata for starting steps by workers is so important and the way that the metadata is shared is a main concern.
Spring batch uses some tables for this issue. I mean, the partition info and other info that we need for completing batch processing, is stored in Execute-Content that persists into related batch tables as a central storage, and workers read these data from tables and use them in step stages.
For example, we put partition number, min ID, and max ID into Execute-Content and related table and fetch it during the batch partitioning process.