I want to process millions of accounts. I want to read 1K accounts (pageSize) per DB read, call a REST service in processor with 20 account id’s per call, then write to network storage (recommends fewer writes with lots of data per write) in writer.
I am thinking two options:-

a) Set pageSize and chunkSize to 1000. In processor, accumulate id’s and as soon as I have 20, call service. Processor then writes output from those 1000 id’s in one DB call.

b) Implement a custom reader, override read() to somehow return list of 20 id’s.

Thoughts, and if anyone has any samples, will appreciate..