I have a JdbcCursorItemReader reader that needs data from another Postgres database. The data from the Postgres database is to be used to enrich the Products with additional data. How do I arrange this so I can only read the data that is needed to enrich the data that has been read for the products. I have a fetchSize of 200 for the JdbcCursorItemReader, however the ItemProcessor interface only takes a single product. It appears that the only way is to read all the product enrichment data all at once so I can use that to look up corresponding productId from the Postgres database. Is rhere another way? How do I incorporate the product detail? I do not want to read the product detail product by product but it seems to only way.
import lombok.RequiredArgsConstructor;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.beans.factory.annotation.Qualifier;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.stereotype.Component;
import org.springframework.transaction.PlatformTransactionManager;
@Component("updateProductsStep")
@RequiredArgsConstructor
public class UpdateProductItemsStep {
@Value("${processing.chunk-size}")
private int chunkSize;
private final JobRepository jobRepository;
private final PlatformTransactionManager platformTransactionManager;
@Qualifier("productValidatorItemProcessor")
private final ProductValidatingItemProcessor productValidatorItemProcessor;
@Qualifier("productReader")
private final ProductJdbcCursorReader productJdbcCursorReader;
@Qualifier("productUpdateWriter")
private final ProductUpdateWriter productUpdateWriter;
//This is the missing part to add - This reads from PostGres
@Qualifier("productDetailReader")
private final ProductDetailReader productDetailReader;
@Bean("updateProductInitialLoadingStep")
public Step updateProductInitialStep(){
return new StepBuilder("ProductLoaderStep",jobRepository)
.<Product, Product>chunk(chunkSize,platformTransactionManager)
.reader(productJdbcCursorReader)
.processor(productValidatorItemProcessor)
.writer(productUpdateWriter)
.build();
}
}
How can I add a step to add the extra detail to the Products after they have been read