Below code i am giving 1000 batch size to insert query into Azure sql db its taking so much time 20 min to process 3 lakh data Is there any way to reduce the time ? also tried increasing batch size getting deadlock for 2000 batch count
LOGGER.info(“Persisting containers {}”, containers.size());
Integer batchSize = config.getContainerDbInsertBatchSize();
for (List<ContainerDE> subList : Iterables.partition(containers, batchSize)) {
List<MapSqlParameterSource> parameters = new ArrayList<>(batchSize);
for (ContainerDE containerDE : subList) {
parameters.add(convertToMap(containerDE));
}
try {
namedJdbcTemplate.batchUpdate(DatabaseQueryConstants.CREATE_CONTAINER, parameters.toArray(new SqlParameterSource[0]));
} catch (DuplicateKeyException duplicateKeyException) {
LOGGER.warn("Duplicate key exception occurred while persisting containers ", duplicateKeyException);
}
catch (Exception e) {
LOGGER.error("Exception while persisting data for containers", e);
throw new InfraException("Exception while persisting data for containers", e);
}
Tried with batching