I think I found a regression in apache camel-file Version 3.20.6, but I want some more opinions before I create a Jira Ticket for it as it might still be a misconfiguration on my part.

The use case is, that I have a lot files, that are dropped into a folder and I want camel-file to pick them up as fast as possible.

Until now, this could be achieved by setting the parameter readLockMinAge=1s, which will cause the files to be picked up with a 1s delay, but then there is no “wait time” after each file.
This has changed with version 3.20.6, as now there is a pause of ~1sec after each file (see logs below)

This is a test, that can be used to reproduce the issue

  @Test
  void test() throws Exception {
    int fileCount = 10;

    String from = "file:" + temporaryFolder.toAbsolutePath() + "?readLockMinAge=1s&readLock=changed";
    String to = "mock:to";
    camelContext.addRoutes(new RouteBuilder() {
      @Override
      public void configure() {
        from(from)
            .log("Received file ${header.CamelFileName}")
            .to(to);
      }
    });
    MockEndpoint fromEndpoint = camelContext.getEndpoint(to, MockEndpoint.class);
    fromEndpoint.expectedMessageCount(fileCount);

    StopWatch stopWatch = StopWatch.createStarted();

    for (int i = 1; i <= fileCount; i++) {
      // My method, that just creates a file with name and content in the temporary folder
      TestUtil.createTemporaryFile(temporaryFolder, "file" + i, "œ¶Μtstr¼j¶jø’)#²" + i);
    }

    MockEndpoint.assertIsSatisfied((long) (fileCount * 1.2), TimeUnit.SECONDS, fromEndpoint);

    stopWatch.stop();

    long ranFor = stopWatch.getTime(TimeUnit.SECONDS);
    int maxAllowedRunTime = fileCount / 10;
    if (ranFor > maxAllowedRunTime) {
      fail(
          "At least 10 files should be processed per second, which means this test should take a "
              + "maximum of " + maxAllowedRunTime + "s to run. Instead it ran for " + ranFor + "s");
    } else {
      log.info("At least 10 files should be processed per second, which means this test should "
              + "take a maximum of {}s to run. Which was successful as it ran for {}s",
          maxAllowedRunTime, ranFor);
    }
  }

With 3.20.5, this is the log output:

10:11:27.148 - route1: Received file file7
10:11:27.154 - route1: Received file file6
10:11:27.155 - route1: Received file file9
10:11:27.155 - route1: Received file file5
10:11:27.156 - route1: Received file file4
10:11:27.156 - route1: Received file file2
10:11:27.157 - route1: Received file file10
10:11:27.157 - route1: Received file file3
10:11:27.157 - route1: Received file file8
10:11:27.158 - route1: Received file file1

In 3.20.6, the test fails with the following log:

10:15:29.407 - route1: Received file file7
10:15:30.414 - route1: Received file file6
10:15:31.415 - route1: Received file file9
10:15:32.418 - route1: Received file file5
10:15:33.421 - route1: Received file file4
10:15:34.424 - route1: Received file file2
10:15:35.426 - route1: Received file file10
10:15:36.429 - route1: Received file file3
10:15:37.431 - route1: Received file file8
10:15:38.433 - route1: Received file file1

I’m also pretty sure, that this is the change, that introduced the issue: https://issues.apache.org/jira/browse/CAMEL-19378