I’m using Puppeteer with Puppeteer-Cluster to run multiple browsers in parallel. I’m using Cluster.CONCURRENCY_CONTEXT for concurrency. However, I’m encountering an issue where waitForSelector often fails to wait for the desired selector when running in parallel.

sleep(ms: number) {
    return new Promise((resolve) => setTimeout(resolve, ms));
  }

I was able to remove this problem by using sleep(1000) before each event call.

const cluster = await Cluster.launch({
      concurrency: Cluster.CONCURRENCY_CONTEXT,
      maxConcurrency: 2,
      monitor: true,
      puppeteerOptions: {
        headless: false,
        executablePath: '/usr/bin/google-chrome',
        args: args,
      },
    });

await cluster.task(async ({ page, data: url }) => {
      try {
        await page.goto(url, { waitUntil: 'domcontentloaded' });
        await page.waitForSelector('#Office');
        await page.select('#Office', 'PEKING');

        await page.waitForSelector('input[type=submit][value=Next]');
        await page.click('input[type=submit][value=Next]');
        await this.sleep(1000);

        await page.waitForSelector('#CalendarId');
        await page.select('#CalendarId', '20804258');
        await this.sleep(1000);
      // another code

   } catch (error) {
        if (error instanceof Error)
          console.error(`Error processing ${url}: ${error.message}`);
        throw error; // Rethrow the error to trigger taskerror event
      }
    });

How can I reliably wait for the desired selector without using sleep in a Puppeteer-Cluster script running in parallel with Cluster.CONCURRENCY_CONTEXT?

Any help or suggestions would be greatly appreciated!