I am running a Puppeteer script on Heroku which logs into a company website, navigates to a reports page, selects a tab which triggers an ajax call (i believe) and displays the data related to the tab selected.

When the script runs in puppeteer (headless = false so i can observe) it clicks everything correctly, and the UI appears correct however when it goes to download the file, the original data (from page load) is downloaded, but not the data which is currently displayed in the UI after selecting the tab.

When i manually interact with the chromium browser that is being run by the server and click the download button myself, it downloads the correct data.

I have added all sorts of delays and wait for request idle functions.. however nothing seems to fix the issue.

Is there anything obvious as to why this is happening?

I should note it is a relatively new issue, the same script was working on this website one week ago.

    await page.waitForSelector('ul.nav.nav-tabs > li:nth-child(4) > a'); // Wait for the fourth <li> to be available
    await page.click('ul.nav.nav-tabs > li:nth-child(4) > a'); // Click the <a> inside the fourth <li>
    
    await delay(3000)

    console.log(7)

    // open the filters and add the activityReference to the filter options
    await page.evaluate(() => {
      // Select the parent div
      const parentDiv = document.querySelector('.col-3.col-md-6.text-right.ng-binding');
      // Select all buttons within the parent div
      const buttons = parentDiv.querySelectorAll('button');
      console.log("buttons1: ", buttons)
      // Click the third button
      buttons[2].click();
    });

    console.log(8)

    await page.evaluate(() => {
      const labels = Array.from(document.querySelectorAll('label.form-check-label'));
      const targetLabel = labels.find(label => label.textContent.includes('ActivityReference'));
      if (targetLabel) {
          targetLabel.click();
        }
    });
  
    console.log(9)

    // save and close the filter modal
    await page.waitForSelector('button[ng-click="close()"]', { visible: true });

    await page.click('button[ng-click="close()"]');

    console.log(10)
    
    await delay(3000)

    // select the download button
    await page.evaluate(() => {
      // Select the parent div
      const parentDiv = document.querySelector('.col-3.col-md-6.text-right.ng-binding');
      // Select all buttons within the parent div
      const buttons = parentDiv.querySelectorAll('button');
      console.log("buttons2: ", buttons)
      // Click the third button
      buttons[1].click();
  });

    // ensure file is downloaded
    await delay(10000)

I have tried a myriad of ways to click the tab button which loads the new data prior to downloading. Each method shows the button being clicked and the correct data loading, however not matter how long I wait before clicking the download button, the old data is still downloaded.