I have created a function in a customised WordPress settings page that fetches products from an API. There are a lot of results and it takes a long time, so if I try to fetch them all at once it takes too long and times out. So I have now set it up to get only 1 page of 100 results at a time and there are (for now) 26 pages of results to fetch.

So, the idea is to load the page with a parameter of “page_to_process=1” and when the page sees that “$_GET[‘page_to_process’]” exists, it runs the function that fetches that page number. That function then uses the ‘header’ function to redirect back to the same page but with the “page_to_process” parameter iterated by 1 so that it will then get the next page of results and so on, until such time as $results is empty then it returns to the page with the parameter removed and ceases the running of the script.

Here’s the code:

<?php 
if(isset($_GET['page_to_process']) && $_GET['page_to_process'] != ''){
  gd_sync_products($_GET['page_to_process']));
}

function gd_sync_products($page){
  $api_credentials = api_credentials();
  if(isset($_POST['action']) && $_POST['action'] == 'get_products'){
      $page=1;
  }
  $products=array();
  $results = gd_get_products($api_credentials['username'],$api_credentials['password'],$api_credentials['baseURL'],$api_credentials['version'],$api_credentials['format'],$page);
  $arrayData = json_decode($results, true);
  $page_count = $arrayData['page_count'];
  $productData = $arrayData['data'];
  $products = array_merge($products, $productData);
  $page++;
  if($page > $arrayData['page_count']){
      $url_parameter = '';
  } else {
      $url_parameter = '&page_to_process=' . $page;
  }

  foreach($products as $product){
      // DOES STUFF WITH THE PRODUCTS
  }

  $redirect_url = get_site_url() . '/wp-admin/admin.php?page=api-settings' . $url_parameter;

  header('Location: ' . $redirect_url); 
}
?>

The problem now is that after a while, the page thinks that it’s stuck in an infinite loop and gives a “Page is redirecting incorrectly” error. It does this after it’s retrieved about 20 pages of results, so is just falling a little short. But ideally, I need it to be able to carry for as long as it needed, even if there were 50 pages of results to get.

Any suggestions?