I am working on a Node.js project where I need to fetch data from an API that requires pagination and stream this data into an Excel file.

The goal is to use as little memory as possible because the data I fetch contains a large number of objects (more than 600,000). Currently, the solutions I find keep the objects in memory, causing my computer to crash.

What is the best approach to correctly stream data from an API with pagination into an Excel file in Node.js, while optimizing memory usage and handling a large amount of data efficiently? Any advice or guidance would be greatly appreciated!