I have a react application (powered by Next.js, and implemented in TypeScript) that includes an api-backed data set in one component. That component’s api data can be cached indefinitely since it is readonly a data set, which should never change on the server. There’s a lot of good examples for how to do exactly this on the web, along with more elaborate full-on CRUD applications. These applications maintain a set of data on both the server and client, keeping them in sync between the two. That work’s fine for apps that should render the full data set form the server. I’ve also seen examples for paginating when the server data set is large.

I am now trying to build a component which needs to function a little differently than I have so far found examples for:

1. On startup fetch only the most recent 100 messages from the server, and render them
2. Periodically poll the api for the next N messages since message M (where M is the most recent message that the web app has)
3. When the polled api yields messages, add these the set of rendered messages.

In the examples I see the trigger for re-rendering of the react component (on new data) appears to require that the component is being built from a state variable array, which is itself set (overwritten) inside the fetcher. I think I cannot use a state variable there since useState implements the corresponding set* method to simply overwrite the state variable’s data, not add to it. I initially tried with two variables: One state variable that is populated with the new messages, and one which contains the superset of received data, however I do not see the react component being re-rendered following update of the state variable that it is build from (packetsToRender).

My current code is below:

React component code

import { useState, useEffect } from 'react';

var packets: PacketSummary[] = [];
var latestPacketId: number = -1;

function CommsPacketList({}: {}) {
    const rows: any[] = [];
    const [fetchError, setFetchError] = useState(null);
    const [isLoading, setIsLoading] = useState(true);
    const [packetsToRender, setPacketsToRender] = useState(packets);
    var url: string;

    function addPacketsToArray(newPackets: PacketSummary[]) {
        var lastId = -1;
        if (newPackets.length == 0) return
        newPackets.forEach((packetSummary: PacketSummary) => {
            packets.push(packetSummary);
            lastId = packetSummary.packet_index;
        });
        latestPacketId = lastId;
        setPacketsToRender(packets);
        console.log('Added packets to array. latestPacketId = ', lastId);
    }
    
    useEffect(() => {
        const fetchNewItems = async () => {
            try {
                url = API_ENDPOINT__COMMS_PACKET_LIST + '?limit=100&offset_index=' + latestPacketId;
                const response = await fetch(url);
                const data = await response.json();
                addPacketsToArray(data);
                setFetchError(null);
            } catch (error: any) {
                setFetchError(error.message);
            } finally {
                setIsLoading(false);
            }

            setTimeout(() => {
                fetchNewItems();
            }, 5000);    
        }

        (async () => await fetchNewItems())();
    }, []);

    if (!packetsToRender && fetchError) return <div>Failed to load. Error: {fetchError}  </div>
    if (!packetsToRender && isLoading) return <div>Loading...</div>
    if (!packetsToRender) return <div>No packets!</div>

    packetsToRender.forEach((packetSummary: PacketSummary) => {
        rows.push(
            <MessagePacket
                packetSummary={packetSummary} 
            />
        );
    });
    return (
        <div className={styles.comms_packet_listing}>
            {rows}
        </div>
    );
}

With this code I have the following issues:

1. The component only gets rendered once – the first time that the api response includes some messages. All subsequent updates to packetsToRender by addPacketsToArray do not yield any component re-rendering, so only the first messages are visible. (I added console.log output to verify that packets and addPacketsToArray are being updated, and that the packetsToRender.forEach... is not being invoked following the initial load)
2. All api requests are duplicated: The network panel shows that the initial request with offset_index=-1 is made twice, as are all subsequent requests made at 5 second intervals.
3. I am using setTimeout to implement the polling, but I wonder if that’s an OK way to do this? Seems a bit hacky to do that inside the fetcher, just curious about good practice there(!)
4. I’m using globals for the main messages array (packets and lastPacketId), something which my code editor tells me is frowned-upon in TypeScript-land (which I’m equally pretty new to – again, this a a “good practice” question)

Regarding alternatives to setTimeout: I have looked into axios and useSWR, but did not yet find anything to help me here. (useSWR has the ability to set a refreshInterval but in my testing I found that I couldn’t drive the updates faster than one per second – whereas I need the updates to run a bit quicker – every 0.1 second is the target)