I am developing back-end APIs where one IPDR file is uploaded in .csv format using one API endpoint “/api/upload”, and this endpoint returns a file token to refer to that file for other APIs calls.

Then, using another API endpoint “/api/execute”, we can process that IPDR file. The file token is provided in the request of the API call.

The IPDR file is from different ISPs, so it parses through a general parser that standardizes the IPDR file into a general format with the same number of columns and the same column names.

After parsing, there are two columns:

– destination ip

– destination port

Based on these, other columns are appended to the IPDR. These additional columns are retrieved from a database table and external APIs. The database table contains almost 2,300,000 records.

The problem is that appending columns and their data from the database and external APIs takes almost 5 to 20 minutes, also one IPDR file contains 5,000 to 70,000 records.

Therefore, I have created another API endpoint “/api/status” that returns the status of that file’s process. The file token is provided in the request of the API call.

The status of the file can be:

– File under process

– Error in parsing

– Error in appending columns

– File processed successfully

Currently, I do not want to make this a real-time API that sends a response when the file processing is completed without client request for file status. Instead, I want the client side to call the “/api/execute” endpoint and not wait for or handle its response. Then, the client can get the status from the “/api/status” endpoint.

What happens to the response of “/api/execute”? Since the server sends a 200 or 400 status response to the client when file process is completed.

Is this a standard approach to do this, or are there better approaches?

The back-end API is created in Flask, the database used is SQLite3, and the external APIs are used from ipinfo.io.