My Falcon + Gunicorn backend accesses a relational database holding data that forms a graph – a table with nodes and a table with edges. Since the graph is not large (a couple of hundred nodes and edges) I load the data once a startup into a Networkx DiGraph
and keep it in memory. Thus, when I request the graph data
- no constant access to the database is required which sounds good w.r.t. performance
- easy use of built-in methods provided by NetworkX to extract different types of subgraphs
Question 1: Is this good practice or are there any (principle) downsides to this approach (beyond my problem below)?
While the graph is not highly dynamic – no “normal” API endpoint edits the graph – it might grow over time in the database. I therefore created an additional endpoint /graph/reload
that when called simply reloads the graph into memory (i.e., the NetworkX DiGraph
object). So anytime there is indeed a change to the database, I call this endpoint.
In principle, this seems to work. However, the problem now is when I run the server using Gunicorn and multiple workers. When I call /graph/reload
then this is done only for one worker instance; the others still use the old graph.
Question 2: What is the best way to handle this?
Of course, I could always access the database for each request. I still would be curious if this can be done with keeping the graph in memory