I’m developing a Django project that basically boils down to:
- A user submits a form.
- The Django view parses the form and compiles a representation of that form which is readable by some other ‘function’. By ‘function’, I mean some kind of executable/callable where the Django view may be agnostic about what language that ‘function’ was written in, or even what hardware it runs on.
- The Django view passes the compiled form into the ‘function’.
- The ‘function’ performs its business on the input, then outputs some data in a format readable by the Django view.
- The Django view reads the output of the ‘function’ and renders the data to a format that is consumable by the user.
In my specific case, the ‘function’ is a large numerical computation not suitable for plain-vanilla Python. Furthermore, this ‘function’ has already been developed in C++11 and Python can talk to the compiled binary via os.system and file IO.
My question then is: why should I not just use os.system or subprocess from the Django view rather than
a) redevelop the ‘function’ using one of ctypes | swig | Boost.Python | pyrex | cython;
b) learn how to write Python extensions and re-write the ‘function’ accordingly?
I have also considered turning the ‘function’ into a daemon, but I am not experienced enough with daemons to be able to gauge what the consequences of such a design would be.
Are there undesirable/undefined behaviours that would arise from using os.system/subprocess as described above for a Django project that was operating under load?
Is there something else I am missing that is super obvious to everyone else?
There is also this related question. But with the scarcity of details in both the OP’s use case and in the answers, I am cautious not to fall for my own fallacious confirmation bias.
There is nothing wring with calling a library routine written in one language from an application written in another. It happens all the time. A good example is SQL code embedded in most other languages.
The issues you face with doing this is simply skillset, to maintain this external function you’ll need to know C++ as well as python.
As you say, you have plenty of options for interfacing to this. I would recommend a ctypes mechanism as it will be the lowest overhead, that’s almost the only consideration unless you cannot rebuild the function into a shared library. Making system calls to an external process works, its just a lot slower to get data to and from a process than it is to pass it to a shared library loaded in-process. You could convert the function into a web service and call it using SOAP or REST calls for example.
A daemon is just a process running that is running constantly. Its faster than simply running the function on-demand as the OS doesn’t have to take all the time to start the process every time you want to call a function. To convert it into a daemon you might have to alter the code to run when several applications call the function simultaneously but otherwise its not any different.
So: if the speed of calling the function in a process is not important (eg its an intermittent, or long-running function) then go right ahead and call it using system calls. But, if you need speed, and this function returns quickly, or is called many time, then I think you should learn how C/C++ programs are written and built. Then you will have the skills to turn this into a library and call it using ctypes.