I’ve been studying AWS ECS, but I cannot understand how to architect the following scenario:
I’m trying to run hundreds of algorithms at once in real-time:
- Each algorithm needs to run in a single isolated container
- Algorithms share the same common code / environment setup and only one file is different in each algorithm (let’s call it the “algo-file”) which contains some custom code that differs from all other algorithms
- Algorithms are long running – they should be perpetually live until they get a termination signal from a main instance that I have built
- If an algorithm’s container fails, the ECS should replace it with a healthy container
- No two “algo-files” are same across algorithms, even though all other files are the same
Each container should also be able to interact with a common database and call external APIs and receive requests from internal EC2 instances.
Any suggestions on how this may be architected? Is AWS ECS even appropriate for this?