I have been professionally writing code for years, but I have always been working in existing (legacy) code bases. I’m currently building a new system from scratch and I’m finding out that I have a massive blind spot when it comes to existing frameworks/libraries. There is so much out there, that I’m having a hard time finding things that are relevant for me. I guess I’m looking for some pointers or even just the right terminology to google.
The product I’m working on is fairly simple: It’s a measurement system consisting of multiple processes that interface to sensors, process, visualize, and store data in a bunch of specific file formats. The system is distributed over multiple machines. We want to be able to centrally control things like
- configuration
- logging
- allocation of processes over hosts
- start/stop processes
- monitoring output/state of the processes
- etc.
I think we can implement all of this by building something on top a pub-sub library (e.g. DDS) and a noSQL database, but it seems like this should be common enough that there are frameworks out there that provide a decent portion of this.
My biggest challenge so far is that whenever I try to google for these things, the results seem to be almost 100% AWS/Azure. Which (I think) is not an option for us, since this system will have to run on site without an internet connection for months.
Some things I have found
- ROS2. Seems to have a lot of the things that we would need (communication, configuration, remote start/stop) but I’m a little hesitant because that’s build for robots which is not what I’m building.
- Apache ZooKeeper. This is pointed to a lot when searching for distributed configuration, but it feels a little heavy for what we’re trying to do.
Any pointers would be greatly appreciated.