I asked this question over at stackoverflow and it was suggested a tighter form be posted here.
Many early career numerical researchers face the prospect of having to create performance critical software from the ground up. Most are ill equipped to do so, and my lab is no different. I currently work extending a sophisticated C application that is typical of a lot of academic code — it is a nightmare to work with and adding new features takes much longer than it should.
I want to learn how to do it better, and I learn by doing. With that in mind, I would like to jump in the deep end and create an application that is parallel (OpenMPI + maybe OpenCL), combines script driven Python UI with C++ for the heavy lifting, and most importantly, makes use of appropriate design patterns to make the code as modular as possible. The primary reason for doing so is to learn the major gotcha’s and some general performance techniques that can be documented and shared around the lab.
I have read some of the books around OO and pattern’s, but I find it challenging to take their abstract nature and apply it to the problem domain in a useful way.
I guess the question I am asking is: if you were given the task to design this kind of software, what would be the focus of your first hour in front of a whiteboard?
1
In one hour on a whiteboard, I’d focus on the following:
-
Requirements: what does the system do? Who will be using it? What are their minimal expectations? With an extant code-base you may have a head start on this, so if high-level stuff like that is already known, I’d imagine that for scientific software you can tighten the focus to data: what are the inputs? What pre-processing/cleaning do they need? Then what are the main processes/transformations the system does (think workflows and dataflows here)? What are the outputs that matter? How should they be formatted and/or presented to users?
-
Architecture: nothing fancy needed, just boxes of related functionality, e.g. a simple 3-block structure of Inputs -> Processing -> Outputs may seem too naive, but you can build a discussion from there with colleagues, and start breaking out pieces, e.g. Inputs -> time-series, parameters, vectors vs. scalars; Processing -> what’s serial and what’s potentially parallel? what models are needed?; Outputs -> charts/graphs? csv dumps? The point is to separate and modularize at this level when it’s cheap and easy to do so, before much sunk-time/energy in code.
I suspect the above will easily consume several hours. 🙂 Also, for scientific code that’s data-centric, OO may or may not be a good fit. Don’t use “design patterns” for their own sake, a lot of those patterns are convoluted work-arounds for weaknesses in languages that don’t have higher-order functions. Hope this helps.