I’m looking at building a customer data platform on top of AWS that can support both analytical and real-time use cases.
At a high level, events can flow into storage as well as trigger actions:
analytical stuff
|^
v|
|---> processing ---> storage
| | ^
events | | |
| v |
|---> processing -> enrichment -> actions (200ms)
For example, segmenting (non real-time) the data or enriching incoming (real-time, ~200 ms).
Redshift (OLAP) appears to suit the analytical side, however does not suit the real-time aspect of data enrichment. RDS on the other hand seems to handle the real-time stuff just fine (OLTP) but not the analytical.
For the storage
component of the diagram, would it be better to go with Redshift + some in-memory cache? Or is there a catchall for this?