Everyone
I am new to Kafka. I am writing code for a python module that will process messages from Kafka. The module will be containerized using docker. The module will read a Kafka message, do some processing that involves a lot of I/O, and send a kafka message to another topic. This module is expected to receive a lot of messages every second (100 for example). Each message takes about 3 seconds to process, which means I will need a lot of containers to process those messages, but I want to increase the utilization of each container to reduce the number of overall container and reduce the costs. There are two approaches that I thought about: