My team has built a log storing mechanism which uses Kafka to store logs in the queue and store it in the database after consumer receives it.
The producer is running on an EC2 instance which is managed by a load balancer, and the consumer is on another instance writing into a database.
Mechanism of sending messages when instance of producer exists
When the traffic increases, multiple producers instances start with the same configuration as existing producer instance which sends data/messages to the pre-existing consumer EC2 instance, where the set condition for the spawning instances is to spawn in the same region but based on availability of zones.
Multiple producer instances sending data to consumer