I have a small project where only send around 400 messages per day but it can randomly send those messages at any time, it can be just 1 message, or it can send more than 1 message during a period of 1 to 5 seconds.
My code is designed in the way that once that there’s an event it starts doing something in the background that could take around 10 to 20 seconds, depends on every situation. What I’m trying to achieve is to ignore upcoming events and not queue them because once the process finishes, it starts again over and over depending on how many messages were sent by the producer.
Is there a way to ignore or not put those messages in the queue and only trigger the event when the process is ready to start? Please see the following example of my code
consumer = createConsumer(
KAFKA_TOPIC,
KAFKA_BROKER_SERVER,
KAFKA_PORT_NUMBER
...
)
for message in consumer:
# once a message is sent by the producer it start doing something
# for around 10 to 20 seconds, once it finishes, if the producer
# sent more than 1 message during that 10-20 seconds of period of time
# the event is triggered again
In the example above if 10 messages arrives in a period of 1-5 seconds, it will repeat the process 10 times.
That’s a basic piece of code of the consumer, but only looking to ignore messages/events while the process is running.