I have a use case wherein my program has to consume kafka messages and process them.

I have made a python library which just consumes kafka messages for re-usablity. I am using pub-sub mechanism so that client programs can subscribe to this librabry and receive messages.

Clients have to implement an interface(MessageListener) and pass it to the library to register.

In client program, following is the code structure:

consumer = KafkaConsumer()
listener = MyMessageListener()
consumer.register_listener(listener)
consumer.start_consuming()

My understanding is that this would make both library and the client run in the same thread.

What I want is to run them in seperate threads so that they don’t interfere with each other.

On some google search I came across asyncio and python’s inbuilt threading, but I am not very sure which one should I go for.

Any input would be helpful here.