I have a question

We designed out rabbitmq model as having one queue per customer.

This way we can guarantee concurrency and FIFO for each of them separately and maximize distribution across our consumers.

However this poses a big question to us, we will have for each consumer, consume from all these queues.

Which for short means, we have to loop throught all of our customers, call the channel.consume on their respective queues.

If anothre customer is added we will need to add accordingly, maybe do it every 1 or 2 mins, so new queues also getting consumed.

Is this a good idea. Or we will run into consumer side bottlenecks such as long time for setting up consumption or updating these.

Also as the number of customers grows , we have a growing numer of consumption plus other supportive queues.

What d you guys think of our design choice?

Thanks
Pedro

We want some feedback on this design of if there is a better way to consume many queues at once. Our idea is similar to sharding, however we are controlling the shards and exactly to which they are going to. The biggest issue is we cannot consume the as one logical queue, which would be perfect to solve this ever inceasing number of queues issue.