I have a platform that has some requirements including the following:

• Generating PDF/Image files from raw data in async.
• Uploading these files to DB and/or AWS S3 Buckets
• Complex RBAC verification processes.
• Sending emails/SMS/Notifications to users
• Integrating several different third-party services
• Performing CRON Scheduled tasks like metric aggregation or cleanup processes of database.
• Other common REST API, MQ service procedures.

We’re also having to ensure these scenario working:

• We’ll deploy this platform in several countries, and on each country some processes vary, for example: A Payment process can allow for usage of different services on each country, as well as emitting receipts or notifying of payments in different third-party services.
• Different costumers might use alternative ERP’s/Accounting third-party software to process payment information and we want to send payment info to each one separately.

So to achieve this functionality we thought about implementing several microservices for each task, and have them read their tasks from several queues that somehow, work differently for each country/customer. This also would allow us to migrate each indepent service to other frameworks separately, that we cannot do now due to them all being in the same service.

I thought about implementing something along the lines of this:

1. WebUI Portal emits a requests to get payment options to a Payment REST API Service
2. Payment REST API emits and RBAC Request to RBAC Service through a QUEUE1
3. Payment RBAC Service verifies the operation and emits the operation request to QUEUE2
4. Several ThirdParty-based Services subscribe to QUEUE2, each modifying what would end-up being the expected response to the original REST-API adding their own Payment option information.
5. When all thirdparty services finish up, the message gets send to QUEUE3/Returned to Payment REST API.
6. REST API Verifies the message and responds the HTTP Request with the payment options/info.

My questions regarding this would be:

1. How could I use an MQ, like Kafka or RabbitMQ, to have different services subscribe to these messages on each country/server, each without knowing each other, and have them all process the message sequentially and have the message return to the final queue after all are done? If none of these consumers know if they are the last processor, how can they acknowledge the message or send it to another queue?

2. Is having an MQ interposed between the request and response process of a REST API something acceptable or desirable? I realize having a WebSocket on the WebUI would probably be better for this async nature but having an REST API in front has several advantages that I wouldn’t like to forfeit.

I also realize that I could somehow configure each service/step of the processing chain oneach server, indicating to each of them in which order they must process the messages and which is the last one, however I’m afraid that this configuration could get really complicated over time and that not all scenarios could be solved by and ordering. This payment procedure is just one example of the many variations that we face.