I am currently developing a microservices architecture consisting of two services, and I’m facing a challenge related to transaction management and error handling.
-
Microservice A: This service is responsible for receiving data, saving it to a PostgreSQL database using Entity Framework Core, and publishing relevant events to Microservice B via Kafka using MassTransit.
await _context.DeliveryDocuments.AddRangeAsync(deliveryDocuments, cancellationToken); await _context.SaveChangesAsync(cancellationToken); await _publisher.Publish(new DeliveryRequestSavedEvent(deliveryRequest, EventType.Create), cancellationToken);
-
Microservice B: This service consumes the messages published by Microservice A and performs certain business logic based on the received data. However, there are scenarios where Microservice B can fail during processing, potentially throwing exceptions.
public async Task Consume(ConsumeContext<DeliveryRequestFromSiteBusMessage> context) throw new Exception();
Given this architecture, I want to ensure that if Microservice B encounters an error, any associated changes made by Microservice A (i.e., saving data to the database) are not committed. Essentially, I am looking to implement a rollback mechanism to maintain data consistency across the services.
In my service A the logic is very simple. It’s very unlikely that data will fail during saving to db, otherwise on service B its alloat of risk that something goes wrong, alloat of complex logic so i need to provide some mechanism that will commit transaction on microservice 1 after the data is consumed and handled on service B, the delay doesn`t matter, about cleanup i think its not possiable because im not just creating new objects, im also existing once, and i dont want to hold state of previous objects, so i dont think cleanup is a good idea.
I would like to know the best approaches, tools, and design patterns to effectively manage transactions across these microservices. Specifically:
- What are the recommended patterns for ensuring data consistency, such as the Outbox Pattern or Sagas? Which one of them is suitable in my case?
- How can I implement compensating transactions to roll back changes in Microservice A if Microservice B fails?
- I read about transaction outbox pattern on masstransit
https://masstransit.io/documentation/patterns/transactional-outbox
but i understand it’s not working with kafka, so i guess i left with saga?
Any insights or examples of how to achieve this type of rollback mechanism in a microservices environment would be greatly appreciated!
Thank you!
5
If all work in A is undone when B throws an exception, then these microservices are obviously not independent.
It sounds like just two stages of a queue. A receives data provisionally for processing. B then performs the processing, and either commits the data permanently or scraps it completely.
A standard requirement for a microservice is that any one of the set works when any other is offline and unavailable, and recovers spontaneously when connections resume.
And the standard size of a “microservice” is that a single one has a full-time team dedicated to looking after it. You appear to be looking after two related microservices single-handedly.
For heaven’s sake just throw out all the junk layers and frameworks and clutter you mention, and write the whole thing in one Postgres database engine using a transaction.
This approach will almost certainly differ from your existing one by being simple and by working correctly.
4