How to design a scalable notification system? [closed]

I need to write a notification system manager.

Here is my requirements:

I need to be able to send a Notification on different platforms, which may be totally different (for exemple, I need to be able to send either an SMS or an E-mail).

Sometimes the notification may be the same for all recipients for a given platform, but sometimes it may be a notification per recipients (or several) per platform.

Each notification can contain platform specific payload (for exemple an MMS can contains a sound or an image).

The system need to be scalable, I need to be able to send a very large amount of notification without crashing either the application or the server.

It is a two step process, first a customer may type a message and choose a platform to send to, and the notification(s) should be created to be processed either real-time either later.

Then the system needs to send the notification to the platform provider.


For now, I end up with some though but I don’t know how scalable it will be or if it is a good design.

I’ve though of the following objects (in a pseudo language):

a generic Notification object:

class Notification {
    String                 $message;
    Payload                $payload;
    Collection<Recipient>  $recipients;
}

The problem with the following objects is what if I’ve 1.000.000 recipients ? Even if the Recipient object is very small, it’ll take too much memory.

I could also create one Notification per recipient, but some platform providers requires me to send it in batch, meaning I need to define one Notification with several Recipients.

Each created notification could be stored in a persistent storage like a DB or Redis.

Would it be a good it to aggregate this later to make sure it is scalable?

On the second step, I need to process this notification.

But how could I distinguish the notification to the right platform provider?

Should I use an object like MMSNotification extending an abstract Notification? or something like Notification.setType('MMS')?

To allow to process a lot of notification at the same time, I think a messaging queue system like RabbitMQ may be the right tool. Is it?

It would allow me to queue a lot of notification and have several worker to pop notification and process them. But what if I need to batch the recipients as seen above?

Then I imagine a NotificationProcessor object for which I could I add NotificationHandler each NotificationHandler would be in charge to connect the platform provider and perform notification.

I can also use an EventManager to allow pluggable behavior.

Any feedbacks or ideas?

Thanks for giving your time.

Note: I’m used to work in PHP and it is likely the language of my choice.


Edit (according to morphunreal’s answer)

  • How many messages per second are you sending (Define current/initial levels, define a maximum level that the system should handle before being re-designed)
  • What hardware constraints does the system have (memory, cpu etc available to the system)
  • How will hardware scale (ie, adding more servers, cloud computing etc)

  • What languages / systems will be generating notifications?

It is on my own, I’m in charge to create the notification programmatically but constructed from an UI.

  • Does the generator know the recipients of the message (?) or are they provided by some other means (ie, business rules for certain alert types go to certain recipients)

It should be possible to create a notification for a specific recipients, a group of recipients (for example using a tag system), or for a whole platform.

  • Are there business rules for adding CC/ BCC / Read receipts

Yes. Note that this is really platform specific and the read or cc are not available on all platforms.

  • Does the generator know the type of message it’s sending (ie, SMS/ email) or is it based off the recipient

It is based on the recipient, however, as the recipient is platform related, and platforms have different way to handle data, the UI is likely to be platform specific to allow to set up images, a sound or something.

  • Does the generator require confirmation of messages being sent/received/read (async vs synchronous sending)

Well the system should be error prone but we would like to handle the error to define a set of rule, for example, if the server is unreachable, the notification should be requeued for further process, but if the notification is incorrect (or has been defined as is by the platform provider) it should not be requeued but notified.

  • Are there requirements to store a history of message sources / recipients (how long for?)

Yes, we likely want to make some statistics and report.
* Define Notification End Points


  • What services are being used to send messages?
    Depends, some are classical REST webservice, other are some exotic protocol, it is really up to the provider.

  • What feedback / confirmation is provided (sync / async)

Depends, some are synchronous and reply with an error while some other need to be pulled afterward to check for errors.

  • Is it likely to add new end-points [even if so, does it even need to be abstracted]

Yes, actually, our app is growing and we likely want to be able to add new provider, but it is a ratio of something like 1 or 2 per year.

4

Start by separating out your functional and non-functional requirements, and carefully defining them. It’s very easy to over-design a system based on ambiguous requirements.

For Example, your non-functional requirements in regards to scalability is quite ambiguous. It can relieve a lot of mental load if you put actual numbers on your system. For example,

  • How many messages per second are you sending (Define current/initial levels, define a maximum level that the system should handle before being re-designed)
  • What hardware constraints does the system have (memory, cpu etc available to the system)
  • How will hardware scale (ie, adding more servers, cloud computing etc)

Now that you have that out of the way, you can set up some tests to ensure that the systems design / architecture is capable of fulfilling your non-functional requirements.

As for the business/ functional requirements of sending notifications/ messages, the following clues may help (if you provide more information I can add to this answer):

Define notification sources

  • What languages / systems will be generating notifications
  • Does the generator know the recipients of the message (?) or are they provided by some other means (ie, business rules for certain alert types go to certain recipients)
  • Are there business rules for adding CC/ BCC / Read receipts
  • Does the generator know the type of message it’s sending (ie, SMS/ email) or is it based off the recipient
  • Does the generator require confirmation of messages being sent/received/read (async vs synchronous sending)
  • Are there requirements to store a history of message sources / recipients (how long for?)

Define Notification End Points

  • What services are being used to send messages
  • What feedback / confirmation is provided (sync / async)
  • Is it likely to add new end-points [even if so, does it even need to be abstracted]

I’m hesitant to provide a concrete architecture example as it would depend heavily based on the factors involved; but often the simplest message systems involve a basic queue, either in memory/application, no-SQL, disk, or relational database. Then a separate process (or several separate processes if distributed) can poll the queue for their message types [ie a cron job]; and send off as necessary.

This could also be enhanced using some push-based techniques (ie PostgreSQL NOTIFY/LISTEN) if there is a requirement for messages to be sent immediately.

The advantage of a simple queue, is that the system generating the message has little work to do, and the processing system can be balanced based on hardware/performance requirements.

1

Have you considered the flyweight design pattern for your notification objects? I am not too sure about it, since I never implemented the pattern, but I recall it involves sharing some state among the flyweight objects. I also think, the members with more redundancy had bigger impact if shared. So it depends if you send only a few msgs to a lot of people or vice versa.

Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa Dịch vụ tổ chức sự kiện 5 sao Thông tin về chúng tôi Dịch vụ sinh nhật bé trai Dịch vụ sinh nhật bé gái Sự kiện trọn gói Các tiết mục giải trí Dịch vụ bổ trợ Tiệc cưới sang trọng Dịch vụ khai trương Tư vấn tổ chức sự kiện Hình ảnh sự kiện Cập nhật tin tức Liên hệ ngay Thuê chú hề chuyên nghiệp Tiệc tất niên cho công ty Trang trí tiệc cuối năm Tiệc tất niên độc đáo Sinh nhật bé Hải Đăng Sinh nhật đáng yêu bé Khánh Vân Sinh nhật sang trọng Bích Ngân Tiệc sinh nhật bé Thanh Trang Dịch vụ ông già Noel Xiếc thú vui nhộn Biểu diễn xiếc quay đĩa Dịch vụ tổ chức tiệc uy tín Khám phá dịch vụ của chúng tôi Tiệc sinh nhật cho bé trai Trang trí tiệc cho bé gái Gói sự kiện chuyên nghiệp Chương trình giải trí hấp dẫn Dịch vụ hỗ trợ sự kiện Trang trí tiệc cưới đẹp Khởi đầu thành công với khai trương Chuyên gia tư vấn sự kiện Xem ảnh các sự kiện đẹp Tin mới về sự kiện Kết nối với đội ngũ chuyên gia Chú hề vui nhộn cho tiệc sinh nhật Ý tưởng tiệc cuối năm Tất niên độc đáo Trang trí tiệc hiện đại Tổ chức sinh nhật cho Hải Đăng Sinh nhật độc quyền Khánh Vân Phong cách tiệc Bích Ngân Trang trí tiệc bé Thanh Trang Thuê dịch vụ ông già Noel chuyên nghiệp Xem xiếc khỉ đặc sắc Xiếc quay đĩa thú vị
Trang chủ Giới thiệu Sinh nhật bé trai Sinh nhật bé gái Tổ chức sự kiện Biểu diễn giải trí Dịch vụ khác Trang trí tiệc cưới Tổ chức khai trương Tư vấn dịch vụ Thư viện ảnh Tin tức - sự kiện Liên hệ Chú hề sinh nhật Trang trí YEAR END PARTY công ty Trang trí tất niên cuối năm Trang trí tất niên xu hướng mới nhất Trang trí sinh nhật bé trai Hải Đăng Trang trí sinh nhật bé Khánh Vân Trang trí sinh nhật Bích Ngân Trang trí sinh nhật bé Thanh Trang Thuê ông già Noel phát quà Biểu diễn xiếc khỉ Xiếc quay đĩa
Thiết kế website Thiết kế website Thiết kế website Cách kháng tài khoản quảng cáo Mua bán Fanpage Facebook Dịch vụ SEO Tổ chức sinh nhật