A WebApp writes a stream of data coming over a network to disk as key/value pair and then reads & send it over network again after few milliseconds in 99% of case. In 1% of cases write/read can be few hours to days apart (really don’t know).
Latency is strict “no” in the app and we have to serve 100% of customers therefore I am planning to put data into RAM and later read and send it over network and only write to disk if the data is not read from RAM within a certain prefixed interval.
What could be possible solution to this problem?
P.S – Memcached looks like a good solution for using RAM instead of disk but Can memcached read/write of disk after time_to_live?
I also looking at Couchbase but I don’t want to write to disk in all cases as 99% write/read happens in the matter of milliseconds.
Is it possible that after time_to_die Memcached could write the key/value to Couchbase instead of removing it from RAM and read it on demand?
[Edit 1]
After spending last few trying to explore various options. The following looks like a possible solution
- Write the data stream simultaneously to the RAM (using Memcached) and disk using (GlusterFS).
- In case the data is demanded by the application layer check it first in the RAM (Memcached) if it return the data ok, if it does not check the data on disk GlusterFS.
- Remove data stream from RAM if data is demanded by application or when time_to_live is reached whichever is earlier.
Suggestion most welcome.
3
A lot of how you handle this problem is going to hinge on that requirement of 100% of the customers being served. When the goal is absolutely no downtime then performance will almost always have to take a second position to reliability.
You say that your timespan for a typical response is a matter of milliseconds but how do you want to handle the situation where an error occurs within this timeframe? It is easy to imagine a machine getting reset while it is waiting on a long process (say on the order of hours) but that reset could also happen directly in the middle of a transaction that would typically happen in a matter of milliseconds. Maybe the machine loses power at that exact moment or the RAM ends up corrupted.
If reliability is the key to your system then I would say that your only real option is to write the data to a safe storage location before you process it. If the data is not safely stored then it is not processed by the system.
The better solution would most likely be to lower your requirement form 100% to 99.9% or similar and then you could ignore situations like the aforementioned error within a typically fast process.
1
Remember that RAM is volatile and if the server has to reset then that data would be lost. This sounds like it would be a concern, as you say you have to serve 100% of customers.
I would recommend writing your own buffering system which keeps data in RAM but writes it to disk after a small delay. If the data is served within this time then there is no reason to write to disk, and the resources can be freed.
Remember to reload unsent data at start-up.
3
Memcached is just a cache. The out of box java version at https://code.google.com/p/quickcached/ supports writing to disk on exit but for what you need, I think you will have to add your own code.
One thing to consider is how much of data, at peak load do you think you will have 3 years from today? If that is not too much can use your own RAM with a in memory cache like whirly, and your own thread that peeks in the to cache every 5 minutes and saves to disk items that are not yet picked up. Note: your POJO will have to have the checked in time, whirly wont tell you when an item was put into its cache.
3
To put the things simple. Use Redis and you will never speak of memcache again. Following are some plus points:
First of all in Redis you get the option to persist data in the disk.
Second, you have data structures like list, set etc. Also, supports atomic operations.
1