I have some stream workers. They are listening to twitter stream and save into mongodb. I need to aggregate saved tweets for intermediate analyze result. I have analyzer workers. They are periodically aggregated with some (15 query) queries. At this time mongodb loads increase and use whole cpu and ram. This is reasonable, okey but I need to decrease loads.

I have thinked doing aggregation in stream workers (in memory) and periodically save results into database.

However I could not be sure that solution.

What should I do for decreasing loads and improve performance?

*Especially I do not want to change or modify database solution because that changes will be so expensive.

I wish I could explain my problem.

Thank you.