I am looking to add an audit trail to our Rails app. At the moment, I am trying to lock down my requirements and determine the appropriate software architecture.

Our website gets about one hundred million page requests a week, and growing. We currently log a subset of actions into a MySQL table, but that table is much too large and drags down performance of the rest of the app. It’s therefore hard to imagine solutions such as audited or paper_trail would work out. We also can’t just keep around our log files, even compressed, because they are huge.

So, what sort of software architecture is appropriate here?