I currently have several web applications that offer document upload and storage.
I am now trying to create a central document management service that these application can leverage to store and retrieve these documents.
Currently the documents are stored on disk. The file name is a Guid and its real file name and datatype are stored in a database.
Am I following the right approach whereby I have a database table that stores the documents information (Name, ContentType, CreatedDate, CreatedBy, SystemId etc), and then store the document in a folder on the local file system of the server ?
When I started storing documents, they were originally blobs in the database. This I found an unfavourable approach. My database was suddenly MASSIVE and impossible to backup and restore. Be separating the documents into the local file system the database is now far simpler and my backup strategy easier. (Backup database, File Diff backup from folder)
Is there a more sensible way to do this I am missing?
Is there some open source project that I can leverage to improve the design of my new service that I am now making from scratch ?
4
This is probably the best approach.
My only caveat would be that you need to think about backup and error recovery of your document files.
Storing two copies on separate disks would give you roughly the same recoverability as a blob in the database.
1
Depending on the size of your enterprise, as the demands of your content management system increase, you may need multiple instances and so your architecture should be designed in such a way as to target not only a document id, but the content management instance id for which it resides.
The service users should not have to worry about this routing detail, however. Architect a solution where you have some sort of rules engine that figures out the best instance to use to store the document. It could be based on some sort of statistics gathering or monitoring process (perhaps some other scheme altogether) so that the most efficient routing path is used.
The point is that this “routing system” is decoupled from both the content management system instance AND the clients who use them, allowing you to maintain your routing scheme without too much disruption to either the users or the content management system instances.
As the document management process runs over an extended period of time, you will have a better feel about what rules to apply for routing, possibly changing your rules engine.
Your guid would be thus created by your routing system, not the content management system (for example docid + “:” + instanceid). The routing system database that stores metadata should have sufficient information that can be used to support future changes to rules engine (to make it work even more efficiently)