I am trying to implement a virtual file system structure in front of an object storage (Openstack). For availability reasons we initially chose Cassandra, however while designing file system data model, it looked like a tree structure similar to a relational model. Here is the dilemma for availability and partition tolerance we need NoSQL, but our data model is relational.
The intended file system must be able to handle filtered search based on date, name etc. as fast as possible. So what path should i take? Stick to relational with some indexing mechanism backed by 3 rd tools like Apache Solr or dig deeper into NoSQL and find a suitable model and database satisfying the model?
P.S: Currently from NoSQL Cassandra or MongoDB are choices proposed by my colleagues.
4
As I could see from your questions is, that you first should think about what you need and what is NoSql made for.
NoSQL – made for huge amount of distributed data. Scales well from performance an amount of data. and depending in the Type of NoSQL system you can just put and get objects very fast, or also do long running jobs on the distributed data, it’s not made for fast search/queries.
Search – it’s just made for searching references to data fast. Depending on system it scales well with data and performance. its not made for query huge amount of data nor data relations.
RDBMS – it’s made to store data which have relations to each other. System scales depending on system itself and based on your data design. means even the fastest rdbms solutions could perform out with wrong data/query design. It’s not made for fast search on huge amount of data. And scaling is depending of rdmbs product, it’s not a feature by default.
So if you want to search data from a huge amount, choose the system that is made for it: Search Engine.
If you want to store huge amount of data, where data needs to be distributed (because of amount) and performance of getting data should be independent from amount of data: choose a NoSQL System.
If you don’t have that much data,but data needs to have relations to each other, then choose a RDBMS, and think well of you data design.
If you need a Search which stored the index distributed, combine a search product with distributed storage (filesystem).
If you need a distributed filesystem, have a look at Apache Hadoop.
If you need a NoSQL System like Google Big Table, which is somehow compareable to a RDBMS, have a look at Apache Hbase: hbase.apache.org or Hypertable.
Cassandra is more like Amazon Dynamo. It’s distributed, but more far away from rdbms.
and MongoDB, is more close to LotusNotes. It’s a good storage for Documents/Objects.
And MAIN POINT to solve the problem, think solution dependent, means: do NOT think like RDBMS when using a NoSQL System, you need to think in that specific NoSQL System (they are all very different from each other).
3