I’m getting into web development, and I’m trying to get a grip on databases.
In an app I want to make, there is a read only database that will be filled up with a large amount of data, which will be the heart of the application, and also a lot of work to put in.
I’m uncomfortable with putting all this data into the database, as the database is “separate” from my web app code. I feel like I should rather write my data in code somewhere, and then have my application put the data into the database for me.
This way I would have my entire project written in code, and not an integral part of my software hidden away in the database, which is a bit complicated to work with, and seems difficult to move from computer to computer.
But is this a sensible way to do this, however? Should I write my data in code, and every time I launch the server, make the application drop all the tables and read them in from my code again?
Is this the wrong way to go about it? Would it cause considerable overhead? How do people do this?
2
Maybe it’s not a feasible solution for you.
What we are doing for some cases: We use the dump-features of inmemory-databases.
- During the building-process we are building up an inmemory-database
(hsqldb…) - dump the database in a file
- add it to the deployment-package
- After starting the webapp we load the dumped file
and use the database.
So you
- don’t deploy the “how do I get the database”-code
- you build your database during the automated build and not during the startup of your app, which can last a lot of time.
1
If it’s all read-only, why do you even need a database?
But there is a reason that CREATE TEMP TABLE exists, don’t be afraid to use it (but reloading it every time your process starts is likely to be more expensive than just leaving it on disk).
2
It depends on what you want to do with that data.
If you need to search that data and only need small parts of it at a time, then keeping all of it in the database is a good idea. The database already implements fast searching and memory caching algorithms for the data. The database is also a natural structure for data if the data itself has many relations. Finally the database also provides a way to load data into your system without having to recompileredeploy the system.
However, if you need all of the data for almost all of the computations, the data doesn’t change, your application is performance critical (and does no searches) and you’ll never need to look or search for a slice of the data, then you could place the data in the application code.
In terms of setting up and moving a database from computer to computer, that’s typically the job of an installerdeployment process. If you keep all of your configuration settings (server name, server id, user name, password, etc) in a editable settings file, then you can more easily move the software from one machine to another. If the database’s license permits, you could install the database software along with the schema and data as part of your installation process.
1
If your database is truly read only then you have no reason to ever rebuild it, because it never changes, then you have no reason to rebuild it every time.
If what you’re concerned about is that you may have your database out-of-sync with your application code version, then one way to work around this is to use a file-based database system like SQLite or as a bunch of JSON/XML in a directory. Just check in the SQLite .db file together with the application code.
Hard coding your data in the application is considered poor practice for anything but the most trivial data.
If you do decide to do this code the “data” in a separate module!
If I understand your requirements — you want to distribute the data with the application in a single cohesive package. Then–
Code up your data in a single JSON/XML file which you can package with the application, and load it into a memcache stype database when the application is initalized.
-or-
Create an sqlite database. An sqlite database consists of a single file which can be deployed alongside your application. Most php/python distributions come with sqlite built in.
In web development, the app code and database data are kept separate. Don’t ever put the code in the database, it just doesn’t make any sense. To answer your original question:
No, you shouldn’t rebuild your read-only database every time. You would create the database once and add data to it over time.
One thing that didn’t quite make sense to me is why would you be rebooting your server often? Servers are designed to remain constantly online so any reboots should be rare.
1