HDD Failure Paranoia
I tend to make small programs for myself. Whenever I make an application I use a database application like MS Access or MySQL. I can CRUD my data as I want. No problem in finishing a task.
The problem is when I try exploring more into File IO. I have heard of HDD crashes due to high read/write operations. I used to think that these happened only on server machines. But then I came to know that it happens on desktop machines too! So I now I have this huge mental blockage. Whenever I try converting my former software into ones with raw FILE IO, I lose the “mental drive/motivation” and freeze. I get thoughts like “OMG this task will cause this many reads/writes to the disk”. I fear for loosing my hard disk. My goodness! It has happened so many times that I wish that the computer engineers had redesigned the internals of the computer to fix the problem and so I curse them a lot. A lot of my time gets wasted remotivating myself and the projects never get completed.
Ya, ya, I know that behind the scenes the database applications use raw File IO to do their work! But who thinks of that when using them. All you are mostly thinking of is SQL and the resulting table in the resultset/recodset. The database applications are like a black box to raw File IO.
Does anyone else think the same? Are my fears valid? What should I do? Any views/ consolations? Help! This makes me mad and unproductive!
4
There’s a lot of discussion of this and it’s actively studied (obviously). The key terms are “mean time between failure” and “average failure rate”, MTBF and AFR. Those failures are usually driven by how many times the drive is power cycled, not by read-write loads. Even finding numbers for read-write cycles on legacy drives is hard.
Seagate numbers here and an Ars Technica article giving stats across brands. The short version is: don’t worry about it. As long as you do regular backups you’ll be fine.
With solid state drives (SSDs) it’s more complex. Solid state memory does often fail through write cycle exhaustion, but there is a lot of circuitry and software in the drive to stretch the 10,000 cycles of the chips out to 10x or more for the drive as a whole. Then If you don’t fill the drive right up then you do the maths it usually works out that you’d need to continually rewrite a patch of disk for a year to get close to the wear limit. There are articles like this PCWorld one on what you can do to help.
Finally, your operating system is likely to be buffering your writes, and your database engine definitely is. Especially when you re-write a lot quickly the data is unlikely to end up on your disk every time. I suggest putting the database into “in memory database” mode before worrying about a ramdisk, because then the database is doing the work and it’s usually easy to also go “write the whole database to disk” in one easy operation.
1