I’m writing a device Software in which the device must handle much data. Therefore it has a database, either on NV-Ram (Non-volatile memory) or a SD-Card.
Due there’s a DB which I can use, I also use it to store configurations which should survive restarts or power loss.
Everything is working fine so far, but how can I prevent corrupted configs?
Using SQLite3, I can be relatively sure, that changes either aren’t saved, or are correct.
But what happens if wrong data is stored? This is a viable scenario.
Which techniques are good to check a set of data for coherency?
So far:
By (re-)start of the device, I check if DB holds a config and if not, I’m writing the hard-codet default config to the DB.
There are multiple problems with that, it does only check if there IS an existing config table in the DB, but not if the table is validt.
One Idea I had so far, is to iterate through default-config and check every entry in config if it has the specific entrie, if yes check for datatype, if not throw error. Still, this would not prevent invalid data.
Example:
Entry “reader-port” (hardware-port on which an specific device is connected)
Expected datatype -> String (example: “ttyscom2” )
Given data -> String (“paoghsfodgvfnamox33” )
As you can guess, this is wrong.
Note: This question is not language specific. But beside of this, I’m working with Lua.
2
To protect yourself against garbage input, you should validate the input as the user enters it. In the large majority of cases, your garbage input won’t be as bad as “paoghsfodgvfnamox33” for a com-port, but rather clumsy typing, like “ttyscomn2”. If you catch these errors as soon as possible, while the user is still editing the configuration, then the user can immediately correct the error.
As for the database being modified outside your application, there are two lines of thought here:
- The user isn’t supposed to access/modify the database directly. If they do so anyway, then they have broken the warranty seal of the product (figuratively speaking) and they are themselves responsible for the consequences.
If you follow this line of though, there is no point in validating the configuration any further than you currently do, as your testing will have proven that the approved methods of changing the configuration can’t corrupt it. - The user is allowed to do whatever they want with the database and we have to be robust against it.
If you follow this line of thought, then you should have additional measures against incorrect configuration values. This can be as simple as first trying the value read from the database and if that doesn’t work, falling back to a hard-coded default value.
Or you can add a checksum column to the database table with the configuration values. That way you can immediately detect that a configuration value has changed outside your control.
In general, one can write a checksum with the data that can be recomputed at read time and compared with the written value. If they do not match, you know there has been data corruption (or your checksum is out of date).
You could go a step further and provide enough redundant information to repair the corruption (error correcting codes), but this might be overkill in your case.
Since you are using a database (SQLite), it generally provides its own checksums and data integrity validation internally. However, I think it is unlikely you would read a string from it and see garbled data, as in your example. Typically if a database file gets corrupted the structural information in the file that the database itself relies on will also be corrupted, so the database will (at best) throw an error or (at worst) crash.