Context

I work on an embedded system (STM32) that needs various configuration to work properly.
The config data looks like a key/value dictonnary. Values can be INT, UINT, FLOAT, CHAR *.
I have a software module serving the configuration as a C structure that looks like this :

typedef struct {
    int Param1;
    char Param2[16];
    float Param3;
    ...
} MY_CONFIG;

As the data is dynamic and can be changed by the user, I choose to store it into a flash memory.

The naive solution is to write the structure into a sector of the Flash.

Versioning

Now comes a new firmware version with a new “Param4”. It could also be “Param2” evolving from [16] to [32], or even the deletion of Param1.

What ever the change, the structure of the configuration stored in the flash memory doesn’t match anymore it’s new layout in the new firmware.

The main issue here is to be able to keep previous configuration data after upgrading the firmware (downgrading is not relevant here).

Solutions

Here are some solutions I found/encountred.

Adding a version to struct

We add a Version field to the structure and create new struct for each version :

typedef struct {
    int Version;
    int Param1;
    char Param2[16];
    float Param3;
    ...
} MY_CONFIG_V1;

typedef struct {
    int Version;
    int Param1;
    char Param2[32]; // Increase size
    float Param3;
    ...
} MY_CONFIG_V2; // New struct is created for each changes

Then we create a function to upgrade the version of the configuration :

void config_upgrade_v1_to_v2(MY_CONFIG_V2 *pNewConfig, MY_CONFIG_V1 *pOldConfig)
{
    newConfig->Param1 = pOldConfig->Param1;
    strcpy(newConfig->Param2, pOldConfig->Param2);
    ...
}

This function is called by the new firmware when the Version field is read as 1 from memory flash.

This gives so much work even for a tiny change and is not maintainable at all. The duplication of the MY_CONFIG_Vx structs gives an awfull file of hundreds of lines.

Storing as a dictionary

To avoid versioning the structure, we can store the data as a key/value pair :

typedef struct {
    int Param1;
    char Param2[16];
    float Param3;
    ...
} MY_CONFIG;

would give us this data to store into flash memory

PARAM1=6500
PARAM2=SOME STRING
PARAM3=3.333334

Saving and reading back config would look like this :

With old firmware :

• For each params, build an ASCII string KEY=VALUE with the correct formatting of VALUE acording to its type.
• Write the ASCII output to flash memory

With new firmware :

• Read each line and extract the string KEY and VALUE
• If KEY is unknwon to new firmware, it might be an obsolete parameter not supported anymore, just drop it
• If KEY is known, parse its value and try to use it into the data structure MY_CONFIG.
• If a field of MY_CONFIG has no key/value pair in the ASCII representation, it might be a new parameter, use a default value for it.

This solution is much more memory consumming and needs to parse the ASCII data at reading and writing time but resolve the issue of versioning.

Question

Do you known an intermediate solution to concile versioning issue with memory and CPU usage ?
I think the key here is to link the data with it’s meaning using a unique identifier and not only with the positionning offset inside the structure.

This post is also a mean of sharing as I did not find threads discussing this topic.

Dave