Data structure migration strategy as database records

Currently data structures are serialized via streams (CDataStream, CVectorWriter) to database, that means at any moment we should know the correct compile time type to get back data from database. The very first action that we take is to allow partial reading from stream, currently it's done via BytesToDbType it gives ability to read a stream via data structure which is a part of entire serialized one or it's a bigger one, it means the rest fields stays uninitialized. Allowing partial serialization is the key moment in data structure migration strategy. During development we extend or shrink data structures which may result in inconvenience how to upgrade data structure from one version to another keeping chain data consistent. By split data structure to base, intermediate and newer is the strategy:

• base data structure - all its fields present in newer version
• legacy data structure - inherits base one adds only removed fields
• newer data structure - inherits base one adds only newly fields

Depend of situation we can use base or legacy data structure and via upcoming hardfork one time upgrade to newer version. Consider type1 as type that we should upgrade to type2

struct type1 {
    int data1;
    char data2;
    double data3;
};
struct type2 {
    int data1;
    float data4;
};

so as first step we split type1 including old fields that match type2 and type0 which includes fields from type1 including fields to be removed.

struct type1 {
    int data1;
};
struct typeO : public type1 {
    char data2;
    double data3;
    // serialize as type1 + data2 + data3
};
struct type2 : public type1  {
    float data4;
    // serialize as type1 + data4
};

So till newer hardfork reads are done via legacy type0 after one time upgrade newer type2 towards. RPC layer will be kept via compatibility functions between type0 and type2.

For reference https://github.com/DeFiCh/ain/pull/826