Format version upgrade script

[achilleas-k]

We need a way to make old files compatible for writing with newer version of the library.

With the upcoming changes in v1.5.0, we will be upgrading the file format version to 1.2.0. This will make it impossible to write to old files (< 1.2.0) with the new library. We should have a command line script that upgrades a file to the newest version. This script can be maintained in the future for further format changes.

Currently, for the 1.1.x to 1.2.0 file format transition it would have to:

• Change the format of the Property objects (no more Value object).
• Add a File UUID.

[achilleas-k]

By default running the script should open the file in Read Only mode, determine the upgrade tasks that are needed, print them to the user, and warn them that the file will be upgraded in place. This should warn the user about having backups of their data.

A flag can be set (--force) to perform the operation without asking, for batch processing/scripting.

[matham]

Seems like there's already a script in cmd/ that upgrades the files!?

Would it be possible to make this script available through the API so that I could call e.g. nixio.upgrade_file(filename, output_filename=None). That would be easier to use for me to help users upgrade through my program. Also, it would be nice if the function that checks for compatibility is part of the public API (i.e. can_write and can_read).

Also, while I have been aware for some time about the version incompatibility issues, I don't see anywhere on the site that lists which nixpy versions break file compatibility with previous versions or such plans for the future. It would be nice to have a list somewhere listing these versions and possibly what they changed. But what would be even more helpful is having a roadmap somewhere that indicates any future plans for breaking compatibility so that users could plan for it. Although I get that may not be knowable before it happens. But still, something about any such future plans would be helpful in the readme or somewhere prominent.

[jgrewe]

Hi @matham regarding the changes I am currently update (actually rewriting) the docu for nixpy. It will contain a section about changes in 1.5.
Will have a look into your ather points as well.