Handling versions of fs-repo, and fs-repo-migrations mid release cycles

[jbenet]

The 0.4.3 release will include a migration. because there have been 3 RCs, there are many users out there with a migration already performed, and repos at version 4. This migration seems to be just fine, and version 4 as is seems final. It is not expected to have to change before final release, and any changes deemed necessary seem that they could come as a version 5. So this issue is for the future.

However, that said, this has brought up a set of important questions with how we handle migrations and releases.

• a repo version may need further changes before final release. (in some but not all cases, this could be ameliorated by bumping one higher -- i.e. 2 versions)
• the migration (or repo version) may have bugs that just need to be fixed (this cannot be solved by bumping to a higher number, and need to change or "silence" the existing number)
• IF there are problems, or versions need to change, or migrations need to change, some users who tried -devs or -rcs could be left in a weird state... they would need recovery (revert and re-roll fwd).
• They should also be encouraged to try RCs with caution (i.e. backing up important repos).
• The RCs will need to be able to download and auto-install migrations. these should not conflict with the final versions/migrations. (i.e. they should be final, or they should be numbered/named differently (-3-to-4-rc3?)

So we should come up with a convention on how to deal with migrations during a -dev and -rc release cycle.

• My personal favorite is: don't merge a migration into go-ipfs master until it has been thoroughly proved to work and will not be removed. i.e. any further changes would just be another version on top. Depending on the repo change, this may be too cumbersome, leading to large side branches. But perhaps in most cases it will be fine, and cause us to think and make the change carefully and decisively in one go (i.e. dont leave the new version and migration code in limbo "being tested" for weeks). this would be the least work, i think. and cleanest.
• If not, then we will have to consider shipping the migrations as 3-to-4-rc3 or something to signify a version that's not final. (this would need semver-ish support in fs-repo-migrations, which kinda sucks). We need to make absolutely sure than it can be reverted and rolled back, to then re-apply the proper final version. this would be complicated automatically. this would be the most work, and most error prone.
• Another option is to have a special build of fs-repo-migrations that tracks the go-ipfs RCs (fs-repo-migrations@<go-ipfs-version> that can be used to revert the "non final migration", and would remain available in https://dist.ipfs.io. this would be "middle of the road" in work.

Am sure there are other, better ideas. let's discuss

[jbenet]

Also, the migrations should definitely be available in https://dist.ipfs.io by -rc* time.

[ghost]

IF there are problems, or versions need to change, or migrations need to change, some users who tried -devs or -rcs could be left in a weird state... they would need recovery (revert and re-roll fwd).

One way to ease this would be adding information to the repo about which release of go-ipfs did the last migration. A newer release can then revert and re-migrate if it really has to.

Of course it's best to make sure we don't trash a user's fs-repo in the first place :)

Also, the migrations should definitely be available in https://dist.ipfs.io by -rc* time.

This is already supported -- the migrations code in go-ipfs expects the fs-repo-migration binary to be available under dist.ipfs.io