Nothing guarantees downgrades by 1 minor version works, often broken

Created by: slimsag

Today we have nothing in place to ensure migrating down 1 minor version (e.g. "I upgraded from 3.14 to 3.15, found an issue and downgraded back to 3.14") works. This should be supported, but in practice can sometimes break as we do not test it actively today.

We should add infrastructure which tests this (just with server is fine for now, kubernetes/docker-compose would be a nice-to-have but since DB migrations are the only real issue here and those are identical across deploy types it is not needed).