Skip to content

Normalize changeset_specs database

Warren Gifford requested to merge es/start-normalize-changeset-specs into main

Created by: eseliger

When we first started, it seemed like we'd transform the changeset_specs schema every day. 2.5 years later, we never modified it and I don't expect we will very often, maybe never once we have uploading of execution logs from src-cli. This PR starts a migration towards a normalized database where we don't store a big JSON blob anymore at all. This makes it easier to filter and should speed up some queries, too. In this releases' PR of this series, we store data in the new AND old scheme, and have a migrator converting old records into new ones. Once that is done, in 3.44 we can start to remove the spec column and scan the new columns into the btypes.ChangesetSpec type, and add checks/constraints on the new columns. This'll also eventually raise the maximum size of a diff we can store to 1GB, up from 256MB - len(everythingButDiffInJson).

Test plan

Verified locally that the migratory runs to completion on my DB, and wrote tests for the changes.

Merge request reports

Loading