Migrate to blob store for Workspace Files
Created by: Piszmog
As part of #31792, the initial implementation we went with is storing files in Postgres in a bytea
column. This is a temporary solution. We ultimately want to store our files in a blob store (MinIO/S3/GCS).
Today, Code Intel are the only users of the blob store. This mean for Batch Changes to use the blob store will require more than code changes.
Code Changes
- Update API handlers to use the blob store instead of database for storing content
- The upload handler should stream the file content instead of storing the entire file in memory
- Create a migration to move files from Postgres to blob store
- Code Intel has #41134 in their backlog that will help our integration
Logistics
- Determine how to restructure the current single bucket to accommodate hierarchy structure
- Eric Fritz/Code Intel
- Loop in customer engineers
- Work with delivery to figure out how to land this change
- Figure out backup strategy
See Slack 🧵