codeintel: Retry pulling upload from MinIO (!15808) · Merge requests · Warren Gifford / sourcegraph

Warren Gifford requested to merge ef/minio-get-object-retries into main Nov 16, 2020

Created by: efritz

Previously in the cloud environment we were seeing a lot of transient "connection reset" errors when transferring uploads within the same k8s cluster (worker <-> bundle manager).

We've since moved to reading from S3/GCS, but the default is to use an S3-compatible MinIO instance. This could likely result in the same types of transient network errors and will cause flaky LSIF upload processing errors.

I'm putting back the retry logic that worked around the issue between the previous services. The store interface was also able to be simplified, so I did that as a drive-by.

codeintel: Retry pulling upload from MinIO

Merge request reports