Upload of large LSIF dump fails
Created by: Strum355
- Sourcegraph version: Sourcegraph.com
- Platform information: src cli 3.22.4
Steps to reproduce:
- Index https://github.com/torvalds/linux with lsif-clang (generates a 14Gb dump)
- Run
src lsif upload
Expected behavior:
Actual behavior:
unexpected status code: 500
failed to enqueue payload: failed to compose objects: googleapi: Error 400: The number of source components provided (147) exceeds the maximum (32), invalid
Further Info
"we need to do this in batches, I think" - @efritz https://sourcegraph.com/github.com/sourcegraph/sourcegraph/-/blob/enterprise/internal/codeintel/stores/uploadstore/gcs_api.go#L83