Skip to content

cargo: don't fail unpacking on duplicate files

Administrator requested to merge mrn/unpack-duplicate-files into main

Created by: mrnugget

Turns out that crates.io sometimes has duplicate files in their tar.gz archives.

$ wget 'https://crates.io/api/v1/crates/proc/0.1.4/download' -O proc-0.1.4.tar.gz
$ tar -ztvf  proc-0.1.4.tar.gz
-rw-r--r--  0 1000   1000       18 Aug 14  2015 proc-0.1.4/.gitignore
-rw-r--r--  0 1000   1000     1107 Aug 21  2015 proc-0.1.4/.travis.yml
-rw-r--r--  0 1000   1000      304 Aug 29  2015 proc-0.1.4/Cargo.toml
-rw-r--r--  0 1000   1000    10847 Aug 21  2015 proc-0.1.4/LICENSE-APACHE
-rw-r--r--  0 1000   1000     1071 Aug 21  2015 proc-0.1.4/LICENSE-MIT
-rw-r--r--  0 1000   1000     1149 Aug 29  2015 proc-0.1.4/README.md
-rw-r--r--  0 1000   1000       22 Aug 29  2015 proc-0.1.4/src/lib.rs
-rw-r--r--  0 1000   1000      304 Aug 29  2015 proc-0.1.4/Cargo.toml

There are two Cargo.toml in there.

That leads to errors and retries in production: https://sourcegraph.grafana.net/explore?orgId=1&left=%7B%22datasource%22:%22grafanacloud-sourcegraph-logs%22,%22queries%22:%5B%7B%22refId%22:%22A%22,%22editorMode%22:%22builder%22,%22expr%22:%22%7Bapp%3D%5C%22gitserver%5C%22%7D%20%7C%3D%20%60proc-0.1.4%2FCargo.toml:%20file%20exists%60%22,%22queryType%22:%22range%22%7D%5D,%22range%22:%7B%22from%22:%22now-7d%22,%22to%22:%22now%22%7D%7D

Test plan

  • Manual testing
  • Existing tests

Merge request reports