Skip to content

perf: Use godirwalk in gitserver to speed up listing cloned repos

Administrator requested to merge core/fast-filewalk into master

Created by: mrnugget

Inspired by @keegancsmith's comment in the previous PR I decided to look for a faster filepath.Walk implementation. For our use case (we don't rely on the order of the list of cloned repos, we only need the IsDir() info per path), godirwalk seems to be the fastest.

In this change I replaced the existing filepath.Walk call with godirwalk.

Deployed to k8s.sgdev.org and ran the vegeta test again. Result: max latency is down again, by 400ms, down from ~1.2s to ~800ms:

Requests      [total, rate]            290, 1.00
Duration      [total, attack, wait]    4m49.156038676s, 4m49.000915525s, 155.123151ms
Latencies     [mean, 50, 95, 99, max]  201.288068ms, 138.438191ms, 444.46946ms, 731.865021ms, 882.493109ms
Bytes In      [total, mean]            29000, 100.00
Bytes Out     [total, mean]            48430, 167.00
Success       [ratio]                  100.00%
Status Codes  [code:count]             200:290
Error Set:

Test plan: go test & deploy and test on k8s.sgdev.org

Merge request reports