Skip to content

cmd/gitserver: Add producer-consumer pipeline to clone repos

Warren Gifford requested to merge ig/COREAPP-180-2 into main

Created by: indradhanush

Status

This is more or less the first pass at COREAPP-180 and I am now ready for feedback and thoughts.

What does this PR achieve

This PR continues the work started in #24885.

In this PR, we introduce a new producer-consumer pipeline to clone repos asynchronously without spawning a new goroutine for each new non-blocking clone request. Instead, we spawn two goroutines, one each for the producer and the consumer. The consumer respects the existing cloneLimiter and spawns a new goroutine to clone the repo if and only if it is able to acquire a unit from the cloneLimiter.

However to be safe, our consumer is a stub at the moment and while the pipeline will continue to "processs" the cloneJobs, they don't actually do anything on them at the moment. We leave our existing approach as-is and add some counters to verify our approach before we decide to flip the switch in a follow up PR.

Merge request reports