Skip to content

github-proxy: Use custom IdleConnTimeout to avoid EOF errors

Warren Gifford requested to merge core/github-eof into master

Created by: mrnugget

This fixes the seemingly random EOF error that sometimes came up when repo-updater was querying the GitHub GraphQL API.

The reason for these errors was that GitHub closes open keep-alive connections after 60s. Combined with the fact that repo-updater sends requests every 60s this sometimes lead to a request resulting in an EOF error because GitHub closed the connection we tried to reuse, while we were sending the request.

Fun fact: this didn't happen before because we were often using multiple requests to fetch repositories from GitHub, which means the time between requests was often less than 60s. Now that we're often sending a single request every 60s the chances are higher that we run into this.

To fix the problem, we don't rely on the http.DefaultClient and http.DefaultTransport, but instead use a custom client with a IdleConnTimeout of 30s. That gives us enough time to benefit from keep-alive connections when doing bursts of requests, but also reduces the chances of running into a connection being closed by the remote host.

Script to reproduce: https://gist.github.com/mrnugget/149331bd8fd27e4c49e47262174eb8d0

More information/context/GIF in the Slack thread: https://sourcegraph.slack.com/archives/CHPC7UX16/p1559629058001500

Test plan: run ./dev/launch.sh with an GitHub external service configured

Merge request reports

Loading