Skip to content

optimize graphqlbackend gitserver API usage to improve performance of admin repositories page

Warren Gifford requested to merge sg/multi-repo-info into master

Created by: slimsag

Prior to this change, for every repository in the connection resolver (which on e.g. the /site-admin/repositories page is often a lot), we would independently ask gitserver for information about the repository via its API. This was costly because it meant performing literally thousands of tiny sub-millisecond HTTP requests, individually JSON encoding them (which may create lots of GC pressure, too).

The solution here is simple: When we ask gitserver for info about thousands of repositories, do so using a single request instead of thousands.

On a dev instance with ~12k repositories, before:

01:37:35               frontend | took 3.96825239s
01:37:50               frontend | took 4.236735448s
01:38:01               frontend | took 5.147227239s
01:38:11               frontend | took 6.093235795s
01:38:33               frontend | took 10.372046862s
01:38:57               frontend | took 13.729139161s
01:39:51               frontend | took 14.654338045s
01:42:08               frontend | took 3.814422443s
01:42:19               frontend | took 4.560987174s
01:42:31               frontend | took 4.626022391s
01:42:40               frontend | took 4.8749788s
01:42:48               frontend | took 3.687233796s
01:42:55               frontend | took 3.738414004s
01:43:04               frontend | took 3.672425721s
01:43:14               frontend | took 3.682802146s

And after:

02:07:51               frontend | took 1.25605805s
02:07:58               frontend | took 1.441502375s
02:08:03               frontend | took 1.43875787s
02:08:12               frontend | took 1.410966593s
02:08:18               frontend | took 1.416104619s
02:08:26               frontend | took 1.382661535s
02:08:30               frontend | took 1.282073601s
02:08:34               frontend | took 1.635967947s
02:08:38               frontend | took 1.231089455s
02:08:42               frontend | took 1.230096699s
02:08:47               frontend | took 1.272247102s
02:08:51               frontend | took 1.267581576s
02:08:55               frontend | took 1.307818437s
02:08:59               frontend | took 1.272283421s
02:09:05               frontend | took 1.165448496s

I suspect the result may be even more pronounced on a larger instance, but haven't yet verified that.

Test plan: Covered by existing tests + new tests added (see diff).

Fixes #2779 (closed)

Merge request reports

Loading