Batch changes and specs with large numbers of changesets and diffstats can be slow
Created by: LawnGnome
I started investigating this last week, but I need to break off for now and come back to this later.
Basically, executing a batch spec that creates, say, 200+ changesets, each with diffstats around 200+ lines added, results in the GraphQL queries that drive the preview and batch change pages being slow, in the order of multiple seconds. This appears to correspond to increased database load.
Poking around with a tool to isolate each field in a GraphQL query (self promotion alert), the slowness mostly seems to come from two areas:
- Retrieving fields related to the
codeHost
. This is a fairly uncomfortable abstraction in Go over the lower level external service and repo tables, and is presumably missing some indices and/or retrieving more data than we need to answer the real questions we need to answer on these pages (which is basically "do you have webhooks enabled?"). - Calculating the total diffstat for the batch spec. This happens in the GraphQL layer by constructing all the changeset spec resolvers for the batch spec, then adding their diffstats together, presumably for permission reasons (since users should be unaware of diffstats for hidden changesets). We need to find a way to move that into the database if possible, I think. (Who knew I'd eventually pine for
authzQueryConds
?)
I have a partial branch for the second one, but I'm a bit stuck on the permission side of things, and haven't had a chance to loop back yet.