Skip to content

codeintel: Improve upsert performance

Warren Gifford requested to merge ef/additional-write-docs-updates into main

Created by: efritz

This PR updates the upsertRepositoryName, upsertLanguageName and upsertTags methods used by API docs. The latter query is the currently dominating query for load by total time in codeintel-db, and is almost 100% lock wait.

I believe that ON CONFLICT is quite expensive in this context and we are better to do a multi-stage tags update:

  • Write all tags to a temp table
  • Move only the tags over to the target table that don't conflict
  • Return logical ids of all tags in the temp table (inserted or existing)

We also do something similar for repository and langauge name. If lock contention largely goes away after this, I thikn we can probably put the repo and language name upserts back into the same txn.

Helps #21938

Merge request reports