Skip to content

stats: record search latencies

Administrator requested to merge rvt/log-search-latencies-final into master

Created by: rvantonder

Adds search latency logging. One high level concern: This will log latencies for every query for every user. Should we consider sampling n% of queries instead?

Implementation details for @sourcegraph/core-services:

I started off doing the recording in doResults, based on the search type. That function iterates over different search types (since we may perform a repo search, file search, and content search all in one query), and I initially tried to do fine-grained logging along each type. This was problematic for two big reasons: (1) The search suggestions interface/API calls doResults, meaning logging would by default take places per-character-typed for autocompletions, which we probably don't want and (2) it added complexity for having locks on each part of the goroutines, and also ignores the time taken post-process results (sort, etc), which we should care about in this number. So I got around these issues by shifting the logging to queries executed by Search. The first PR is here: #8866

doResults also by default computes file path matches and file content matches (e.g., in Zoekt, etc.). We can't easily disentangle these at the timing level. So: timing is associated to queries/search kinds by end-to-end behavior, not strictly by whether, e.g., we exclusively ran a search for repo versus file versus content. In other words, when a query like repo:foo file:bar baz is performed for literal search, we record the time as that for search.latencies.literal with the understanding that this implies the possibility of a file search also having taking place.

See inline comments for more details on handling type: field in queries.

Merge request reports